QCon London 2009: A few lessons learned

Last week I attended QCon London 2009, which was characteristically excellent — and I know my colleagues who attended thought the same. Most excitingly this year two of the Guardian development team were invited to give presentations on our most recent work building a large content management system for a very large website. I don’t mind admitting that I learnt one or two things from attending those talks; I also found the presentation on the BBC’s architecture fascinating. Here are some of the things I took away…

Mat Wall on The evolving Guardian.co.uk architecture

Mat’s presentation was structured around rebuilding guardian.co.uk over many months and dealing with the scalability issues that were presented at various stages. Some of the many insights were presented in a deliberately provocative way, such as this one:

Developers try to complicate things; architects simplify them.

I’m not entirely sure about the first part, but it’s certainly true that over-engineering is a much more common trait than under-engineering. The second part, though, is very important. It’s something I’ve always known, but never quite in that pithy way. It’s a useful phrase to tuck away for future reference.

And if you think Mat wasn’t feeling too good towards developers, don’t worry, because he also put up a slide saying

Developers are better than architects.

The message behind this is that it’s the developers who are working on the code each day — they know what’s possible and what not, and what’s reasonable and what’s not. So it’s important to trust them when they say they can or can’t do something.

Dirk-Willem van Gulik on Forging ahead – Scaling the BBC into Web/2.0

Dirk-Willem talked about the programme to re-engineer the BBC websites to enable greater scalability and dynamic content. In an organisation the size and complexity of the BBC you need to spend more time thinking about people than the technology. One of his first slides underlined this by presenting the seven layers of the OSI model (physical up to application) and then said

But most people forget there are two layers on top of that: level 8, the organisation, and level 9, its goals and objectives.

From that point on he kept referring to something being “an L8 problem” or “a level 9 issue”. It was a powerful reminder that technology work is about much, much more than technology.

Another great insight was how much they have chosen to use the simplest of standard internet protocols to join various layers and services within their network — even when those layers and services are organisation- and application-specific. This ties back to Mat’s point about an architect’s job being one of simplification.

Phil Wills on Rebuilding guardian.co.uk with DDD

Phil talked about the role domain driven design has played for us. He also pointed out various lessons that there were learnt along the way, such as the importance of value objects, and the fact that “database tables are not the model — I don’t know how many times I can say this.”

But it was only a day or two after his presentation that I was struck by a remarkable thing: Phil referred so many times to specific members of the team, and talked about contributions they had made to the software design process, even though the people he was talking about were generally not software developers. He put their pictures up on the screen, he named them, he told stories about how they had shaped key ideas. This was an important reminder to me that being close to our users and stakeholders on a day-to-day basis is incredibly important to the health of our software.

Walking away

I didn’t get to see as much at QCon as I’d like to have done, although I dare say most people will say that, even if they went to as many sessions as was physically possible. But what I did see was fascinating and thought-provoking, even when it came from people I work with every day.

The Open Platform: It’s thanks to individuals

The Open Platform trends on TwitterYesterday we launched the Guardian Open Platform, incorporating the Content API and the Data Store. A lot’s been said already about this — and more will be said — so I will add only a few words of acknowledgement. Many have pointed out (Tom Watson, Jeff Jarvis, ReadWriteWeb, Fast Company and others) that it’s a serious move by the company itself. So I really want to spend a moment to reflect on what that really means.

It can really be best captured by the questions asked by the crowd at the launch event: What if other news organisations want to reuse your journalism? Will the ad network really work? Do the numbers add up? These and other questions were asked at the time and were asked inside the organisation in the months leading up to the launch. The questions span journalism, advertising, finance, technology, marketing, strategy, legal and, of course, technology. These were addressed by real people in real departments who sat down together and worked together. They worked to shape the Open Platform to make sure that it is absolutely right thing to do, from so many points of view.

Matt McAlister brought all the strands together, and must be credited for nurturing and co-ordinating such a major move. But there’s also huge credit to be given to so many individuals within so many Guardian departments who made sure it was a unified, coherent and successful whole. This is a major corporate achievement, but in so many ways a corporation is just another name for individuals working together.

How news organisations can harness startup-power

These are some thoughts on how news organisations (and specifically those which have a background as newspaper companies) might create new opportunities for themselves. In particular I want to focus on why and how those opportunities might come from outside the organisations themselves.

The thinking behind this piece has sprung from two articles: the first is by David Carr, in the New York Times, in which he seeks an iTunes-like saviour for the news industry, and the second is Jemima Kiss’s thought-provoking response on guardian.co.uk.

What I’m going to present isn’t new, by the way — it’s about being open and encouraging innovation. But what I do hope to add to conversation is the idea that we don’t have to have all the answers ourselves, and that we can — and should — encourage others to work with us. I’m also adding some practical actions to take.

Here’s the structure of what follows:

Background reading

David Carr’s article in the New York Times is headlined “Let’s invent an iTunes for news” but that’s rather misleading. He’s actually calling for a way to charge for news content, pointing out that Apple has done this very effectively through the iTunes Store (although he never uses that phrase — he conflates iTunes, the desktop software launched in 2001, with the iTunes Store, the online shop launched two years later and accessed via that iTunes software). He does refer to one or two gadgets which could be an iPod for news (“Now all we need is a business model”), but the underlying point of the article is that content needs to be charged for one way or another.

Jemima Kiss picks up on the gadgets theme and dispenses with it quite quickly — “I think newspapers are wrong to put too much effort into pursuing degradable devices with a very limited potential audience.” Instead she says news organisations can find salvation within, and urges them to “start thinking like startups”.

Both David and Jemima refer to the “newspaper” industry, which is understandable but puts us in a limiting mindset. The companies they are referring to have gone to great lengths to diversify themselves into other media (most notably various digital media) with some genuine successes and we should embrace that. I’m going to refer to it as the news industry, although that’s arguably too broad. We’re all referring to the same thing: the-industry-which-used-to-be-the-newspaper-industry-but-now-reaches-across-many-media-including-paper industry.

A brief look at the music and news industries

The music and news industries do have similarities, which is why David has compared them, Jemima has followed, and others have done so, too. Both are facing declining sales, and many people in each industry hold the rise of the internet as partly or wholly to blame. In both cases their core product is content, and they are both facing the atomisation of this product — from albums to tracks, and from newspapers to articles — as the internet becomes more ubiquitous.

But they differ in the way they have faced the growth of digital media.

Figures from the RIAA show the US music industry in near-consistent decline since 1999, after years of near-consistent growth before. This 1999 point is more-or-less when digital music took off and sharing became easier on the web. The music industry tried to lock down its customers’ new-found freedoms, but unsuccessfully. Only when the iTunes Store arrived did some kind of calm prevail.

In the news industry DMGT’s annual report of 2007 shows a steady decline in UK circulation since 1994/5. From the US the Pew Research Center shows a steady decline in total sales of daily newspapers going back to 1990. In both cases the charts don’t go back further. But unlike the music industry we cannot blame this on the rise of the internet. These downturns started before what-was-then-newspaper content was widely available digitally.

So the music industry might point to the internet as the source of its troubles, but for the news industry the internet was just something that was going on at the same time. Furthermore each industry has met the challenge of the internet differently: the music industry has tried hard to prevent copying of its content, while the news industry has made some efforts in that direction, but has largely let its content go in the hope of rewards elsewhere.

As a consequence the music industry has an uneasy relationship with the internet, and still finds it difficult to get on there. By contrast, in the internet the news industry has another means to market, one in which it hasn’t made enemies. The internet could be the source of the innovation and upturn that it needs.

So if this innovation is to be found, where might the news industry look?

Why not look internally?

Why can’t news organisations look internally to find a revitalisation for their business models? Well, of course they can, and should, and do. Looking internally and externally for new opportunities are not mutually exclusive, and to not look internally would be a dereliction of duty. That’s why most commercial companies have a a special team dedicated to this — Business Development — aside from the responsibility of each of its employees every day to push the organisation forward.

But seeking solutions internally won’t maximise its dose of innovation. That’s because people inside organistions are most likely to see opportunities through the prism of their work in the organisation: we people who work in large organisations spend 8, 10, 12 hours of every day in these places, and the problems we’re driven to solve are generally those that relate to our daily work, and the particular people and processes we encounter there. Even the Business Development team is tasked with creating innovative opportunities that fit in with the company’s known strengths and current structure, and what it can practically achieve today — radical ideas beyond this are going to be difficult to handle. In general a company will value people who make the company better; it will not be very interested in people who have a great idea that does not easily fit with what it does.

Additionally, I think the rallying cry of “think like a startup” — as positive as it is — is particularly difficult to act on in a large organisation. That’s primarily because large organisations attract people who want to work in large organisations. They are subeditors who thrive on subediting and don’t want to have to produce business models; they are A&R people who love nurturing new bands and don’t want to sell sponsorship deals; they are marketing people who love planning marketing campaigns and don’t want to negotiate office leasing. They are not generally people who thrive on doing it all themselves, which is the driving force of a startup founder. They are people who thrive on achieving wonderful things by being a specialised part of a much greater whole.

Finally, business success stories are themselves unusual, so casting a net into the internal talent pool of one particular company is unnecessarily limiting. By all means we should look to home to find a success story, but we shouldn’t only do that.

To repeat, none of this is to say that innovation cannot be found internally, but we shouldn’t seek it there exclusively.

Let’s briefly compare internal and external success stories…

An example of an internally-generated idea

Jemima uses the example of the technology blog AllThingsD.com as an internally-generated idea, so let’s look at that in more detail.

The site describes itself as “the online extension of the prestigious D: All Things Digital conference” which started in 2003. Its “creators and executive producers” Walt Mossberg and Kara Swisher drove that for four years before launching the blog in April 2007 — or rather, before being able to launch the blog in April 2007, because the site is under the umbrella of the Wall Street Journal.

In ordinary circumstances such move would have been detrimental to WSJ since AllThingsD.com seems to be taking support and infrastructure from the bigger organisation whenever it wants, and yet retaining independence (on stories, production processes and technologies) when the ways of WSJ don’t suit it. WSJ would only have allowed this if there was some clear upside for itself, and of course it’s the conference, which we can readily expect to be a real moneyspinner. Walt and Kara spent four years proving they had a viable revenue stream before they got the support to expand into their own independent blog.

The conference-extended-into-a-website is an internally-generated line of business from WSJ and its parent, Dow Jones, and conference organisation is very much part of their daily business. Dow Jones organises quite a lot of conferences throughout any one year — on finance, economics, and environmental issues among others — and therefore will have a dedicated team of conference organisers to bring each one together. Guardian News & Media has a similar line of business.

None of this is to talk down the conference or the website, which are beacons in the tech industry. But it is an attempt to show their lineage as an outcome of Dow Jones’ daily operational machinery, and the key revenue-generating part of that machinery (conference organisation) is a well-established part of the company. The conference and website will be highly valued by Dow Jones and WSJ, and the kind of business unit that any company would be grateful for. But I suspect that if this kind of business unit was going to be the key innovation that reversed the news industry’s fortunes then we would have discovered that by now.

Now let’s take a look at…

An example of an externally-generated idea

David looks to the iTunes Store as an example of an industry saviour, so let’s consider that.

The iTunes Store revolutionised and arguably saved — or at least slowed the death of — the music industry, yet it was an innovation that came from Steve Jobs, a man outside the industry. It came at a time when music sales were first declining and it’s not as if the industry didn’t have its own ideas to deal with the rise of digital music — but those internally-generated ideas turned out to be quite misjudged.

The iTunes Store gave consumers more freedom over their ownship and use of music. It was never likely to come from inside the industry because, as David pointedly says, “Mr. Jobs saw music as […] an ancillary software business to generate sales of the iPods and iPhones. That’s not a perspective that flattered people in the music business”. We can readily imagine what reaction might have been received by a hapless music company executive if they’d have originally suggested the idea to their boss.

It took an outsider to devise the most workable innovation.

How to harness startup-power

Individual news organisations will continue to discover and exploit profitable seams from within their own structures. But they also need to be able to identify and exploit innovation from outside.

(By the way, I’ve called external innovation “startup-power” in the title here and above mostly in an attempt to be eye-catching. However it does largely capture the idea, and it does chime usefully with Jemima’s call to arms, so it’s not entirely superficial.)

For a long time the music industry tried to fight consumers’ desire to have more freedom with their digital music, and it did this by trying to create new business models entirely on its own terms. It only started successfully getting to grips with digital music when an outsider entered the game, and that outsider is taking a pretty healthy cut of the revenue. The music industry might have been able to dictate better terms if it had more to bring to the table earlier.

The news industry needs to put itself in a better position than the music industry did. I’d say there are three steps which will enable this to happen.

  1. Create opportunities for external innovators;
  2. Recognise and make contact with those innovators when they come along;
  3. Be in a position to best exploit the innovations.

Those are pretty obvious, but they still need careful consideration, because there’s still room to handle them poorly. Let’s take a look…

Harnessing innovation 1: Create opportunities

Creating opportunities for innovators means presenting what you do in multiple ways of accessibility. For news organisations the obvious example is RSS, which in some sense is less consumer-friendly than nicely designed web pages, but makes the raw material (articles) more accessible, and hence provides more opportunities to anyone else building their digital venture.

But large news organisations tend to provide only summaries in RSS, which parallels music companies’ early paranoia of digital music piracy and prevents external innovation. This puts the content out there, but only a bit — the news organisation is still holding it tight to its chest, still not trusting anyone external, and not giving anyone else the chance to innovate.

RSS is not the only means to create opportunities. The New York Times has released a number of APIs which are simply yet more raw materials for external innovators. Their terms of use prevent commercial use without prior agreement, which means that anyone can experiment and those who have a commercial proposition can work out an agreement with the Times.

Finally, let’s not think that content is the only thing news organisations do. They also provide advertising services (so there’s scope to open that, too) and much else besides. Perhaps everything an organisation does has scope for opening itself up to innovation if just enough imagination is applied.

Harnessing innovation 2: Talk to the innovators

Recognising and making contact with the innovators is a fairly obvious next step, but it’s rather different in the online world from the offline world. Experimentation is easier with digital assets than with physical ones: it can happen anywhere in the world, and due to the way digital media can be copied and distributed it can happen largely without the originator knowing.

Allowing the use of one’s assets or services without the knowledge of the originating organisation might seem daunting, and akin to giving away the assets or services. An obvious action would be to allow registration ahead of use, which means a channel of dialogue is opened immediately. The Times has used this approach with its APIs. But that option has its drawbacks, mainly that it creates a barrier to entry. Given that a small proportion of experiments will lead to something useful there’s a strong argument to remove that barrier, however small it may be, so as to maximise successes.

The alternative to enforcing dialogue is to make it so enticing that innovators will want to enter into it voluntarily. This means creating a forum or other environment that is sufficiently useful for innovators to make the active choice to speak to you. And it also means making yourself available in other ways — at other people’s conferences, seminars, and so on.

The dialogue of course has another direct benefit, which is that you get to understand how people really are innovating with what you offer. It provides you with an opportunity to change and improve what you offer so as to generate more opportunities for more innovators.

Harnessing innovation 3: Exploit innovation

In theory the third step is trivial — after creating the opportunities and speaking to the people who are using them it should be easy for both parties to get to a win-win on making something of lasting value.

However, when it comes to a large news organisation interacting with a very small organisation, such as a startup, the large organisation can easily act far too slowly. It will have committees and contracts and steering groups to go through. But startups worry about cashflow and have sufficiently few staff to be able to deploy them quickly if other opportunities present themselves in the interim. If the large news organisation cannot become fleet of foot then it stands to miss those opportunties as the innovators lose patience and pursue other interests.

This can be a difficult culture change for large organisations. Unfortunately it’s rarely possible to know just how valuable a lost opportunity might otherwise have been. But if a large company were to consistently fail to exploit external innovation then it might be a death by a thousand cuts.

Wrapping up

News organisations already have internal mechanisms to create and exploit ideas so I’ve spent some time trying to set out why internal innovation shouldn’t be considered to the exclusion of external innovation, and outlined a little of how external innovation might be developed.

I think this idea of developing external innovation might be particular to the digital times we live in, and not just because many might think we’re running out of our own ideas. Instead I think it’s a consequence our newly-networked world, where so much data and services are readily available globally, and so many ideas can be tried and discarded in such a short space of time. Previously the limitations of geography and physical objects meant businesses had to focus almost entirely on exploiting known opportunities for themselves. But in a digital world it’s easier than ever to create opportunities for others, and the rewards of creating those opportunties for external innovators is more immediate.

That historical economic perspective, however, is something that others would be better placed to evaluate. For now, those of us in news organisations probably need to focus on the more immediate concern of making innovation happen. Looking externally for ideas is not about waiting for someone to come by on a white horse and presenting a road to a happy ending; it’s not about hoping we get lucky. It’s up to us — we make our own luck.

An ABC of R2: Z is for zones

…which are a top navigation level of guardian.co.uk. However the navigational structure was designed to be quite fluid, and the concept of a zone is really more of an internal reference point than a phrase that’s intended to be used by our site’s users.

The ebb and flow of the online media marketplace meant that while we were working on R2 Times Online in the UK launched a redesign of their own. At the time I was interested to read an interview with their Information Architect who said that they had constructed a three-level navigation hierarchy. For guardian.co.uk we had considered and rejected implementing such an explicit scheme for three primary reasons.

First we recognise that the breadth of our content wasn’t equal across all subjects — we have many more journalists working on news stories than travel stories, for example, so a second-level of navigation under news might have a couple of dozen subjects, which is impossible to display conveniently on a navbar. Second not all subjects break down neatly into three levels — some topics have more depth and complexity than others. Third, Guardian values mean that certain subjects need to be given more prominence than a formal encyclopaedic breakdown allows — for example, in any formal scheme environmental issues would be part of the news or the science section, but the reality is that our planet’s environmental state is far too important to relegate to anything but the highest level.

So R2 was designed to allow fluidity and flexibility in our navigation, and we created words to describe its different aspects. A zone is one of the top-level categories that appear along the top navigation bar: News, Sport, Comment, Culture, etc. At the bottom level of categorisation are very specific subjects such as interest rates, war crimes, Kate Moss and thousands of others. If you’re reading about one of these subjects it doesn’t matter how many levels deep you might be as long as there’s some clear signposting to find your way to your next point of interest. The navigation should give you some kind of orientation, and importantly act as a guide to jumping off to similar subjects or to begin a new journey elsewhere entirely. But exposing any internal classification system is less important. R2 has given us a system that separates the visual navigation from the way content is organised internally.

An ABC of R2: Y is for YAGNI

…which stands for “you ain’t gonna need it” and is an important principle of Agile development, with strong benefits for the business.

The basis for YAGNI stems from a failure common in many software development projects: that when a developer creates a component of a system they tend to give it more flexibility than is immediately necessary, so that it can be reused in more contexts and provide more value. But the result in practice is almost always over-engineering: the component costs more to develop, the suspected additional use rarely materialises, when it does materialise it has requirements that don’t quite match the actual implementation, and the component as a whole is more brittle, less comprehensible to successive developers and more difficult to maintain.

By contrast YAGNI says “you ain’t gonna need it”: develop the absolute minimum to get it working for the immediate problem, and extend it later only if another specific needs arises. This works in Agile development because there is a wider supporting structure which enables changes to be made with minimal cost.

An excellent example of YAGNI in action was our implementation of content workflow. There are lots of things you could do with workflow. You can lock content to ensure that only one person can edit it at a time; you can branch content so that different versions can be worked on simultaneously; one of our senior stakeholders was keen on allowing numbered versions which she had seen elsewhere, allowing individual changes to be tracked and traced, and even allowing someone to switch back to a previous numbered version — but she did say that the feature was almost never used.

What we did in practice was a minimal workflow with minimal cost: we built in the ability to have content as draft and live, and very little more. That was sufficient for the staff who we knew would be using it. It seems like we could have taken the opportunity to build more, in case more people with more needs would use it. But you know what they say — you ain’t gonna need it. In the months following that very early stage in the project the company changed a lot, and among other things this included much more integrated working between editorial teams who were previously designated “web” or “print”… these days that distinction has diminished. The technological impact is that many editorial staff are using an integrated front-end, sitting outside the web system, and that’s where the workflow is handled now.

The industry-wide drive for integrated publishing changed our working environment; if we’d have developed elaborate workflow in the web system it would have been wasted effort, and a large wasted cost for the company.

An ABC of R2: X is for XL

…which was a size of problem we noted but wouldn’t tackle.

When we estimated the work for R2 up front we used t-shirt sizes for each feature: S, M, L and XL. The largest single task the team would tackle was an L, which was the equivalent of five days’ work. We felt this was a good maximum for two reasons: first, it delivered something of value within a reasonable space of time and what should have been one iteration; second, if we could imagine developing it within five days then it was probably sufficiently understood, whereas anything bigger risked being too complex to estimate without a lot more thought.

However, during the time-limited initial planning period we were required to estimate things that needed more analysis than we had time for. These features we labelled XL and gave a nominal equivalent size to: 10 days’ work.

However, we would never actually start a task that was labelled XL. We would always break it down before the event into smaller parts. The aim was that although individual estimates might be over or under, on average they should balance out and the total size of the project should remain constant.

Of course, for XL tasks it was quite possible that a more thoughtful estimate would produce a total that was far too big to be able to be balanced out by other tasks. In these cases we would have to reconsider our options and think about dealing with the problem in a different way — most likely by having a less comprehensive solution. However that didn’t tend to be too much of a problem. In many cases by the time we got to the XL task our stakeholders’ needs and priorities had changed so much that there were a lot of new options and directions that people wanted to explore, and ones that they would not have thought of in those very early planning stages.

This is an example of why planning was important, but very detailed planning would actually have been quite wasteful.

An ABC of R2: W is for Wiimbledon

…which was a semi-regular event of Wii tennis in the office, but a very useful part of our R2 work, too.

Each launch required a small army of technologists to be on-hand: to run the various scripts, check the results, and deal with any problems that might arise. We needed to arrange these teams carefully because launches happened overnight, so we’d need an overnight team and another team in early the next day to pick up any remaining issues.

All this was fine, but after a while someone realised we’d missed a trick. I think he actually wanted to be part of the overnight team (it’s always exciting to see these thngs go live) but wasn’t actually on it. So he arranged with others to bring in some games consoles, and wired one up to a big screen, the others to projectors. It was a big draw, and a great way to have an extra group of techies staying late in case the need arose.

Still, we couldn’t be cavalier with this. For example, we made sure the gamers were located far from the launch team — the launch team had a serious job to do and didn’t need distraction. But that didn’t prevent the benefits: every so often a gamer would slip out of Guitar Hero and wander over to the launch team to check up on progress, offering some advice and support if necessary. One time a critical SQL script was running worryingly slowly and a call came through to see if someone could contribute to the investigation; a couple sat down at a machine a few feet from the Mario Kart players, traced the problem, and suggested a change to the script which was agreed and did the job. That night it made the difference between “go” and “abort”.

If there’s a general lesson to be drawn from this I don’t think it’s to keep a Wii console in the stationery cupboard along with the paperclips and envelopes. But exploiting opportunities that are specific your particular situation is probably a good thing to do, even if they aren’t enshrined in official company policy.

An ABC of R2: V is for video content

…which changed in the way we thought about it (and implemented it) a long time after it was released.

Originally any piece of content was designated as being of exactly one type: it was either an article, a video, a competition, and so on. To take the video content as an example, putting a video onto the site involves opening up a video content editor in the CMS, referencing the appropriate asset in the Brightcove system, and adding some video-specific metadata such as its source and preferred display size. Clearly you aren’t going to confuse that with writing an article, which is why it made sense to say that each piece of content was of exactly one type, no more and no less.

But one day one of the editorial staff made an unusual request: she had an article and wanted us to add a checkbox in the article editor marked “Display on video page”. This puzzled the tech team initially — if it was an article then people wanting videos wouldn’t want it to appear on the video page. Also, we were wary about adding a feature which would complicate the article editor but be used by few people. We quickly found the good reason for her request: she was creating an article with an embedded YouTube video, and the purpose of the content really was to showcase the video. Anyone looking for videos on the site would want to find the content, so it should appear on the video page, but the video should remain outside the Guardian CMS, so the video content editor wasn’t appropriate.

We realised this was a general need: content could fall into several categories at once and editorial staff should be able to flag it as such. The solution, then, was not to add a checkbox marked “Display on video page” because that wouldn’t solve the general problem. Instead we took an existing “labelling” facility present in all the content editors, and we extended it allow content-type labels to be added. Of course, we also needed to ensure the system responded to these new labels, but since it was an extension of an existing feature it was a more natural and reliable solution than we might otherwise have implemented.

The end result therefore didn’t introduce any complexity for end-users — no extra buttons or checkboxes — but it did give them the power to do what they wanted.

This is a good example both of how our understanding of apparently-basic questions (“what is video?”) can change with experience, and of how understanding the source of a requirement can lead to better solutions.

An ABC of R2: U is for URLs

…whose structure we worked hard to get right as part of the project. This was an important part of weaving ourselves into the fabric of the web: to ensure our referencing system had a useful structure to those outside our organisation.

Previously our URLs looked like this:

http://politics.guardian.co.uk/tonyblair/story/0,,2075005,00.html

today the same piece of content is referenced like this:

http://www.guardian.co.uk/politics/2007/may/10/tonyblair.labour3

There are two obvious changes here. First, the obscure numbers have gone — a legacy of our old Vignette StoryServer software, and utterly useless to anyone who happens to be a human being. Second, we’ve included the date in a friendly format, to provide some sense of orientation, and particularly useful if you happen to be looking at several URLs at a time.

There are other changes, too. The “politics” reference has come out of the domain name and into the path, to reflect the fact that all our content is part of the guardian.co.uk site. A couple of keywords also appear in the URL, partly to convey what the content is about, and partly to distinguish it from any other content that appeared in the same section on the same day. As it happens, quite a lot was written about Tony Blair and Labour on that day, so the “3” adds a further unique identifier.

In some ways URL structure is an obscure topic, but getting it right opens up access to more of our content in more ways to more people. Further examples have been given by Matt McAlister in his discussion of the RSS feeds.

An ABC of R2: T is for timeline

…which looked like this:

  • October 2005: Start detailed planning of Travel section. Involved about 12 people from Guardian and ThoughtWorks. Much preparation needs to take place before work can begin.
  • February 2006: Start building the Travel section, initially focusing on the servers, development environment and build pipeline.
  • May 2006: Launch first visible item on Travel section, a keyword component.
  • November 2006: Launch entire Travel section. From a technical standpoint it was then the biggest single launch in the website’s history, and one of the smoothest then known.
  • January 2007: Start planning phase two. Involved around 20 people fulltime for six weeks, but sufficient clarity was achieved after three weeks to start one stream of work then.
  • May 2007: Launch new guardian.co.uk home page. Within hours of the launch, Tony Blair obligingly announces his resignation as prime minister, allowing us to make full use of our flexible templates as the story develops throughout the day.
  • July 2007: Launch Science, Technology and Environment sections. Included a new feature: series. This allows special navigation back and forth through items in a series such as Bad Science or Nature Notes.
  • August 2007: Launch integrated video. Includes reskinned Brightcove player with dynamically-inserted pre-roll advertising.
  • October 2007: Launch sections for Business, Society, Media, Money and Guardian America.
  • February 2008: Launch Observer, News, and Politics sections, plus integrated audio.
  • April 2008: Launch Sport and Football sections. Includes live scores and fixtures, and match facts among other features.
  • June 2008: Launch Comment is Free. Includes community features from Pluck.
  • August 2008: Launch of Culture, Education and Life & Style sections, plus 14 blogs moving over from the old Movable Type system.
  • September 2008. Launch the remaining 21 blogs onto the new platform. Some of these blogs didn’t go in August due to the key part they played in August news events, and some because they were awaiting additional features such as remote working tools.