I was speaking to a group about making agile development work in practice in a corporate environment, and as part of this I had said that committing to estimated stories was very important for successful adoption of agile development. That’s mainly because the people who are paying you deserve to know what they’re getting for their money. It’s also part of building trust: by committing to stories you’re also demonstrating commitment to the business — you’re putting your reputation the line, it gives people confidence to know you’ve got skin in the game.

But talking afterwards to an experienced developer and coach I changed my mind about the details. He said he found it was often useful to commit to delivering stories-as-outcomes rather than stories-as-features.

I had always seen developers estimating user stories as collections of features, but estimating stories as outcomes might be more productive. It should be equally valid, if you can have a meaningful conversation with your stakeholders about outcomes (rather than “I want this because I said so”). It also helps everyone raise their eyes as to what the work is about: software has to be for a purpose.

Of course, software is still, at some level, a collection of features. If I have some doubts about this approach it’s that attention to detail in features can make or break a product, so those low level things can’t be overlooked. I’d be interested in others’ views on conflict. But at the very least the idea to commit to outcomes rather than features does demonstrate that there are plenty of ways to help people work together towards the bigger picture.

On the first page of Facebook’s prospectus for its sale of stock to the public, it pegs the number of its “monthly active users” at a whopping 845 million people. The social networking site arrives at an even more astounding number when it comes to “daily active users”: 483 million people. [But] Those eye-popping numbers should have an asterisk next to them. [...] According to the company, a user is considered active if he or she “took an action to share content or activity with his or her Facebook friends or connections via a third-party Web site that is integrated with Facebook.”

The default view of us digitalistas is “So what? Of course active Facebook users aren’t always on the website. They’re clicking the Like button, or logging on to a site with their Facebook account. Get with the programme, daddio.” Jeff Jarvis also cast doubt on the way Andrew spun his piece, when he tweeted:

Read @andrewrsorkin from bottom up & you may conclude Facebook is smart to be distributed & gain data from users

But it turned out Andrew wasn’t manufacturing scandal — he went on to quote equities expert Barry Ritholtz:

Think of what this means in terms of monetizing their “daily users.” If they click a like button but do not go to Facebook that day, they cannot be marketed to, they do not see any advertising, they cannot be sold any goods or services.

Suddenly it made sense. And I was annoyed that I’d been caught out at my own game. Metrics for the sake of it are pointless. Asking why you need those metrics makes all the difference.

To me, discounting off-site Facebook users is silly — those people still contribute to the size of the service. But investors actually want something tangible, not just numbers for academic satisfaction. If you want to make money out of Facebook how you count your users suddenly really matters… and you’d come up with a different method.

Similarly, I was speaking to someone the other day who wants to know the value of the stock in their warehouses. That’s something they should know, but getting to the answer requires first asking the question “Why do you want to know that?” This will determine whether they count items already sold and waiting to be sent out, items due to be returned, items broken or withdrawn and therefore not saleable, items in their agents’ warehouses, items already bought but not yet received, whether the number has to be up to the minute or up to the month, and so on.

Asking “why” is also at the heart of the lean startup approach. Asking why you want something determines whether you should prioritise it or not, which is essential when you’re trying to minimise your spend. Two startups in the same space will almost certainly turn out different if they keep asking themselves “why” — they’re likely to find they’re focusing on slightly different users, with slightly different needs, and will therefore become very different entities.

Asking why changes (almost) everything.

On the one hand innovation is about change. On the other, you need a reliable — and therefore pretty fixed — mechanism to ensure that change happens. That means you can only successfully innovate in a pre-defined way.

Innovation happens at many levels. Take the low level. A software team might have successfully introduced retrospectives, changing every fortnight the way they work, and they can do this even if the rest of the organisation is paralysed in its ways.

Take the middle level. A food company may have great market researchers constantly able to spot new opportunities and introduce exciting new brands… but they’re still a company producing different packages of food.

And take the high level: Google has long had a particular way of generating new ideas, which is about developers having independence to work on what they feel is worthwhile. But that makes it difficult when the CEO wants to take a directive approach to shift the focus of the entire company — he ends up having to issue ultimatums like “This is the path we’re headed down [...] If you don’t get that, then you should probably work somewhere else.”

I don’t think this is quite “Innovator’s dilemma” stuff. That’s more about the difficulty of innovating. This is saying that continual innovation can only happen successfully in a particular way for a particular company, which means that there will always be limits and bounds on how it innovates, and certain kinds of innovation will always be denied it — or at least there will be without changing the innovation mechanism.

Of course, there may be rare exceptions. I’m reminded that 3M has an explicit remit for 30% of its revenue to come from products introduced within the last five years. Nevertheless, for the majority of those who do innovate successfully, it’s innovation within limits of the process.

Here’s the test question: You run a company that creates software in high level languages. You have a collection of developers who want to work for you. How do you sort the ones who are most suitable from the ones who are least suitable?

The answer is to query each one in turn using questions that most closely match the actual work they’ll be doing.

If you need them to design domain models then give them a toy domain and ask them to model it. If they need to work particularly closely with other developers then get them coding next to one of your current team. And if, for some perverse reason, company policy bans the use of software libraries and they have to implement every basic software function from scratch every time, then it’s a good idea to ask them to write a sorting algorithm.

But if that last situation is not true of your company, then the correct answer to “Show me how to sort an array x of integers” is “x.sort()”. Or some variant according to your local tongue.

If you get that question wrong, then unfortunately you fail, and miss out on some really excellent hires.

One major problem with targets comes about when they are different from the core thing the company is doing. For example, the number of features delivered by a software team may be a rough guide to their productivity, but is the team’s productivity really helping the customers get their widgets? As soon as you target the team members on this figure they will aim towards it to the exclusion of the end goal. You also end up creating or reinforcing a silo — suddenly the team is no longer interested in helping customers get widgets, it’s a team whose purpose is productivity, divorced from anything else.

A key phrase to watch out for is “is a proxy for”. For example, in a call centre: “Call duration is a proxy for resolved problems”. As soon as people are motivated towards a proxy the real target is forgotten.

At Øredev 2010 John Seddon gave an example of a service team whose managers thought they were meeting 100% of their targets, closing all their cases in 25 days or less. But the real work was not about closing cases; it was about resolving their customers’ problems. When they looked at the real activity that went on, they found that a single customer’s problem might be broken into four or more cases precisely because the staff were motivated by a 25 day case-closure time. From the customers’ point of view their problems were really taking 50 or 100 or 150 days to resolve. False targets obscured the reality and let an failing system remain in place. John says (at the 33:34 point):

If you hold people to account with arbitrary measures they learn to do anything to meet the arbitrary measures, and that is not the same as serving the customer.

There are other problems with performance targets, but that’s enough to be going on with.

When implementing a change of working a while back in my development team my boss of the time said, “Well, okay, but I want you to show me that your changes are making a difference”. What’s the metric for better software? I knew all about the dangers of measuring things like lines of code — that wouldn’t do anyone any good.

So I opted for something low-tech. I asked people. I asked team members and wider stakeholders once fortnight: “How are we doing?” I broke it down into ten questions, varying depending on the role of person and over time I could chart our progress. We went from around 30% satisfaction to 80% in four months and then plateaued. The plateau gnawed away at me, but by that time interest had moved on — the team was doing a great job and there were more pressing problems elsewhere.

I subsequently spoke to a programme manager who used a similar, but simpler, technique. He asked: “How would you score us out of five this month?” And then the follow-up question, if it wasn’t full marks… “What would we need to do to make it five?”

I much prefer this version, which is simpler, more direct, and there’s more clarity on what to do about the results. It reminds me of the guerilla approach to lean product development: putting a mock-up in front of someone in Starbucks and (eventually) asking “Would you buy this product? What would it take for you to buy it?”

I’ve used lots of other metrics for different things relating to performance, output and productivity: bugs resolved per week; lines of code per class, and so on. But I don’t think I’ve ever found any for this kind of thing which are so simple and connect the work and the output so effectively.

As part of a project some new requirements came in. I had been concerned that part of the system under development was a little complex, but not overly concerned, as it worked and had comprehensive automated tests.

But the new requirements changed that. They concerned a subsystem which took three inputs and processed them. The problem that the subsystem was trying to solve was roughly stated as follows:

We have an input stream which needs to be combined with two other input streams. But if some of the combining can’t take place because the input streams have gaps then we have to fill in the gaps from other sources. And then we can output the result.

If that seems a bit vague, it’s because it is: I hadn’t paid much attention to expressing the logic clearly.

The new requirement, meanwhile, was to add a fourth input. Since we were using a visual tool I can present the “as is” software design along with the “to be” that was the likely outcome of the requested changes:

The threatened “to be” state would have been excessively complex. Two things concerned me greatly. First was that I couldn’t keep all the “to be” logic in my head at one time, so didn’t have confidence that it was right. And second was that — as a guiding rule — software should be exactly as complex as the problem it attempts to address, and no more so… and that was not reflected here. My gut feeling was that the problem was simple, and adding one more input should have added linear complexity, but it felt like the complexity was growing exponentially. This was compounded by the fact that it was quite apparent there might soon be a fifth, sixth, or even seventh input.

The solution was to take a step back, express the logic of the problem clearly, and redesign the subsystem to reflect that. The problem, once clarified, became this:

We have a number of input streams which are merged. For each missing element of the merged stream we take action to fill it in. And then we output the result.

The refactored “as is” and the eventual “to be” logic became much simpler:

I hope you agree with me that the new flow is much clearer, and additional inputs add only linear complexity.

In terms of software development the sequence of steps was: (1) Make sure all the tests pass, (2) refactor the subsystem to reflect the new logic, (3) make sure all the tests still pass, (4) add the fourth input with appropriate tests.

For me this experience demonstrates a few things:

1. If software internals aren’t continually kept clean and of good quality then complexity increases excessively and progress slows accordingly. Or to put it another way, technical debt is very well named: the longer you neglect the debt the more interest you pay.
2. “Software should be exactly as complex as the problem it’s addressing” is a very good driver for reducing complexity.
3. Automated tests save the day again! The refactoring above would have been quite daunting and unreliable without them. But instead the worst thing about the operation was that it was a little time-consuming. It was not risky, it was not stressful, and it was not really that difficult.
4. Visual software development tools sure do expose messy software design for what it is.

I was thinking about this after reading Mark Foden’s excellent response to Sir Gus O’Donnell’s recent Telegraph article. If you’ve not read it, you should (Mark’s piece, but also the former Cabinet Secretary’s). The structure of Mark’s response is: (1) Sir Gus O’Donnell is another voice saying government needs to take more risks; (2) the truth is government needs to take fewer risks, which it can do through (3) trying things out in a small way and then learning and adapting, over and over again.

In fact this is a fascinating example of “what they say, and what we hear”. Here is a key part of Mark’s setup:

I infer a model of thinking that goes: The old ideas aren’t working any more… so we need to be innovative and adopt radical new ones… because the ideas are new we don’t know what will happen so this is risky… but we have no choice so that’s OK… if things go wrong we must be mature and learn from the failure rather than throw stones… next time we will know better.

When I looked for those words in the Telegraph article I couldn’t find them. And then I realised why: I was caught out by “I infer…”. Sir Gus didn’t say those words, but Mark heard them (or, more correctly, heard an echo of them).

Similarly there are many times when I’ve heard CEOs and others saying “we must take more risks”. And in retrospect, I think they didn’t actually say that — at least not in the way most of us understand it.

When we hear business leaders talking about taking more risks what they are actually saying is: we must move out of our comfort zone. More broadly they are saying: we have become stuck in our old ways, and become too comfortable, while the world around us has changed; the old certainties are no longer with us so we must work without the old systems that grew out of those old certainties.

And of course, working without old certainties feels like taking more risks. Indeed, less certainty is one key part of risk. (The other is undesirable consequences.)

If you gave pressured CEOs a hard choice between increased risks and a change in the way things get done, I think most of them would opt for the latter.

So many organisations are being disrupted — by digital startups, the internet generally, the economic downturn… The civil service is just one of these. Most leaders facing these problems do not want to increase the risk profile of their organisation. What they want is to sweep away their organisation’s old systems and embrace new ones. That might seem risky to some, but it doesn’t need to be.

To use an analogy: If you meet someone who will only ever cross roads at a pelican crossing then you might see them as someone who is stuck using old, time-consuming methods. They might see you a maverick risk-taker. But if you teach them the Green Cross Code they may learn that there are other ways of achieving their ends without taking undue risks.

Gus O’Donnell did indeed use the phrase “take more risks”, but he used it in the context of breaking away from cumbersome systems of red tape, and so I think he meant it more in the sense of moving out of a comfort zone. He also referenced one of the two keys to success that Mark mentioned: learning from experience. So I don’t think he and Mark are very far apart — if indeed they differ on anything much.

Similarly staff who hear their bosses say “take more risks” might want to reconsider what’s actually being said. Usually it will be: let’s do things differently.

Here goes…

Question 1: “Zeebox is now live” said PaidContent in October. “Appearing initially as a TV EPG, it shows users information, apps and further downloads about shows they are watching, creates live hyperlinks out of material discussed in shows and lets users both follow show-based social network chat and see what their friends are watching.”

Which former BBC iPlayer head is one of its two founders?

Answer: Anthony Rose

Question 2: From the New York Times:

So what took so long? Apple and [company] have been in talks for months trying to resolve issues that came up last year when the companies disagreed about [company]’s integration with Ping, Apple’s music-focused social network. Since then they have been caught in a stalemate over disagreements on coming projects.

But their iPad app finally launched in October. Which company is [company]?

Answer: Facebook

Question 3:

…And now suddenly we are living in that future. That future which only yesterday was the future now today is the present. [...] Where were you on Sept. 22, 2011, when the world changed? I, unfortunately, was holed up in my grubby little office in a small town in Massachusetts. Nevertheless, I will never forget this day. Never. Ever. How could I? This is the day when [feature] was introduced. [Feature]! It is, in a word, profound. Deeply, profoundly profound, in fact. “[Company] Just Schooled the Internet. Again” is how MG Siegler put it on TechCrunch. Which is a pretty amazing feat, coming as it does just a year and a half after “[Company] Just Seized Control of the Internet” as MG Siegler wrote in April 2010. It is pretty amazing, after all, to seize control of the Internet. That was bad-ass enough. But to then school the Internet that you’ve seized control of? Who but [Company] could do that? Good Lord I have to sit down and just think for a minute because my mind is reeling …

That was Real Dan Lyons implausibly over-excited about the announcements from which company’s annual conference?

Answer: Facebook, again, who announced Timelines and more at F8.

Question 4: What is the double- (or is it triple?) punning name of the tech startup recruitment fair that took place in May and October in East London?

Answer: Silicon Milkroundabout

Question 5: Here is a proposal from Mike Butcher:

You know what the problem with London is? It’s expensive. When you want to hire five rockstar coders (some prefer “gentleman coders” bizarrely. cc. Tariq Krim / Emi Gal) from Estonia or Budapest or wherever, you can do so – but they can’t even afford to move to London. So let’s requisition – nay, let’s storm “The Bastille” of those athlete villages after the games and claim a few blocks for the tech people. Can you imagine block upon block of innovative people housed so close together? That is your next Facebook or Google right there.

What is the UK government’s name for the East London development he’s talking about?

Answer: Tech City

Question 6: Here’s Mike Arrington, demonstrating his trademark compassion and empathy:

Suddenly everyone’s complaining about how unfair things are in Silicon Valley. How hard everyone has to work so darn hard, and how some people don’t get venture capital or a nice sale to Facebook or Google even though lots of other people are getting those things. [...] If you work at a startup and you think you’re working too hard and sacrificing too much, find a job somewhere else that will cater to your needs. [...] Work hard. Cry less. And realize you’re part of history.

…which was prompted by employee upset at which 4 1/2 year old, 1,600-person Silicon Valley startup?

Answer: Zynga, where “those who do not perform can perish”, according to the New York Times.

Question 7: Here is a story from Forbes:

In December 2009 Jobs beckoned Houston (pronounced like the New York City street, not the Texas city) and his partner, Arash Ferdowsi, for a meeting at his Cupertino office. “I mean, Steve friggin’ Jobs,” remembers Houston, now 28. “How do you even prepare for that?” When Houston whipped out his laptop for a demo, Jobs, in his signature jeans and black turtleneck, coolly waved him away: “I know what you do.”

What service do Houston and Ferdowsi run, which they refused to sell to Jobs, and to which Apple then responded by building its own competing service?

Answer: Dropbox, against which Apple compete with iCloud.

Question 8: Here’s the start of a story in Businessweek:

About five years ago, Apple design guru Jony Ive decided he wanted a new feature for the next MacBook: a small dot of green light above the screen, shining through the computer’s aluminum casing to indicate when its camera was on. The problem? It’s physically impossible to shine light through metal.

Fortunately the laws of physics are no barrier to Apple. What technology did Apple buy up in bulk to enable the green light to be seen?

Answer: Lasers. Hundreds of them, provided exclusively to Apple, and normally going for $250,000 each.

Question 9: “Apple does a great job. We need to improve our game and our products to take over the leadership position. Apple could go past [us] in 2012. We will try to become the champion in 2013. It will take time for the products that I have influence on to make it to the market.”

The new CEO of which company admits defeat, if only for the short term?

Answer: Meg Whitman, incoming CEO of HP.

Question 10: Oracle issues a scathing statement…

Either Mr. Lynch has a very poor memory or he’s lying.  ‘Some bank’ did not just happen to come to Oracle with [company] ‘on a list.’  The truth is that Mr. Lynch came to Oracle, along with his investment banker, Frank Quattrone, and met with Oracle’s head of M&A, Douglas Kehring and Oracle President Mark Hurd at 11 am on April 1, 2011.  After listening to Mr. Lynch’s PowerPoint slide sales pitch to sell [company] to Oracle, Mr. Kehring and Mr. Hurd told Mr. Lynch that with a current market value of $6 billion, [company] was already extremely over-priced.  The Lynch shopping visit to Oracle is easy to verify.  We still have his PowerPoint slides.

…hitting out at which British company?

Answer: Autonomy, which had just been bought by HP.

Question 11: Which company launched Qwikster… and then swiftly unlaunched it in the face of hostile customer reaction?

Answer: Netflix, which tried to disrupt itself. Cutting edge business theory confounded by old-fashioned reality.

Question 12: Computer Weekly reported:

The Department for Work and Pensions (DWP) has signed a seven-year contract with IBM worth £75m per year to provide systems, which will include the delivery of its flagship [redacted] programme.

What is the name of the programme?

Answer: Universal Credit, costing over £2bn, and one of the largest projects to use Agile techniques.

Did you have fun? Probably more than Leo Apotheker…

Normal service to this blog resumes soon.

So here is a quiz of the tech world in 2011. It’s entirely partial, with all the questions coming from links I’ve previously found interesting and posted to this blog.

Answers after the (festive) break.

Launches…

Question 1: “Zeebox is now live” said PaidContent in October. “Appearing initially as a TV EPG, it shows users information, apps and further downloads about shows they are watching, creates live hyperlinks out of material discussed in shows and lets users both follow show-based social network chat and see what their friends are watching.”

Which former BBC iPlayer head is one of its two founders?

Question 2: From the New York Times:

So what took so long? Apple and [company] have been in talks for months trying to resolve issues that came up last year when the companies disagreed about [company]’s integration with Ping, Apple’s music-focused social network. Since then they have been caught in a stalemate over disagreements on coming projects.

But their iPad app finally launched in October. Which company is [company]?

Question 3:

…And now suddenly we are living in that future. That future which only yesterday was the future now today is the present. [...] Where were you on Sept. 22, 2011, when the world changed? I, unfortunately, was holed up in my grubby little office in a small town in Massachusetts. Nevertheless, I will never forget this day. Never. Ever. How could I? This is the day when [feature] was introduced. [Feature]! It is, in a word, profound. Deeply, profoundly profound, in fact. “[Company] Just Schooled the Internet. Again” is how MG Siegler put it on TechCrunch. Which is a pretty amazing feat, coming as it does just a year and a half after “[Company] Just Seized Control of the Internet” as MG Siegler wrote in April 2010. It is pretty amazing, after all, to seize control of the Internet. That was bad-ass enough. But to then school the Internet that you’ve seized control of? Who but [Company] could do that? Good Lord I have to sit down and just think for a minute because my mind is reeling …

That was Real Dan Lyons implausibly over-excited about the announcements from which company’s annual conference?

Startups…

Question 4: What is the double- (or is it triple?) punning name of the tech startup recruitment fair that took place in May and October in East London?

Question 5: Here is a proposal from Mike Butcher:

You know what the problem with London is? It’s expensive. When you want to hire five rockstar coders (some prefer “gentleman coders” bizarrely. cc. Tariq Krim / Emi Gal) from Estonia or Budapest or wherever, you can do so – but they can’t even afford to move to London. So let’s requisition – nay, let’s storm “The Bastille” of those athlete villages after the games and claim a few blocks for the tech people. Can you imagine block upon block of innovative people housed so close together? That is your next Facebook or Google right there.

What is the UK government’s name for the East London development he’s talking about?

Question 6: Here’s Mike Arrington, demonstrating his trademark compassion and empathy:

Suddenly everyone’s complaining about how unfair things are in Silicon Valley. How hard everyone has to work so darn hard, and how some people don’t get venture capital or a nice sale to Facebook or Google even though lots of other people are getting those things. [...] If you work at a startup and you think you’re working too hard and sacrificing too much, find a job somewhere else that will cater to your needs. [...] Work hard. Cry less. And realize you’re part of history.

…which was prompted by employee upset at which 4 1/2 year old, 1,600-person Silicon Valley startup?

The obligatory Apple section…

Question 7: Here is a story from Forbes:

In December 2009 Jobs beckoned Houston (pronounced like the New York City street, not the Texas city) and his partner, Arash Ferdowsi, for a meeting at his Cupertino office. “I mean, Steve friggin’ Jobs,” remembers Houston, now 28. “How do you even prepare for that?” When Houston whipped out his laptop for a demo, Jobs, in his signature jeans and black turtleneck, coolly waved him away: “I know what you do.”

What service do Houston and Ferdowsi run, which they refused to sell to Jobs, and to which Apple then responded by building its own competing service?

Question 8: Here’s the start of a story in Businessweek:

About five years ago, Apple design guru Jony Ive decided he wanted a new feature for the next MacBook: a small dot of green light above the screen, shining through the computer’s aluminum casing to indicate when its camera was on. The problem? It’s physically impossible to shine light through metal.

Fortunately the laws of physics are no barrier to Apple. What technology did Apple buy up in bulk to enable the green light to be seen?

Question 9: “Apple does a great job. We need to improve our game and our products to take over the leadership position. Apple could go past [us] in 2012. We will try to become the champion in 2013. It will take time for the products that I have influence on to make it to the market.”

The new CEO of which company admits defeat, if only for the short term?

Business (actually, it’s “Miscelleneous” but that sounds like a cop-out)…

Question 10: Oracle issues a scathing statement…

Either Mr. Lynch has a very poor memory or he’s lying.  ‘Some bank’ did not just happen to come to Oracle with [company] ‘on a list.’  The truth is that Mr. Lynch came to Oracle, along with his investment banker, Frank Quattrone, and met with Oracle’s head of M&A, Douglas Kehring and Oracle President Mark Hurd at 11 am on April 1, 2011.  After listening to Mr. Lynch’s PowerPoint slide sales pitch to sell [company] to Oracle, Mr. Kehring and Mr. Hurd told Mr. Lynch that with a current market value of $6 billion, [company] was already extremely over-priced.  The Lynch shopping visit to Oracle is easy to verify.  We still have his PowerPoint slides.

…hitting out at which British company?

Question 11: Which company launched Qwikster… and then swiftly unlaunched it in the face of hostile customer reaction?

Question 12: Computer Weekly reported:

The Department for Work and Pensions (DWP) has signed a seven-year contract with IBM worth £75m per year to provide systems, which will include the delivery of its flagship [redacted] programme.

What is the name of the programme?

That’s it. Have fun.