Category: Governance

Better goal-setting with ratios

I’ve been thinking about goal-setting  — partly in the process of writing my last blog post, and partly because I’ve been reading about the Beyond Budgeting approach to management. There are plenty of thought-provoking ideas in Jeremy Hope and Robin Fraser’s book, but I want to throw out just one of the smaller ones which caught my attention: the idea of using ratios as KPIs, targets, and limits. These seem a really effective way of getting things done.

Bicycle ratioIntroducing ratio metrics

The kinds of targets that I hear about are very often single-unit targets: We want to have 200,000 subscribers; We want to break 3m monthly unique browsers; We want to get to 1m comments on the website. These are measured in single units: subscribers, monthly unique browsers, comments.

But perhaps more useful are ratio targets. Hope and Fraser have a financial perspective and talk about return on capital, cost-to-income ratio, and cost per head. There are also some standard ratio measures in the industry I work in, such as average revenue per user (ARPU). But we’re not exploiting this kind of thing enough, and such measures don’t have to be industry standard — it’s often more useful if they’re specific to an organisation.

Let’s try to imagine some more. Take the example of Quora, which has just hired product lead Sandra Liu Huang from Facebook. What kind of measurements might we consider for her as she enters her new world? We might have cost to serve per unique user;  answers per question; questions answered per user; updates per user per month; and plenty of others.

Why they make a difference

What really makes a difference with these kinds of measures is that they provide people with more levers to make success happen:

  • Seeking to reduce cost to serve per unique user offers the chance to reduce the costs of our server infrastructure, or increase (presumably) the number of unique users, or (much smarter) manage our server infrastructure so that it costs less to flex it in response to changes in usage.
  • Seeking to increase answers per question suggests we might look at the user interface to emphasise answering, or make it more difficult for people to ask silly questions.
  • …and so on.

If people have more levers then they have more opportunities to make success happen. That means the measures can’t just be kept inside the finance department or put in the shareholders’ report — they must be exposed and explained to everyone who contributes. Then everyone in the organisation can work together.

And this needn’t just be about relative improvements. In the bullets above I’ve used the metrics in relative improvement targets (decrease this, increase that) but they could equally be used in absolute targets (we want this figure to be X) or limits (make sure this number doesn’t fall below Y).

Mixing financial and non-financial numbers

Picking up from my last post about innovation from trust, this becomes very powerful when we mix financial and non-financial units. So previously the board might have set a goal to “increase our userbase to 2.5m users by the end of the financial year”, and (if they were really organised) might also have set aside, say, £300k capital to achieve it. Unfortunately that doesn’t offer the team (the people who have to deliver) much opportunity to make it happen, because (a) it’s very unlikely that in the rush of the annual budgeting process £300k is anything but a guess, and (b) if the team know they have £300k they know they may as well spend it all.

But if our target was something like “increase our userbase for no more than 12p per user” then they’ve got much more scope for success. They may propose a large project or a small project, and in particular may experiment on a small scale to see which strategies are mostly likely to be successful before proposing anything. Either way it will be efficient. Where does the board get the figure of 12p per user? By thinking in ratios, and looking back to see historically how much new users have cost.

Summary

In the end these are the key points for me:

  • A ratio measure provides a team or individual with more levers for success.
  • Such measures can (and probably should) be specific to what’s important to the organisation.
  • They can be used for relative targets, absolute targets, or limits.
  • This really only becomes effective when the numbers are communicated throughout the organisation, so everyone knows what they’re working towards.
  • Mixing financial and non-financial numbers (cost per this, or revenue per that) generates better judged and more efficient outcomes.

How innovation depends on trust

Innovation requires trust and freedom. But freedom only comes from trust, so the primary requirement for innovation is trust. And broadly speaking, the more trust you extend to a development team the more innovative they’re going to be. At the very least, innovation will not extend beyond the trust they’re given. Here are four levels of trust, and the innovation each can lead to.

Innovation is driven by trustTrust = 0

Let’s start at the very beginning, where trust = 0 and hence freedom = 0. “I don’t trust you to work by yourself; everything you do must be done in conjunction with me.” This is micro-management and generates nothing unexpected whatsoever. The only opportunity any team member has for creativity is during their bathroom break. We can do better.

Trust with tasks

Let’s extend trust a little bit: we allow our team members to complete individual tasks or deliverables that we specify — but let them determine how each task should be completed. This is work to a tightly-defined plan, and so any innovation will be under the hood — the internal architecture, the software design, and so on. But what you’ll see is what you expect. The tangible product, and every visible aspect of the product, is exactly what you imagined.

Trust with milestones

We can extend trust further still: we allow our team to deliver to clear product goals and trust them to work out the details of those product goals. This is where we start seeing innovation — as opposed to having innovation under the hood. Innovation of this kind is driven by project milestones or product principles, like “We want some personalisation features” or “The user must be able to share their work”. We want the development team — the programmers, user experience people, designers and so on — to be free to determine how to achieve this because it’s they who have the most intimate understanding of the product, how it fits together, what it’s capable of, and what it might be capable of.

In this scenario those project milestones or product principles come from outside the development team. It’s what they rely on product managers or other senior stakeholders for. Amazon does this with a press release: a description of the product in terms of end-user benefits, written to inspire and guide the development team.

Interlude: Where we don’t want to innovate

If you’re thinking that this amount of freedom, from this amount of trust, is just hippy nonsense that leads to escalating budgets and timescales, then let me bring us down to earth.

One area where we almost certainly don’t want to be innovating is project management. There are plenty of good project management methodologies out there, one or two of which will be good enough for us, and we don’t need to be inventing any more. So we may not go in with a detailed task list and instead trust the team to define the tasks themselves; and we may not go in with any predefined product features and instead trust the team to work out for themselves what features will achieve the stated aims of the product; but to not go in with any idea of how we might responsibly spend our budget, manage risks, showcase progress, communicate with stakeholders, meet timescales, and adapt to feedback is downright irresponsible. And to generally deprive the team of project management expertise is foolish. Unless we explicitly want to invent a new project management methodology we really should ensure the team picks one of these tools and has the wherewithal to use it well.

Trust leads to innovation, but let’s be clear about where we want to innovate.

Now back to our regular programming.

Trust to define projects

We can extend our trust even further if we allow our team to define not just product principles or milestones, but entire products or projects. We can set an organisation-level goal such as “we want to double our revenues” or “we need to make our service a social experience” and then see how our team can achieve it.

A goal like this is important for two reasons.

First, it eliminates irrelevant ideas. Anyone in an organisation can have ideas, but we really want ideas that push our business forward. If my media company specifies that our goal is to make what we do more social then an interesting idea like making an iPhone game will be considered irrelevant because it doesn’t meet the goal of making our existing work more social. In other circumstances it might be promising, but our current circumstances are about being more social and that’s what we need to focus on.

Second, it allows us to compare alternate ideas. I might propose that people could be given virtual biscuits every time they share an article. Already this is more relevant than the iPhone game idea thanks to having to focus it on our organisation-level goal. But if someone else has a proposal to create common interest groups around our content then I’d say my virtual biscuits are looking pretty poor, relatively speaking. Our organisation-level goal has provided a scale of comparison.

And round again…

We’ve just extended trust to enable people to envisage an entirely new project. This is the very first stage of development — the concept. We also have to implement it. And this takes us right back to the beginning: how much trust do we want to give to the team? None? Just enough to complete a series of predefined tasks? Or do we trust them to devise the most appropriate means to meet the principles of the product? This is our chance to inject innovation again.

Dealing with uncomfortable ranges

Lots of questions went unanswered in my last piece on putting ranges of projections into project proposals and I want to follow up on one of them here: What if our range of values isn’t something we’re happy with?

In my previous example our project had a revenue projection of £150k to £500k. But suppose the project was expected to cost £200k. Then that range encompasses both profit and loss. What can we do?

Uncomfortable boundsHere are some suggestions:

1. Collect more data

We may be very confident of our potential revenue, but perhaps we could be more confident. More research data should change our perception of what might occur, and so narrow the range of expectations. This might include customer research or research into other companies’ experiences.

Of course our findings may just serve to confuse the issue: we may collect six examples of other companies’ experiences only to find they differ wildly. In such cases we should be more discerning about how we see the data. For example it would be a good idea to learn more about each example and see which are closest to our own proposal.

2. Break down the problem

Our revenue projections (or other measurement) are likely to be a factor of several other elements. If we can break down the problem we can perhaps analyse those elements more easily. With more confidence over simpler measures we ought to be able to make tighter estimations of the whole.

For example, revenue may be seen as a factor of market size, typical customer spend, product visibility and ability of the product to meet customers’ needs. There may be market data readily available on market size and customer spend; we should be in a position to gauge product visibility if that’s a result of our own marketing; and focus groups or similar may be able to help us understand how the product is seen to meet customers’ needs.

3. Move the breakeven point

If we’re worried that we may generate £150k to £500k from a product or project that costs us £200k then maybe we should seek to reduce the costs. Of course if we reduce functionality or quality then there is likely to be some detrimental effect to the revenue. But we may be able to find high cost/low value items which change the game sufficiently.

4. Change the factors that influence the range

It may be that we can reduce uncertainty by actually influencing some of the critical elements. If it transpires that customers don’t see the product as meeting their needs sufficiently then we should change that. That may be by changing the product, or by improving our marketing.

Or we may find it’s useful to do the opposite of our previous suggestion of reducing cost: increasing quality (and adding a bit of cost) may pay extra dividends by increasing take-up. This kind of thing can be seen in places like the Apple App Store. I am at least twice as likely (probably four to five times more likely) to buy an app rated 4.5 stars than if it was 3.5 stars. Yet the difference in production cost to get to that level would have been far less than two-fold.

I’m sure there are more ways to respond to an insufficiently compelling range of possibilities in a project proposal. Overall, though, all these actions have the same beneficial effect I pointed to before: the relevant issues are surfaced so that they can be addressed appropriately and we increase the likelihood of our project’s success. And they are much more likely to be surfaced an addressed if we also surface that fact there is a range of outcomes, rather than project a single outcome.

Better figures in project proposals

Reading Doug Hubbard’s excellent How to Measure Anything I got thinking about the proposals I read and write, the problems they face, and how to fix them.

Some things are best not stated just as single numbersA typical proposal (I’m thinking mainly of internal projects) will make a claim like “This project will add 500,000 unique users in the first year” or “This will bring in £400,000 of new revenue by October” or “This will save £250,000 next financial year”. I know because I’ve written some of them. In choosing what number to use you don’t want to put in anything too low or the project won’t look worthwhile, so you tend to put in something that’s a little generous (but not implausibly so) and assume that all the contributing factors will work out for the best — the marketing, the timing, the user take-up, and so on. Depending on the audience you’re presenting it to you might even have some evidence for your projection.

The problem is, of course, that it won’t happen like that. Some things will work out well, some not so well. Everyone knows that, and if you’re facing a sceptical inquisitor then you’re just setting yourself up with less credibility before you start. Thinking back on the last such proposal I wrote I can imagine a conversation with a fantasy inquisitor that might have gone like this…

Fantasy inquisitor: You say this will bring in £400,000 before October.

Me: That’s right.

FI: Can you guarantee that? Can you guarantee £400,000 of new revenue?

Me: [fearing liability, and wondering which members of my family might get taken away if it doesn’t happen]: No, of course I can’t absolutely guarantee it.

FI: So what are you saying about the £400,000 if you can’t guarantee it?

What indeed? The truth is that will be the revenue if things generally go well. We might make even more. But realistically we might make less. Being honest with ourselves and our audience will help everyone. Let’s return to the scene…

FI: [repeating for the sake of the drama] So what are you saying about the £400,000 if you can’t guarantee it?

Me: I’m confident we’ll make that.

FI: How confident?

Me: Really quite confident.

FI: So you’re “really quite” confident it will make at least £400,000. And I suppose it could make us as much as… what? £800,000?

Me: Er, no I don’t think it will go that well. Maybe £500,000.

FI: So you’re confident it could make as much as £500,000. And you can be confident it won’t generate a penny less than £400,000, can you?

Me: Well, it certainly could make less, I suppose, if not everything went to plan.

FI: Would you be surprised if it didn’t make us anything at all?

Me: Yes, I would. Even being realistically pessimistic it really should make at least £150,000.

FI: Okay, so you’re very confident we’re looking at making £150,000 to £500,000 with this?

Me: Yes, I’m very confident of £150,000 to £500,000 of new revenue.

So our single (and therefore implausible-sounding) figure becomes much more realistic when we change it into a range, even if it goes down at one end. It also goes up at the other end.

I think this goes a long way in dealing with a sceptical audience becaues it is more realistic. For the same reason we should be more confident when we talk about it, too.

I’ll add two final notes. First, there’s an important question that needs to be asked, which I’ve omitted: What is your evidence for this? This may be calculations based on observations, historical data from similar projects in the past, market research, etc. Either way, it’s important to know that the figure(s) in question aren’t entirely random.

Second, the statisticians in the audience will point out that “very confident” is almost meaningless when we could say “90% confident” or attach some other actual number. I certainly appreciate that, and mostly concur. But for me a significant, easy, and meaningful step forward is to move from single figures to more realistic ranges. And this should promote more honest and confident dialogue.

The need (or otherwise) for a technology roadmap: Part 2

In the first part of this post I considered four ways a technology roadmap might come about. I dubbed them the bureaucratic, the defensive, the directive and the aligned. The aligned roadmap in particular is the ideal way to deliver the overarching organisational goal. Now in this post I’m going to suggest that…

Sometimes a roadmap just won't be usefulMaybe you don’t need a roadmap

So those are all respectable reasons to have a technology roadmap. But there are also respectable reasons to not have a roadmap.

To start with, not all companies have a clear strategy which determines where it aims to be in the medium or long term. If our company is like this then we can’t produce an aligned roadmap.

(By the way, that doesn’t mean our company doesn’t have a clear driver. Steve Blank says that “the purpose of a startup is the search for a business model (not execution.)” So for startups at least it’s possible to have a strategy without it leading to a technology roadmap.)

Next, we might think about a directive roadmap: “my company doesn’t have a strategy or direction, so I’ll create opportunities and drive out a strategy or direction from that”. But as we saw before, that has risks associated with it, and ultimately the risks might not be worth the reward.

Then, we might recognise that our department is in quite a healthy organisational position, so doesn’t need to create a roadmap for defensive reasons. Good for us. No point making grand plans for the sake of it.

And finally, if no-one outside our team is demand a roadmap for its own sake we aren’t looking to create a one to satisfy bureaucracy.

So what are we left with? Not much material from which to create a legitimate roadmap. Now there are two options.

Option one: forget it, and don’t worry. We don’t have a grand plan, but it’s not as if there’s no work to do. There are still projects, bugfixing and general consultancy that needs to happen. And as long we don’t feel the need for structure in every aspect of our lives then we will find nobility in honest labour. In fact, most people’s work is like this.

Option two is what I’m going to call “lean roadmapping”. Just because we can’t produce a roadmap with several milestones it doesn’t follow that we can’t decide what the single next milestone is going to be. A plan with only one milestone is hardly a roadmap, but it’s certainly more than operational or project delivery.

As an example, we might see that the company is generally increasing its global sales, or that it’s broadly shifting to more corporate clients, or that customer responsiveness is tending to become more central to the business. There’s no explicit strategy, but there is a general direction nevertheless. It’s this kind of thing that we can respond to. Increasing global sales may mean increasing support for global working. More corporate clients may require a shift to a more robust infrastructure. Customer responsiveness may mean better integration of CRM-type systems.

By spotting a general direction we can work out our next milestone. In fact, it would be wrong to work out any further milestones: there isn’t a clear strategy and it would be an expensive mistake to act as if there was. We might line up a number of candidates for what might follow, but we don’t need to make a commitment immediately.

Lean roadmapping moves us forward strategically, beyond what a project or daily operations might do, but it does so without requiring wasteful planning or doomed prophesising. Only when we reach the end of this milestone will we determine the next one, and only then by reassessing the business context as it is at the time.

So in summary: There are a few reasons we might usefully create a technology roadmap, ranging from the not-so-great to the excellent. But while having a roadmap for the right reasons is a good thing, the best business solution may be just to plan one milestone at a time to honestly reflect the realities of an uncertain future.

The need (or otherwise) for a technology roadmap: Part 1

This is a post in two parts, inspired by a recent conversation with a colleague. In this part I’ll set out what a technology roadmap is and circumstances in which it would be useful. In the next part I’ll set out why it might be best to go without.

Roadmaps can be usefulIntroduction

Let’s first make sure we know what we’re talking about. A roadmap is a series of milestones to achieve some end; a technology roadmap is a roadmap for the technology base of an organisation.

Simple examples might be: we will replace the following legacy systems so that we can respond more quickly to our users’ demands; we will move our non-core systems into the cloud, in this sequence; we will implement the following technologies so that we can operate our distribution system globally.

In reality such a roadmap is likely to be more complex, with several different elements.

People get very excited about roadmaps. If you have two departments about to spend your time and money, and one has a roadmap and the other doesn’t, then the one without is going to look pretty shabby. And that’s even without looking at the content of the roadmap…

Any roadmap has some positives

Now, setting aside for one moment the benefit that any particular roadmap or plan might or might not give us, any technology roadmap clearly has some positives. It demonstrates purpose to those outside it (“They’re on a mission…”) even if those outside don’t fully understand it (“…and I guess I have to trust them on that”). It gives purpose to those implementing it (“I have to go in today despite being sick; there’s a key deadline for our roadmap”). It shuts down any number of unwanted conversations (“Can you do this for me?” / “Sorry, we’re maxed out delivering the roadmap, and you know the board’s expecting results”).

Obviously this is an overly cynical perspective, but like I said, that’s setting aside the benefits of the roadmap itself. So roadmaps are good, and have benefits even if they don’t actually deliver any real value themselves.

But really and truly a roadmap does need to have a purpose, and (because technology for its own sake has no benefit) that purpose must be driven by something outside technology. Here are four suggestions of where that purpose might come from, in order of increasing maturity:

4. The bureaucratic roadmap

Here, a roadmap is created purely because corporate bureaucracy requires it. Maybe it’s budget time and your budget won’t be approved unless it looks like there’s a grand plan. Maybe the CEO — or the VCs who are bankrolling your startup — want to see things mapped out before they’ll give you the time of day.

This scenario is not much better (possibly worse) than the cynical reasons given above. Except this time it’s not you who is made to feel better for applying some arbitrary order to daily pressures, but the corporate powers that be.

3. The defensive roadmap

This is where a tech team realises it must do something for itself to head off something less palatable. An example might be integrating a corporately-approved web service, such as Basecamp, before too many staff sign up to similar services themselves, directly, and scatter company data all over the Internet. Another is modernising the mail system ahead of a merger with a larger company to avoid being utterly taken over by the larger company’s technology base.

Clearly it’s not great to be on the defensive, but at least we’re responding.

2. The directive roadmap

This is where a tech team recognises the company doesn’t have a particular direction, so it implements something in order to create one. An example here might be enhancing all the systems’ robustness and security because there’s an opportunity to shift from a consumer to a corporate clientbase, and the tech team realises this would be the way to open up those kind of opportunities.

The directive approach can really only be instigated by the most senior one or two people in the tech team. Also, it’s a bit risky because it could be called into question if there’s a lack of trust or change of leadership. The IT director could find themselves in a difficult situation if their peers suddenly thought they were running an expensive project which wasn’t aligned with any current company purpose. But if it works then it’s a good result: a previously directionless company gets a new lease of life. Or at least new opportunities, whether or not it chooses to exploit them.

1. The aligned roadmap

Almost certainly the highest form of technology roadmap. In this situation the organisation has a clear strategy which becomes the basis of every other department’s own, localised strategy — the the technology department is one of those.

If a company decides it needs to increase its overseas sales then there are implications for distribution, legal, advertising, procurement, and of course technology. Even without a single co-ordinated programme to achieve a specific goal every department suddenly has a purpose, each one can work out for itself what it needs to achieve that, and all departments are working to achieve the same thing.

That this is highest form of technology roadmap should not be surprising. It’s born of a very high level of organisational maturity which may seem easy but isn’t in practice. Just recall the story from eBay where the company had a long-term strategy to emphasize fixed-priced goods, but they managed to forget to tell the technology team.

In the next part of this article I’ll consider reasons why it might actually be better not to have a roadmap…