Last week, two days in succession, I had cause to solve a problem using impact estimation tables. Unfortunately this technique is not widely known, so I thought I’d have a quick pass at explaining it. For many more details—beyond the simplicity of the description below—you really need to see the work of Tom Gilb, who created them. I have a copy of his Competitive Engineering, but he describes them elsewhere, too.
Impact estimation tables (or IETs for short) are used when we want to choose a solution, but the evaluation criteria are complex. Last week in one conversation I was helping a small team choose a development framework for a new project. The next day I was discussing with someone which technology strategy would be best for the next phase of their product. Success in both cases was judged on several criteria, and there were several potential solutions to choose from.
The first step of an IET is simple: create a table.
Each row should be one of our evaluation criteria. These might be things like security, usability, speed of delivery, and so on. The important (and challenging) thing is to make each of these criteria as unambiguous as possible. Ideally you’ll have a quantified metric. For example, security might be “chance of a successful phishing attempt within the next 12 months”; usability might be “time taken for a new user with domain experience to learn to perform [some specific task]”.
Each column of the table should be one of our potential solutions.
Then in each cell we need to decide how well that solution delivers against that evaluation criteron. We could do this in a number of ways. We might just enter a single number, perhaps representing a percentage of how much it achieves our target for that criterion. We might enter a range. (“We’re 90% confident it will deliver between this and that much of our target.”) One person I know uses just a tick and a cross to keep things really simple.
When we have all the data we can make a decision. Or at least, we can have a much more focused conversation. Perhaps something jumps out at us which immediately renders one potential solution the obvious choice… or perhaps rules it out immediately.
Or perhaps we’re still not sure, because some of those ranges reveal too much uncertainty. In this case we may well have to do some investigation work to reduce the uncertainty. But at least the purpose of our investigation is clear.
There is more to this subject than I’ve described here, but I think even this basic understanding is very helpful. Impact estimation tables don’t hand us the answer on a plate, but they do allow us to focus our thinking much more effectively.
niksilver.com 