A long while ago I spent a fair bit of time looking at “risks” on a risk list. There was a long list of “things that might go wrong”, and beside each one were three numbers. There was a “likelihood”, which was a rating of 1 to 5, an “impact”, which was also a rating of 1 to 5, and finally the “risk score” which was the multiplication of these two numbers. Then we were expected to do something—or not—depending on the risk score. Over some threshold we needed to seriously consider acting. Over a higher threshold we definitely needed to do something.
I’ve thought about that a lot since, because it’s long seemed wrong. I’ve written before about some of the problems [1], [2], [3], [4]… but let me add two more.
One is that multiplying categories isn’t meaningful. It’s true that a likelihood of 1 is less than a likelihood of 2, but it’s not half of that. Nor is it one fifth of the likelihood of 5. The number is just a category, with some broad idea of order. Similarly, an impact of 4 is not twice the impact of a 2 in any mathematically-valid way. Again, it’s just a number for a category. And that means multiplying the two together is especially meaningless. You might as well rate the Covid-19 danger level of Bolton Wanderers FC as 6, because they’re in English Football League 2 and are based in Greater Manchester, which is Covid tier 3. Multiplying categories doesn’t produce a meaningful result.
Another problem is that we’re assessing the risk without any reference to a reward—or, to put it in more “management” terms, without any reference to our objectives. We can’t make sensible decisions about downsides without also considering the upsides. Simply deciding what to do about a “risk” without referencing organisational or project objectives is bound to give a skewed outcome.
To give an example, I once worked with a team towards the organisational objective of having them be more agile in their work. As part of that we—eventually—visualised their work on a movable whiteboard. There was not wall space for a whiteboard fixed to the wall. The Health & Safety team tried to stop use of the whiteboard because it was deemed an unacceptable risk; someone might trip over the base. But this risk was evaluated without reference to the benefit it was bringing us.
I do believe that saying an event is unlikely is saying something meaningful about it, and saying it could have a high negative impact is saying something meaningful about it. There is some information there. But it’s information whose roughness we have to recognise, and which we must put into a bigger context if we’re to do anything useful with it.
niksilver.com 