A couple of weeks ago I said that, although I think it’s a mistake to “score” risk events in terms of simple likelihood and impact, it’s not entirely devoid of information.
My preferred way to express uncertainty about a situation is using a probability density distribution, such as some kind of bell curve, or other shape. And that led me to wonder if there is some kind of translation we can do from a risk score to a probability density distribution. It would be rough, of course, but it might be helpful. For example, something with “low probability and high impact” might translate to a distribution that’s skewed to the right, say. If so, it would help us move from a less helpful approach to a more helpful one with minimal effort.
And I think the answer is… maybe. To explain that, and to understand what it depends on, let’s take an example.
Consider a danger such as a ransomware attack on our desktop computers happening this year, and suppose we deemed that “low probability” and “medium impact”.
This is still isn’t useful unless we quantify “low probability” and “medium impact”, so let’s further suppose that these mean 10% and £50,000 respectively. Fine so far.
But there are still a couple of other important questions to clarify. The first is to ask what the likelihood refers to. We might be talking about the event of an attack, in which case we are saying “there is a 10% chance of an attack happening, and if it happens then it will cost us £50,000.” But we might be talking about the effect of an attack, in which case we are saying “there is a 10% chance that ransomware attacks will cost us £50,000”.
These might sound the same, but they are not. In the first version we are implictly saying there is a 90% chance of ransomware attacks costing us nothing (or next-to-nothing). In the second version we are implicitly saying there is a 90% chance of ransomware attacks costing us something notably different from £50,000; and that might be something much more or less than £50,000.
This leads us onto the second question: when we say a £50,000 impact do we mean up to £50,000? Or roughly £50,000? Or at least £50,000? Or on average £50,000? (There’s next to no chance it will cost exactly £50,000. Not really. Not to the penny.)
If they do clarify this, usually people mean “roughly”. That’s because they often use ranges of cost for each impact score 1, 2, 3, etc or low/medium/high/etc. So an impact score of 1 might be £0 to £10,000; an impact score of 2 means £10,000 to £100,000; etc. Each of these is “roughly” some figure. And in ordinary language “roughly” some figure implies some range either side of it.
So let’s go back to our question of translating our risk score to a probability density distribution, and let’s take a number of scenarios.
Scenario 1: Our likelihood refers to an event, and our impact is roughly our given figure. So we’re saying “there is a 10% chance of an attack happening, and if it happens then it will cost us £10,000 to £100,000.” This does give us a probability density distribution, but a rather implausible one. It’s a distribution where there’s a 90% chance of near-zero impact, 0% chance of a zero to £10,000 impact, 10% chance of a £10,000 to £100,000 impact, and 0% chance of more than a £100,000. Those 0% bands are pretty implausible—are we really saying it’s impossible that it will cost us a penny more than £100,000? If the response is, “well, it’s not impossible…” then we need to apply a second risk score and until we do so we are missing important information. In other words, our original score might not translate to a probability density distribution because we may have information missing.
Scenario 2: Our likelihood refers to the impact, and our impact is roughly our given figure. So we’re saying “there is a 10% chance that ransomware attacks will cost us £10,000 to £100,000.” This definitely means there is information missing, because by implication there is a 90% chance that ransomware attacks will cost us less than £10,000 or more than £100,000. That 90% could be balanced all on one side, or all on the other side, or… some other way. Again, there is information missing and we cannot create even a rough probability density distribution.
Scenario 3: Our likelihood refers to an event, and our impact is at least our given figure. So we’re saying “there is a 10% chance of an attack happening, and if it happens then it will cost us at least £50,000.” This does translate well to a probability distribution. The distribution says there is a 90% chance of ransomware attacks costing us (next to) nothing; there is a 0% chance of them costing us up to £50,000; and there is a 10% chance of them costing us £50,000 or more. For me, this is not a great distribution, mainly because we don’t get a feel for the chances of, say, a £1m impact (is it 5% or 0.1%?). And we should check if we think it’s really impossible that it won’t cost us less than £50,000. But otherwise it is at least complete.
There are eight scenarios in all, and I won’t bore you with all of them. But in the end we can’t always translate from a risk score—a point risk—to a probability density distribution. And that’s a great shame, because it means a lot of time and effort that’s gone into organisations’ risk management gives only a partial view of the uncertainties the organisation faces—and “partial” doesn’t mean “approximate”, it means “information is missing”.
niksilver.com 