Paul Clarke has an excellent post in which he talks about the importance of having a purpose when trying to measure things. That’s not quite the point of his post, but it does nevertheless provide a really telling concrete example of why it can be meaningless to try to measure something if there isn’t a concrete objective to the measurement.

When referring to Doug Hubbard’s book “How to measure anything” I sometimes uncharitably say he performs some sleight of hand right at the start, part of which is to insist that we must be measuring with the intent of influencing some concrete decision. It’s sometimes difficult to explain why that is so important, but Paul provides good examples.

He describes his conversation with “a doughty advocate for public transparency”:

Our debate arose from his astonishment that it wasn’t possible for “government” to say at any one time how many people it employed. Despite this being an “obvious” factual issue in his eyes, no amount of requests seemed to be able to produce a meaningful answer.

My response “well, it’s not really a meaningful question” – didn’t go down too well.

Paul lists a number of reasons why the word “employed” is ambiguous “with all its colour and texture of vacant posts, secondments, part-funded posts, long-term absentees and part-timers”. The word government is similarly problematic (as indicated by Paul’s use of quote marks above): does it include people employed by local authorities? Quangos? Working in NHS hospitals? In government-funded voluntary organisations?

It was this segment that caught my eye and triggered my post here:

If asked by an economist with a specialism in operational research or organisational productivity, I could possibly, possibly see some sort of tangible purpose to a question, but more likely a version targeted at a more specific organisation or sector than just “all of government”. Possibly.

Indeed: possibly. If that economist was on a mission to do something specific. Then we could nail down the exact definition of “employed” and for “government” depending on what concrete thing they wanted to achieve.

Here are some things someone might want to achieve, and what we therefore might (arguably) include or exclude in our “employee” count.

• We want expand the public sector because we think that will boost the economy. Include the number of people directly on the payroll of local and national government, and all services funded by local and national government. Pro rata for part-funded services. Exclude services contracted out. We will then seek to increase this number.
• We want to improve public service by putting more staff on the front line. Add up the number of people who deal directly with the public as a key part of their job description (which is a questionable definition in itself). Include people whether or not they are directly employed. We will then seek to increase this number.
• We think big government is bad, so we want to check we’re spending less on people. Add up all costs spent directly on people. Include contractors employed directly. Exclude people employed by companies we are contracting out services to. Include or exclude local government depending on whether “big government” includes local government. We will then seek to reduce this number.
• We want to decentralise decision-making, so want fewer people employed centrally. Count the FTEs performing functions at a national government level, regardless of whether or not they are directly employed. Exclude anyone not working at a national level, such as those in hospitals and local government. We will then seek to reduce this number.
• We want to write a Daily Mail article with maximum harrumphing. Count the number of people working on any kind of government-backed projected. Include people regardless of whether they are employed directly or work for a company contracted to fulfill a service. Count everyone equally, whether they work full time or one hour a year.

I have a number of concerns even with this list, not least of which is that I don’t think any of the proposals is complete in any way. But it does show that the specific, concrete purpose does influence exactly what you measure, and conversely that having no such purpose makes it impossible — certainly in this case.

The purpose of Paul’s article, by the way, was really to say that a quick and easy answer to almost any question about government technology is a near-impossibility, and that this needs to be explained in ways that resonate with harrumphing Daily Mail readers. Addressing that is not the point of my article, but perhaps the examples above are also the start of how we might achieve that.