A little while ago I was working with a team that had a lot of user research to do. Naturally, we only had limited time, so we needed to decide how to prioritise our work. This is how we did it.

First of all we wrote out all the ideas we wanted to examine. Some of these were actual hypotheses—do our users respond better to this or that? If we did X would they conclude Y? Some of the things we wanted to examine were more general—what do our users think of this way of presenting information? If we offered them five alternative ways of doing something, what would we discover were the strengths and weaknesses of each. And so on.

Next we drew two axes and positioned each idea somewhere in the x/y space. The x-axis ran from 0% to 100% and represented our confidence in the result. If we were certain our users would hate it we scored it at 0%. If we were certain they’d love it we scored it at 100%. Something at 75% meant we were 75% sure people would like it, but had about 25% reservation. Anything we were wholly unsure about was 50%.

The y-axis ran from 0 to an arbitrary 10. It represented impact on our goals, on some made up and unspecified scale. If we thought an idea would have no impact we’d put it at 0 on the y-axis. If something had middling impact it would score 5. A score of 10, of course, was maximum impact.

Once we had everything positioned we could prioritise the testing of our ideas much more effectively. The highest priority tests were for anything closest to 50% on the x-axis and maximum on the y-axis. These were things where we had no sense of how people would respond, but knew getting it right would have the biggest impact. The further anything was from the (50%, 10) point the lower its priority for testing.

Of course, this is all based on estimation and an arbitrary impact measure, but it gave us a good step forward for prioritising our user research tests.