Metrics, measures and indicators: a few things to bear in mind
Metrics, measures and indicators help you track and evaluate outcomes. They can tell us if we’re moving in the right direction, if things aren’t going well, or if we’ve achieved the outcome we set out to achieve.
If you’ve reported on key performance indicators (KPIs), checked progress against objectives and key results (OKRs) or looked at user analytics, you’ll have some experience with metrics, measures and indicators.
These words are often used interchangably and, in general, the difference isn’t important. Not for this post anyway. We can talk about the difference between metrics, measures and indicators later.
In this post we’ll cover some guiding principles for designing and using metrics, measures and indicators. A few things to bear in mind.
Guiding principles
- Value outcomes over outputs
- Measures, not targets
- Balance the what (quantitative) and the why (qualitative)
- Measure the entire product or service
- Keep it light and actionable
- Revisit or refine as things change
Value outcomes over outputs
We acknowledge that outputs are on the path to achieving outcomes. You can’t cater for a memorable birthday party without making some sandwiches.
But delivering outcomes is the real reason why we’re here. So we don’t measure whether we’ve delivered a product or feature, we measure the impact it’s having.
Measures, not targets
Follow Goodhart’s Law: ‘When a measure becomes a target, it ceases to be a good measure.’
There are numerous factors that contribute to a number or reading going up or down. Metrics, measures and indicators are a starting point for a conversation, so we can ask why and do something about it (or not). The measures are in service of learning: tools, not goals.
Balance the what (quantitative) and the why (qualitative)
Grown-ups love numbers. But it’s very easy to ignore what users think and feel when you only track quantitative measures. Numbers tell us what’s happening, but feedback can tell us why.
There’s no point doing something faster if it makes the experience worse for users, for example – we have to balance quantity and quality.
Measure the entire product or service
If we can see where people start, how they move through and where they end, we can identify where to focus our efforts for improvements. The same is true for people who come back too, we want to see whether we’ve made things better than last time they were here.
If you’re only measuring one part, you only know how one part is performing. Get holistic readings.
Keep them light and actionable
It’s easy to go overboard and start tracking everything, but too much information can be a bad thing. If we track too many metrics, we run the risk of analysis paralysis. Similarly, one measure is too few: it’s not enough to understand an entire system.
Four to eight key metrics or indicators per team is enough and should inspire action.
Revisit or refine as things change
Our priorities will change over time, meaning we will need to change our indicators, measures and metrics too. It’s no use tracking and reporting on datapoints that don’t relate to outcomes. Measure what matters.
We should aim not to change them too frequently – that causes whiplash. But it’s all right to change them when you change direction or focus.
Are we on the way? Or did we get there?
Those principles are handy for working out what to measure, but there’s two types of indicator you need to know about: leading and lagging.
Leading indicators tell us whether we’re making progress towards an outcome. Are we on the way? For example, if we want to make it easy to find datasets, are people searching for data? Is the number of people searching for data going up?
Lagging indicators tell us whether we’ve achieved the outcome. Did we get there? In that same example, making it easy to find datasets, what’s the user satisfaction score? Are they requesting new datasets?