What makes a good metric go bad?

When hitting the metric becomes the end goal rather than driving an intended outcome.

Set a target with a sufficiently motivating reward and humans will find a way to hit it.

That’s great, until it isn’t.

Every metric needs a counterbalancing metric or you’ll have built a single sided measurement with unintended long term consequences.

Measure vulnerabilities in production as a proxy for the outcome of building a secure platform and there’s now a disincentive to find more vulnerabilities. The ones you find are already in production as well so you’re not doing anything to prevent them either.

Leading indicators take things a step in the right direction by aligning what you measure with actions leading to the desired outcome.

Take bodyweight. It’s a common dieting metric but it only shows if you are succeeding after the fact. If you miss your target, it’s already too late. Measuring what you eat and how much you exercise are much more likely to drive the outcome you want and to let you know if you’re doing the right things before you weigh in.

With any metrics, you have to consider what unintended behaviors they may drive. This helps design counterbalancing metrics to offset actions that would otherwise work against your ultimate goals.

Metrics are just another system you manage.  Consider how the system could break or function in unintended ways, then find ways to prevent those problems from happening.

Some not-at-all-original reminders when creating your most important metrics:

🎯 Develop leading indicators: Measure preventative actions that drive the outcome you want, not what already happened.

⚖️ Build a counterweight: Incentivizing single-sided metrics leads to unintended consequences.

💡 Ensure awareness of intent: Make sure everyone understands the purpose behind each metric and encourage feedback if they see it not driving the intended outcome.

🌎 Promote a holistic view: Encourage teams to consider the broader impact of their actions beyond just meeting specific targets.

You get what you measure so make sure you’re measuring what you want.