Security metrics tell a story, do you know your audience?

As a security leader, if you aren’t aligning the metrics you  present with the audience you’re presenting to, you’re off on the wrong  foot with the exact people you are hoping to influence.

To be effective, you have to understand who the audience is, what  they care about, and why. Only then can you assemble a package of  meaningful metrics that conveys what the audience needs to do their jobs  effectively.

Bad metrics cause issues for the presenter and the organization

On the personal front, the presenter loses influence every time  they present information the audience deems irrelevant or  counterproductive. As a leader, they may not receive this feedback,  making it critically important that they put in the time to ensure what  they present aligns with what their audience needs to see.

First person view of presenter showing irrelevant metrics to the executive team. Again.

Beyond being personally detrimental however, presenting unaligned  metrics has a real, unaccounted for cost. By focusing efforts on the  wrong problem, they waste valuable time and money that could have been  put to more productive uses. Prioritizing efforts is one of the most  important jobs a security leader undertakes and poorly thought out  metrics directly undercut the ability to succeed in it.

I read a recent security newsletter where they described a number  of metrics that are useful for the board, including phishing email click  through rates and the number of vulnerabilities. On top of being  generally useless, putting them in front of the board would be sending a  clear message that you don’t understand their role in the business. For  the CISOs wondering where their “seat at the table” is, looking at what  you report upwards is a good place to start.

What makes for good security metrics?

There’s no one right answer - it will differ from industry to  industry and company to company and even within a company, will evolve  over time. There are however, some basic questions you can ask yourself  to shape your thinking on what measurements to use and how to convey  them:

1. Who are they for? An engineering manager and the  board are going to have very different perspectives on what information  they need to do their job well. Do you know what they care about and  why? Digging into their concerns can spark discussions that not only  help you but help them understand the context around the metric.
2. Have  you aligned with the rest of management on what the actual goal is?  While a wonderful ideal, your goal is very likely not to build an  infallible security program. Have you discussed the business's risk  appetite in the context of confidentiality, integrity and availability  as well as the tradeoffs involved for each?
3. Does  the metric influence outcomes or does it just report on things that  already happened? You want your key metrics to be leading indicators so  that when you move them in the appropriate direction, they move you  closer to your goal. Looking at the number of breaches tells you what  happened. Showing coverage of critical security processes and tools  conveys risk changing over time. Ensure your key metrics show how you’re  addressing risk, not just the symptoms.
4. How many metrics do you  focus on? If you have too many, it’s hard for the audience to know what  to focus on. Consider what are the 3 - 5 most important metrics you  want them to understand and buy into? While there will certainly be  others you use, highlighting what’s important sharpens your message and  empowers teams to make decisions that meaningfully contribute to key  outcomes.
5. Are there counterbalancing metrics you can use that  protect against inadvertently weakening your security posture elsewhere?  While not being the key metric, they ensure improving one metric does  not come at the expense of another. As an example, if you're focusing on  reducing the time to patch exploitable vulnerabilities, the stability  of systems post-patch could be something to watch.
6. Does the  audience know why you’re measuring it and the outcome it drives? Without  context, metrics can be misinterpreted or there can be a lack of  understanding why they’re important. It might be very well thought out  but if they don’t understand why it’s important, you’ll lack buy-in  which lowers effectiveness.

Selecting key metrics is foundational to guiding your team toward  the strategic goals for your organization. What you focus on measuring  will be what the team will manage towards so it’s well worth it to spend  the time to get them right.