by James Kaplan and Jim Boehm
“If you don’t measure it, you can’t manage it,” said George, the chief strategy officer, to a nodding CEO Tom.
Cybersecurity performance can be managed, but only if measured.
The ability to measure performance has always been at the heart of effective management, underlying decisions about how to allocate resources, which practices to employ and whom to reward. Much more so than in the past, this is an age of granular and systematic performance management. Senior executives are exploiting massive amounts of data to understand which products generate profits, which salespeople sell effectively, and which operational teams execute with the highest degree of efficiency.
Sadly, in many respects, cybersecurity is an outlier to this trend. Measuring cybersecurity performance is hard. Traditional business performance metrics such as revenue or cost are not really relevant. Analogues to market and credit risk metrics like value at risk do not exist for cybersecurity. And measuring cybersecurity incidents might lead you to believe you are doing a good job protecting the organization, when in fact you are doing such a bad job monitoring the environment you cannot even detect ongoing attacks.
The difficulty in measuring cybersecurity performance does not make it any less important. The dynamic nature of the cybersecurity environment—threats escalating rapidly, new technologies introduced constantly, and operational practices evolving quickly—makes it dangerous for cybersecurity executives to rely on experience and instinct in making decisions.
Fortunately, there is a better way. With enough creativity and true understanding of sources of value, cybersecurity elements worth managing can be measured, even if only by proxy. Measuring performance and organizational health is critical to catalyzing progress, instilling accountability, and ultimately achieving an organization’s strategic aspirations.
There are a number of pitfalls organizations should avoid in measuring cybersecurity, including:
- Irrelevant metrics. Many reports to the senior management team we see include some reference to the millions of attacks the organization faces per week or per day. While eye-catching, this number is entirely irrelevant. The overwhelming number of those attacks come from “script kiddies” (unskilled hackers who cause minimal damage) that a minimally competent security capability can deflect with ease. For most organizations, the tiny percentage of attacks from sophisticated attackers represents the true risk.
- Focusing on lagging indicators to the exclusion of leading indicators. The frequency and severity of security incidents is important information but is inherently a lagging indicator. It represents an output, rather than a lever or an input that a management team could choose to affect directly.
- Assuming more is better. Even those organizations that look at leading indicators such as the extent of encryption can make the mistake of assuming that more and tighter controls are always the right answer. Ten years ago, when environments were more likely to be wide open, this might have been the case. Today, organizations can very easily incur too much cost and create too much complexity by creating metrics that encourage the encryption of every piece of data and the application of two-factor authentication to every system when in many cases neither may be necessary.
- Relying on subjectivity. In a world where quantitative metrics are challenging, cybersecurity executives may be inclined to report that their data loss prevention program or identity and action management program is red, yellow, or green. Even if the team performing the color coding has the best intentions in terms of objectivity, subjective assessments like this one will always be less than credible with senior management in terms of driving decisions unless those colors are tied to specific measurable or milestone-driven targets.
- Measuring the cybersecurity organization rather than enterprise resilience. We are fond of saying that 80 percent of what you have to do to be secure happens outside the chief information security officer’s organization. The cybersecurity team cannot write secure code for developers or apply patches quickly for data center managers, even though both actions are critical to an organization’s overall security posture. As a result, it is easy to focus cybersecurity metrics on what the security team does directly, rather than what it is supposed to achieve by driving resiliency across the entire organization.
It is easy to want the highest level of capability, but there are real constraints to consider. Achieving the hallmarks of digital resilience requires real organizational change across many business functions, so organizations have to ask what level of appetite exists for change. It also requires a level of skill and sophistication on the cybersecurity team that many organizations do not have and would have a hard time obtaining.
On the other hand, organizations also have to balance challenges like these against imperatives for change: How important is sensitive information to the future of the business? How sophisticated are attackers? What is the level of regulatory scrutiny? How important are cybersecurity capabilities and protections to customers?
James Kaplan is a partner in McKinsey’s New York office. Jim Boehm is a solution manager for Cyber Solutions in our Washington, D.C., office.
Excerpted with permission of the publisher, Wiley, from The Cyber Risk Handbook: Creating and Measuring Effective Cybersecurity Capabilities by Domenic Antonucci. Copyright (c) 2017 by John Wiley & Sons, Inc. All rights reserved. To order, visit www.wiley.com/buy/9781119308805