Security metric No. 3: Code defect density
Defect density, or the number of issues found in every thousand (or million, depending on the codebase) lines of code, helps organizations assess the security practices of its development teams.
Context is key, however. If an application is at an early stage of development, then a high defect density means all the issues are being found. That's good. On the other hand, if an application is in maintenance mode, the defect density should be lower -- and trending downward -- to show the application is getting more secure over time. If not, there's a problem.
Security metric No. 4: Windows of exposure
An organization may identify defects in the application, but until they've been addressed, the application remains vulnerable. The window of exposure looks at how many days in a year an application remains vulnerable to known serious exploits and issues. The "goal is to have zero days in a year during which serious defects found are known and have not yet been addressed," Wong said.
Management in general likes to focus on security incident prevention, in part due to the legacy notion that organizations can stop all attacks at the perimeter. For example, it might make everyone feel good to see the number of intrusion attempts that were blocked, but there's nothing actionable about that information -- it won't help security teams figure out which attacks were not blocked. "You're not fixing anything," says Joshua Douglas, CTO of Raytheon/Websense.
Mean response time, or how quickly the issue was found and mitigated, is another metric that may be less than helpful. Response time ignores the fact that attackers tend to move laterally through the network. You may fix one issue, but if no one tries to determine what else the attacker may have done, a different system compromised by that same attacker may go unnoticed. Focusing on individual issues alone and not on security as a whole leaves environments vulnerable.
"It's not one and done, it's one and understand," Douglas said.
Another common metric tracked is reduction in vulnerabilities, but it isn't so useful on its own. If a lot of low-level vulnerabilities have been fixed, the organization's risk remains the same while critical issues remain open. Some vulnerabilities mean more than others.
Only 28 percent of executives surveyed in a recent Raytheon/Websense survey felt the security metrics used in their organizations were "completely effective," compared to the 65 percent who felt they were "somewhat effective." Security practitioners need to explain to senior management how to focus on security questions that help accomplish well-defined goals. Otherwise, too much attention is wasted on information that doesn't actually reduce risk or improve security.
"Is that really the best place for you to be spending your limited time and money?" asks Wong.
Sign up for CIO Asia eNewsletters.