Big Data Security
Before the term "big data" became common nomenclature in the security industry, there was a trend largely inspired by SIEM and log management solutions. This trend resulted in the mass collection and storage of log data. This helped placate auditors and make storage vendors a lot of money, but without capabilities like threat intelligence feeds, automation, analytics such as correlation, anomaly detection, pattern discovery and prioritization, their effectiveness was limited. Simple collection and storage isn't enough. Today, with big data being measured at levels never before operationalized, such as the Yottabytes of storage that some military-level data centers are being built to handle and the Undecillion IP addresses in IPv6.
Perhaps the most important variable, so that security can be managed by exception in the face of staggering data volumes, velocity and variety, is context.
Big Data Security Context
Folks I spoke with in ANZ want to move beyond thinking of data, regardless of that data being logs, alerts, packet captures, metadata, flows, threat feeds, malware detonation outputs and the like, in terms of what they can collect and store. They want to automatically extract value from it. They want machines to:
- Evaluate all data sources across traditional IT, cloud and mobile
- Illustrate root cause
- Visualize the attack sequences
- Associate identity information
- Weigh the incident against historic knows
- Consider the attacker source and attack type
- Associate target system intelligence such as operating system, applications, data, regulatory mandates, etc.
- Prioritize output
- Incorporate incident workflow
- Allow for human analytics from a single pane of glass
- Offer mitigation solutions with weighted impact relevance
More simply put, they want to have context delivery automated so security analysts are given a prioritized list of "stories" to review as opposed to some sentence fragments that they need to piece together.
In ANZ — and frankly everywhere in the world — deriving this level of context is a bit of a utopia at least today. All the pieces of the puzzle are being provided at some level by disparate solutions. Some of these solutions are even integrated. But having a unified, inclusive solution made up of all the necessary best-in-breed technologies that's scalable and effective and will allow security analysts to truly mange by exception is still a ways off but certainly worth striving for.
As organizations begin to embrace big data security, or are already starting to tune their program, context must be at the core of the requirements list. Without context, the simple math of the problem will introduce far too much complexity to be of value and big data security will become be a big waste.
Sign up for CIO Asia eNewsletters.