Although vendor-written, this contributed piece does not advocate a position that is particular to the author’s employer and has been edited and approved by Executive Networks Media editors.
Big data is the hot buzzword in security analytics today, but buyers are skeptical because many companies have spent years building “data lakes” only to discover it was impossible to “drain the lake” to get something useful.
And unfortunately, today’s solutions often include expensive clusters coupled with static business intelligence reports and “sexy” dashboards that look good but add little to useful and productive security analytics. Focusing on the analytics and how to use the data (very valuable data) in order to make real time decisions, discover critical patterns, determine on-going and changing security policies and dramatically improve security – ah – that’s useful.
We only need to look at companies like Google, Amazon and Netflix to realize that big data can be a successful enabler for real time data mining techniques for complex data sets that have high velocity, variety and volume (3Vs). These companies use big data as key part of their business with predictive analytics that tell them what we want to buy or watch. This should be the model for truly useful security analytics.
Here are five “sniff tests” that will help you determine whether an approach being proposed will use big data techniques that will get you a useful outcome:
Sniff Test 1: Is your big data solution only about the “3Vs”? If a vendor is only addressing the Velocity, Variety and Volume issues of big data, then your big data system may be more efficient than your SIEM (Security Information and Event Management) but it will end as a big data storage trap. Your vendor needs to be talking to you about Bayes theory, regression, classification algorithms, dimensionality issues, etc., as a means to make big data useful by making it predictive and truly actionable. Yes – it sounds like rocket science and it may be scary – but it is a must for the dynamic nature of security events.
Sniff Test 2: What answer do you get when you ask “what do you mean by security analytics”? If you hear things like correlation, dashboards, queries, and alerts – it’s old school. You need to hear about machine learning libraries, cubes, cosine matrices, etc. Everything has to be based on laws of large numbers / outliers – i.e. techniques that make use of a lot of data and a lot of history to build things automatically (and constantly more precise) as opposed to a user that needs to stare at static aggregated data or manually define explicit security policies.
Sign up for CIO Asia eNewsletters.