While so-called "big data" initiatives are not new to a number of industries such as large financial services firms, pharmaceuticals, and large cloud companies it is new to most organizations. And the low cost and ease of access of the software and hardware needed to build these systems, coupled with an eagerness to unleash any hidden value held within all of those enterprise data, are two trends that have sent large, next-generation database adoption soaring.
Unfortunately, the efforts to secure these systems haven't soared equally as high or as fast. But fortunately, that appears to be starting to change.
In many cases, analysts say, big data initiatives began organically, within small enterprise departments or teams, and without much, if any, IT oversight or governance. In a recent survey by IDG Enterprise of more than 750 IT decision makers, almost half (48 percent) of enterprises anticipate big data will be widely used by their enterprise within three years, while another 26 percent expect significant use within a business unit, department, or division.
When it comes to security, big data poses a number of interesting challenges. Some of the challenges arise for similar reasons that make the consumerization of IT and BYOD trends so challenging for many organizations. "This is a very compelling security story because we're watching small organizations pull down open source tools and, with only a couple of programmers, be able to out-scale the largest Oracle databases in existence," says Adrian Lane, analyst and CTO at information security research firm Securosis.
"We're not talking about millions of dollars of infrastructure; we're not talking about large services teams parachuting people in and spending a couple of million dollars. We're talking agile, cost-effective, scalable modular databases that can be setup quickly by anyone," he says.
Now, add to that widespread and inexpensive access to large data sets and the reality that many enterprises don't know how to go about securing these implementations, and many vendors and open source projects don't have the security features that organizations need. There's the recipe for large privacy violations or a very large and costly enterprise breach.
It turns out that groups are starting to use these data. When Lane starting surveying organizations, he found that groups within the organizations actually were using these tools. "I was talking to marketing organizations that actually had hired data architects, under their own budgets, because they had interesting data that they wanted to mine. So, some of that went up to the cloud. Some of it was in-house, but there weren't any security controls on it, because that wasn't even part of the project's scope," Lane says.
Many times, these data were actually customer data that internal groups wanted to find out what behaviors and trends they could discern. Both Lane and David Mortman, another security analyst at Securosis, say that, almost universally, these teams believed there weren't any sensitive data in the database, but invariably that was not the case. "I'd ask them what they were doing for security, and they'd tell me they have logins; that was about the extent of it. It simply wasn't a part of the project scope," he says.
Sign up for CIO Asia eNewsletters.