Simply having the technology doesn't guarantee effective use of Big Data. Stefen Smith, CSO at SecureForce, agrees with Kim Jones that most enterprises are "not up to speed" when it comes to Big Data analytics.
The tool sets now available, which besides Hadoop include EMC's Greenplum, Teradata, HP's Vertica and Palantir, offer plenty of value, he said, but need a significant amount of human expertise to be used effectively, since they all are different technologies that are focused on different areas.
"To find data related to an insider threat or regulatory compliance, things have to be configured to find what's important to the organization," he said. "Until somebody is able to deploy these disparate technologies, it's going to be tough for organizations to achieve success."
One vendor, Smith said, has an "awesome suite," but on its website makes the point that it needs the expertise of "data scientists. So, you're talking about needing people with advanced degrees who know how to find patterns and look for it and organize it."
Bob Rudis agrees. "It's not really about the tools," he said. "It's about the people and processes."
That includes, he said, backing (including money and policy directives) of senior management, smart security people who know what questions to ask, smart data analytics people who know how to ask those questions and solid governance and maintenance models in place to ensure tools and processes are kept up-to-date.
"All that," he said, "plus storage -- lots and lots of storage."
BT's Bryan Fite emphasizes the human element as well. "Big Data doesn't work if you don't have humans handling it. You can't buy technology and get rid of humans."
Then there are the risks and responsibilities. The fact that the tools are available to aggregate and analyze Big Data means regulators and the courts increasingly expect those involved in discovery proceedings to make use of them.
Heather Clancy, writing on Smart Planet, noted that, "analytics and 'big data' technology is making e-discovery software smarter, helping legal departments avoid costly fines associated with failing to produce all relevant documents related to lawsuits or other government investigations."
But failure to use it, she wrote, "can also be a huge liability. Consider the 2008 case of Qualcomm and Broadcom, which were embroiled in a patent dispute. Along the way, things got ugly when the judge fined Qualcomm $8.5 million for withholding evidence."
In law enforcement investigations, the reality of Big Data means collecting more than just the laptop computer of a suspect. The list also includes loose hard drives, modems, routers, digital cameras, games consoles and, of course any smartphones or tablets.
A shifting legal strategy
Kim Jones notes that it is also changing legal strategy. "It has long been the practice when one side gets data requests for trial or prosecution, to deluge the other side with data, under the assumption that they'll never find what they're looking for. But Big Data means they can find it. Even worse, given the analytic capability of the tools, they might find more than you thought they would."
Sign up for CIO Asia eNewsletters.