Companies are using the cloud to gather data and analyze it on the spot, eliminating the need to bring it into the data center. "Companies like Amazon give you a compute layer to analyze that data in the cloud. When you're done analyzing it, you can always move it from, say, the S3-type layer to a Glacier-type layer," Nadkarni adds.
Glacier is an extremely low-cost storage option that Amazon Web Services announced earlier this year. It's designed for keeping data "on ice" for decades. Other companies are introducing similar cloud-based archiving services, says Csaplar, noting that these offerings are professionally managed at a very reasonable price and could, for example, serve as the ultimate resting place for old tapes.
With prices as low as pennies on the gigabyte, it's hard to pass up. "As long as your data is scrubbed and doesn't have any sensitive information, you can dump it into this kind of archive and reduce your data center footprint," says Nadkarni.
Words of Advice
There isn't just one approach [to big data storage]. You really need to look at the use cases you have internally and understand which technologies would best suit [them]. In the old days, we would try to use one tool and make that tool a sledgehammer for everything. Now we have a whole toolbox. So go out and understand how to use those tools and when those tools apply, and then effectively use them.— Lloyd Mangnall, vice president, MIS systems architecture, VHA, parent company of Novation
Don't Store Everything
There's a temptation to think that you're just going to store everything. First, that's a fool's errand because it will break the bank. Yes, storage is getting cheaper, but it's not getting cheaper as fast as we're getting more data. And second, it just doesn't make good business sense. Your need for all that data varies with time.— Jeff Crigler, CTO, Catalist
Big Data Isn't for Everyone
You have to be a fairly large company to generate that amount of data. For [small and midsize businesses], it's about being able to get more and more granular data out of what they've already got, and being able to mine and manage it.— Dick Csaplar, analyst, Aberdeen Group
Granted, [outsourcing] doesn't always provide the data you need at first blush, but with some effort and custom code, you can get great results.— Jeff Brown, CTO, Cheezburger.com,an internet humor destination
Mainstream enterprises are also showing interest in using the cloud for storing and analyzing data. Some 20% of IT leaders surveyed by IDC report that they've turned to the cloud as a way to augment their analytics capabilities, even though they have their own data centers to perform analytics.
Sign up for CIO Asia eNewsletters.