While awareness of Big Data is growing, only a few organisations -- like Google or Facebook -- are really in position to capitalise on it now. However, the time is coming and organisations that expect to leverage Big Data will not only have to understand the intricacies of foundational technologies like Apache Hadoop, they will need the infrastructure to help them make sense of the data and secure it.
In the next three to five years, we will see a widening gap between companies that understand and exploit Big Data and companies that are aware of it but don't know what to do about it, said Kalyan Viswanathan, global head of information management with Tata Consultancy Services' (TCS) global consulting group. The companies that succeed in turning Big Data into actionable information with have a clear competitive advantage, Viswanathan said.
"Today, most companies are aware of Big Data," he said. "There's a lot written about it. There are conferences about it. Awareness has become quite pervasive. But if you look at actually exploiting Big Data, I would say we're at the very beginning stages of it."
Viswanathan believes that Silicon Valley Internet-based businesses like Facebook and Google -- where the entire business is based upon the management and exploitation of data -- are leading the charge when it comes to Big Data. Industries like financial services won't be far behind, he said, and neither will the intelligence nor military communities. Other verticals like retail, telecom, healthcare and manufacturing will follow.
"In terms of readiness to exploit Big Data relatively soon, I would say the companies have to be market leaders in their industry segments," he said. "They will be the ones that tend not to wait until others have exploited new technology. They would rather forge ahead and set the standard for their industry vertical."
The Role of Big Data
What role would Big Data play? Well, for instance, a pharmaceutical company might want to identify the top 100 opinion-makers in the pharmaceutical world. To do so, it could crawl the web and go to millions of pages related to the industry, ingesting the data while weeding out anything that is not related to the objective. Or a car manufacturer could collect instrumentation data live from its cars in real-time as they are driven on the road.
In many cases, said Larry Warnock, CEO of Big Data encryption and key management specialist Gazzang, we have not yet imagined the ways in which we would leverage Big Data.
"It's like a giant fishing net dragging the bottom," Warnock said. "There's big fat tuna and swordfish in there, but also mussels and lobsters and flounder. They're just scraping data and they don't know yet what they're going to do with it. The correlations that could be drawn from that data haven't even been determined yet."
Sign up for CIO Asia eNewsletters.