The volume of digital information is expected to nearly double every two years between today and 2020, reaching 40 trillion gigabytes, in just seven years according to analyst firm IDC.
To stay ahead of this storm of data, business leaders, entrepreneurs and even start-ups are scrambling for ways to take advantage of the insights everyone agrees are hidden in this Big Data.
The big data question: How?
The term Big Data today is a commonplace in tech publication headlines. However, even CIOs thinking about the impact of Big Data, business leaders outside IT management have been slow to engage. The business world, in general, is still focused simply on how to deal with an overwhelming volume of information. Researchers at the Gartner Group expect that by 2015, some 85% of Fortune 500 companies will still not be effectively exploiting Big Data's value for competitive advantage in their markets.
While some companies ponder how to turn a growing volume of data into an asset, others sit on corporate data without a plan to tap its hidden value. Some CIOs may think they already have done what they can to create business insights; they gather, extract and query the data. Still many CIOs are struggling to unlock the insights and business opportunities buried in their corporate data.
What are these CIOs missing? In my opinion, they have overlooked a critical part of the success equation — how to make the most of the data resource their company owns. Is the promised land of business insights beyond our reach? Absolutely not.
Oftentimes organizations simply don't go far enough in giving users tools needed to make discoveries happen. This is unfortunate. Today, however, this is easily remedied with access to user-driven business intelligence (BI) systems which enables employees across an enterprise to leverage data for insights to make the business more agile, answer questions previously considered beyond reach, and ultimately leverage corporate data to make faster, better informed business decisions.
Hive into Hadoop
Data collection and integration are the first step to making data discovery possible. However, hadoop queries are usually slow as traditional BI systems were never designed for user-driven analytics, and certainly not for consumers.
Today's computer users, however, have skills and demands far advanced from their predecessors even five or six years ago. BI users today are not IT managers or data scientists with advanced analytic skills. Business user today are keen to "mash up" and find insights in new and emerging types of big data from all sorts of data sources.
Data gathered can be structured or semi-structured data such as CRM systems and spreadsheets as well as real time unstructured data from sources like Facebook updates, tweets, forum posts, just to name a few.
Sign up for CIO Asia eNewsletters.