"Big data" has arrived as a big business initiative. But the hip, experimental, ad hoc veneer of blending data streams to surface bold discoveries belies a massive cultural and technological undertaking not every organization is ready for.
Without a strategic plan that includes coherent goals, strong data governance, rigorous processes for ensuring data accuracy, and the right mentality and people, big data initiatives can easily end up being a big-time liability rather than a valuable asset.
Following are five strategic tips for avoiding big data failure. In many cases, the advice pertains to any data management project, regardless of the size of the data set. But the advent of massive data stores has brought with it a particular set of pitfalls. Here's how to increase the chances that your organization's urge to mix large data pools from disparate sources is a success.
Rearden Commerce CTO Phil Steitz succinctly sums up the single most important driver of big data success: You must integrate analytics and data-driven decision making into the core of your business strategy.
"If 'big data' is just a buzzword internally, it becomes a solution looking for a problem," Steitz says.
For Reardon Commerce, whose e-commerce platform leverages big data and other resources to optimize the exchange of goods, services, and information between buyers and sellers, the concept of "absolute relevance" -- putting the right commercial opportunity in front of the right economic agent at the right time -- is key.
"It is an example of this kind of thinking originating and centrally driving strategy at the top of the house," Steitz says.
Part of this approach includes developing a small, high-powered team of data scientists, semantic analysts, and big data engineers, then opening a sustained, two-way dialog between that team and forward-thinking decision makers in the business, Steitz says.
"The biggest challenge in really getting value out of contemporary analytics and semantic analysis technologies is that the technologists who can really bring out what is possible need to be deeply engaged with business leaders who 'get it' and can help winnow out what is really valuable," Steitz says.
Another key success factor in making big data a part of the overall business strategy is effective management of data partnerships.
"Really optimizing customer experience and economic value in today's world inevitably requires sharing data across enterprises," Steitz says. "Naïve approaches to this -- 'just send us your full transaction file nightly' -- fail miserably for operational as well as privacy and security reasons."
Big data projects bring with them significant concerns over security, privacy, and regulatory compliance. Nowhere is this a more sensitive issue than in the health care industry.
Sign up for CIO Asia eNewsletters.