"One of the biggest pitfalls for a program is disconnect between IT and the business on who controls strategy and initiatives," Arnold says. "In less mature organizations there is no documented strategy, a hodgepodge of tools are in production, and decision makers favor intuition for charting strategic direction. These types of firms are mostly unaware of the asset value of data."
Business leaders and IT professionals can ensure their big data project is successful by carefully identifying objectives, needs, and requirements; calculating a return on their investment; mapping analytical capabilities to business/mission needs; and installing a mechanism for continuous feedback, DRC's Chabot says. "A big data project should be divided into multiple phases, incrementally adding value to the organization," he says.
But getting IT and business leaders to agree, as well as getting departments to work together on data initiatives is not always easy.
"In my experience, for the major companies this is becoming a real corporate challenge," Venture Development Center's Stryker says. "Does the job responsibility associated with chief data officer rest within the IT department, the marketing department, the risk management department, or do each of these departments have their own big data initiatives and coordinate with each other?"
Companies also need to bring in the necessary expertise to exploit big data technologies such as Hadoop, which has enabled low-cost, computationally efficient management of very large data sets and analysis tasks.
"The paradigm shift to big data introduces a new role in the corporate organization, the data scientist," Caserta says. "This role requires deep understanding of advanced mathematics, system engineering, data engineering, and [business] expertise." In practice, it's common to use a data science team, where statisticians, technologists, and business subject matter experts collectively solve problems and provide solutions, he says.
Many of the people already working in data analytics will need to prepare for culture shock, Caserta says.
"Before a big data project is launched, a strategic readiness test should be performed to assess the adoption of the new paradigm," he says. Business analysts will need to be retrained or repurposed. The goal of shifting to a big data platform may include changing from reactive analysis (for example, how well a campaign worked), to predictive (what should the next campaign offer), he says, "because now we can proactively influence nonbuyers to follow behavior patterns of loyal customers; or restimulate active customers when their behavior pattern begins to look like a lost customer."
What are the risks of not building a strong, cohesive big data strategy? Launching expensive endeavors that fail to deliver on their promise.
"Typically big data projects are multidimensional and complex initiatives," Chabot says. "They require significant upfront planning." Before embarking on a big data project, he says, organizational leaders should ensure alignment between strategic, functionality, data, analytics, and technology road maps. These road maps need to be reflected in a business, system, software, data, and technology architecture.
Sign up for CIO Asia eNewsletters.