Credit: Thinkstock via CIO
In just a few short years, big data technologies have gone from the realm of hype to one of the core disruptors of the new digital age. 2014 saw big data initiatives inside the enterprise increasingly move from test to production. In 2015, big data will push further into the enterprise with even more use cases -- specifically real-time use cases -- says John Schroeder, CEO and co-founder of Hadoop distribution specialist MapR.
"This is the year that organizations move big data deployments beyond initial batch implementations and into real time," Schroeder says. "This will be driven by the realization of the huge strides that existing industry leaders and soon-to-be new leaders have already made by incorporating new big data platforms into their analytics with "in-flight" data to impact business as it happens."
Schroeder says five major developments will dominate 2015.
1. Data Agility Emerges as a Top Focus
Data agility has been one of the big drivers behind the development of big data technologies, as the processes around legacy databases and data warehouses have proven too slow and inflexible for many business needs. In 2015, Schroeder says data agility will become even more central as organization shift their focus from simply capturing and managing data to actively using it.
"Legacy databases and date warehouses are so expensive that DBA resources are required to flatten summarize and fully structure the data," he says. "Upfront DBA costs delay access to new data sources and the rigid structure is very difficult to alter over time. The net result is that legacy databases are not agile enough to meet the needs of most organizations today."
"Initial big data projects focused on the storage of target data sources," he adds. "Rather than focus on how much data is being managed, organizations will move their focus to measuring data agility. How does the ability to process and analyze data impact operations? How quickly can they adjust and respond to changes in customer preferences, market conditions, competitive actions and the status of operations? These questions will direct the investment and scope of big data projects in 2015."
2. Organizations Move from Data Lakes to Processing Data Platforms
In some ways, 2014 was the year of the data lake (or data hub), an object-based storage repository that stores raw data in its native format -- whether structured, unstructured or semi-structured -- until it's ready for use. Data lakes have a strong value proposition in that they represent a scalable infrastructure that's economically attractive (with a reduced per-terabyte cost) and extremely agile.
Schroeder says that the data lake will continue to evolve in 2015 with the capability to bring multiple compute and execution engines to the data lake to process the data in-place. That's not only more efficient, it creates a single point of governance and a single point of security.
Sign up for CIO Asia eNewsletters.