Besides that, the shift to Hadoop might seem daunting as it is a relatively new technology. To overcome this, Cutting advised organisations to start by applying Hadoop to a business process or project which its success can be easily measured. "People oftentimes want to start with something complicated like predictive analytics but they should start with something simple. For instance, a company can bring in some of the workloads that are running in their data warehouse into Hadoop to see if doing so results in cost savings. At the same time, the employees will gain hands-on experience using Hadoop. This may not be an exciting application of Hadoop but you end up with measurable benefits and gain a better understanding of the software."
However, Hadoop experimentation calls for support from the management level and IT teams. "The management team needs to understand the value of data and permit such experiments to happen. As for the IT teams, they should build sandboxes to enable business units to experiment, and provide the relevant tools. By being involved earlier in projects, IT teams will be able to better control what technology/tools are being used, and help ensure that the projects are maintainable," explained Cutting.
Future of Hadoop
When asked about the future of Hadoop, Cutting said that it might "replace relational database systems and become the default place to build apps that store processed data."
"Hadoop was initially used only for batch processing but it has vastly improved over the past decade, with new functionalities and capabilities being added to it rapidly. Today, Hadoop can support structured query language (SQL) to allow users to query Hadoop data with a search query engine. You can also lay open-source cluster computing framework Spark on top of Hadoop to analyse streaming data in real-time for predictive analytics or machine learning. Since Hadoop is an open platform, it will continue to incorporate new projects/features so I forsee Hadoop becoming more powerful over the next few years," concluded Cutting.
Sign up for CIO Asia eNewsletters.