The only other happy party in this new world are recruiters, who are able to reap rich rewards for bagging unicorns the fabled data scientists who possess a mastery of statistics, PhDs in computer science, and untold experience with Python, Hadoop, MapReduce, JSON and Hive literally, the stuff of legends.
Business managers don't want to worry about how to take advantage of YARN. They don't want to learn the meaning of new phrases like Hive, Spark, data reservoirs, data lakes, all of which now populate the tech discourse. They don't want to have to ask IT to write a query or to merge in additional data sets. In short, they don't want another system with lots of moving parts. They just want a simple tool that they can use to get answers, as quickly and painlessly as possible.
At the end of the day, Hadoop 2.0 remains a framework for programmers. As necessary as low level technologies are, it's time we shifted our attention to the end user for whom low level technologies are as interesting as the wiring inside their office walls.
Hadoop may ultimately enable a renaissance in the user experience, but it hasn't so far. After several years of hype, you can't blame business users who sometimes feel that Hadoop is a white elephant. Let's focus on business user oriented software — software that allows users to easily access and analyze unlimited amounts of data from various sources, on their own and without numerous degrees or the overhead of the traditional stack. Only then, will business users see the full value of their data.
Sign up for CIO Asia eNewsletters.