Subscribe / Unsubscribe Enewsletters | Login | Register

Pencil Banner

Hadoop is probably as mature as it's going to get

James Kobielus | March 12, 2015
Five years ago, Hadoop came roaring into the mainstream as the solutions to all big data problems. Now that reality has settled in, it's time for a more realistic assessment.

Related to that "hammering the same old themes" trend is the matter of Hadoop's still-blurry market scope. As I stated in this Dataversity column from last April, Hadoop still has no clear boundaries (vis-à-vis NoSQL and other big data approaches), which was essentially what I had said three years previously in my Forrester days. Then and now, the Hadoop industry's "identity crisis" stems in part from the group's lack of standardization and failure to coalesce a unifying vision for what Hadoop is and can evolve into.

If you look at the Apache Software Foundation's definition of Hadoop now, it still feels like a catch-all rather than a definitive architecture. For example, the recent inclusion of Spark into the scope of Hadoop feels as arbitrary as continuing to include Cassandra. Nobody in the industry seriously considers Spark anything other than a competitor to Hadoop, not a component of it. By contrast, Cassandra isn't even the hottest open source, real-time, big data community out there, and its growth days seem to have waned considerably.

Also, you sense that a segment is starting to saturate its target market when discussions increasingly focus on its still-puny adoption rate among mainstream users. That's front and center in the cited article's discussion of its survey findings:

[InformationWeek's] data suggests that train hasn't left the station just yet: Just 4% of companies use Hadoop extensively, while 18% say they use it on a limited basis...That is up from the 3% reporting extensive use and 12% reporting limited use of Hadoop in our survey last year. Another 20% plan to use Hadoop, though that still leaves 58% with no plans to use it.

If you've been in the analytics industry for more than a handful of years, this smacks of déjà vu. More than two decades into its existence as a discrete segment, the business intelligence (BI) market continues to agonize over low adoption rates among mainstream knowledge workers. Perhaps BI -- or Hadoop or any other big data segment -- was not fated to be as ubiquitously adopted as, say, smartphones.

That doesn't mean Hadoop can't develop into a hugely important and lucrative segment within its own well-defined niche. After all, nothing's stopping a mature person from growing rich and popular as their hair fades to gray.

Source: Infoworld


Previous Page  1  2 

Sign up for CIO Asia eNewsletters.