Actually deriving meaningful information means consolidating data from multiple data sources. That sounds a lot like data consolidation, data lakes, and BI, doesn't it?
The talent gap
If Gartner had been right about Hadoop stalling, it wouldn't be such a problem to find people with Hadoop skills. I can tell you recruiting experienced Hadoop people is not easy, and once you train them, you have to pay them well and keep them engaged as 1,000 recruiters descend upon them.
According to AtScale's survey, 61 percent of respondents view the talent pool as the biggest challenge to adoption. This doesn't change once customers adopt Hadoop, although they find that management, security, performance, governance, and accessibility are bigger issues than they realized.
A brighter picture
In the AtScale study, 49 percent of respondents have already achieved value and 45 percent are optimistic about achieving it. Only 6 percent are pessimistic, and 3 percent plan to use Hadoop less in the future. Are these numbers too rosy to be believed? Probably.
Nonetheless, I'll bask in the glow of a survey that reflects what I've seen in the field: Brisk and constant adoption -- mainly for consolidation -- and a rapidly maturing technology that people get value from.
AtScale plans to do a follow-up survey. I hope to see a deeper view of companies that never bought Teradata and Netezza, how companies are using Spark, and whether they are buying hardware for Hadoop. Meanwhile, I'd love to hear from others in the Hadoop community about adoption levels, challenges, and opportunities going forward.
Sign up for CIO Asia eNewsletters.