Hortonworks executives say a new wave of data is fueling the need for new platforms like Hadoop. Web sessions, social media interactions and machine sensors generate massive amounts of data, but they don't fit neatly into traditional relational enterprise databases, hence the rise of NoSQL database platform.
In the past, most of that information handled by these databases fell to the floor and was never picked up. Now, companies like Neustar realize they can actually do something with the data, if they can manage it. Hortonworks Data Platform (HDP) is the name of the company's distribution and it's 100% open source Apache Hadoop code, compiled by Hortonworks and shipped as an enterprise software kit meant to run on top of commodity hardware.
Hortonworks deployments have been mostly focused thus far on supplementing existing data warehousing tools, usually SQL databases, says David McJannet, vice president of marketing for Hortonworks, who just recently joined the company after being at VMware and Microsoft. HDP can be used in conjunction with traditional platforms to manage new unstructured data that organizations usually don't have a good way of managing today. The data can be managed by Hadoop and either feed directly into analytics tools that sit on top of Hadoop, or feed it back into more traditional SQL-style databases that the enterprise may already have.
The data processed by Hadoop can be extremely valuable for companies. Retailers - from hardware and grocery stores - to e-commerce sites, can log significantly more data about each individual visitor to their site, their patterns and history, all in an effort to serve them better. Hadoop thrives in scaling out horizontally to massive sizes without impacting performance.
And now some of the biggest names in technology are buying into the platform too. Hortonworks has been on somewhat of a partnership spree during the past few months. First Teradata and Microsoft announced they would incorporate HDP into their analytics offerings. Then Rackspace announced that Hortonworks would be used to explore a Hadoop-as-a-Service type offering on its OpenStack-powered public cloud. Hortonworks has since joined OpenStack, the open source cloud management platform.
"They approached us," McJannet says about how the Hortonworks-Microsoft partnership began. Microsoft has integrated HDP into its business intelligence products, specifically HD Insights Server. In doing so, Microsoft has begun contributing back to the open source community too. Microsoft was the first to enable Hadoop to run on Windows - it previously only worked on Linux - and a Microsoft engineer chairs the Apache Hadoop project now. Matthew Aslett, an analyst at the 451 Research Group says Microsoft blessing Hadoop, and specifically Hortonworks, could expose the big data platform to the enterprise masses.
Sign up for CIO Asia eNewsletters.