Hortonworks Inc. Chief Executive Rob Bearden grabs a napkin off the hotel conference room table at the San Jose Marriott and scrawls a bell curve to illustrate the typical adoption pattern for new technologies.
The technologies in question here are Hadoop, a way of storing huge amounts of data across clusters of computers, and related Big Data analysis software sold by Hortonworks and other companies. Just past the curve’s initial ascent, where tech-savvy companies have started buying and the curve arcs upward, Bearden draws an X. That’s his way of showing that mainstream customers are just starting to adopt the software in large numbers — similar to a curve for the enterprise resource planning (ERP) software in the early 1990s that made giants out of Oracle Corp., SAP SE and others. Big Data software, he declares, is “tracking on the exact same route.”
Or is it? Big Data technologies led by Hadoop,developed originally by Yahooto handle its massive Web index, have taken 10 years to get to this point. Yet the largest companies that bought billions of dollars worth of ERP software have yet to crack open big budgets for the new wave of software.
The upshot: It’s not clear when or how the new software, which in alliance with cloud computing has the potential to reinvent every operation inside corporations, willcross the chasminto mainstream use. “This is Big Data’s come-to-Jesus year,” said George Gilbert, an analyst at Wikibon Research, owned by the same company asSiliconANGLE.
Ditto for software companies such as Hortonworks,Cloudera Inc.andMapR Technologies Inc. They’re currently growing fast by selling services, subscriptions to product support, and related software for Big Data systems. Hortonworks, the only public company among them, saw revenues jump 85 percent in the first quarter, to $41.3 million. And Wikibon expectsBig Data software and services revenuesto grow from $19.1 billion this year to $29 billion in 2018.
But Hortonworks’ first-quarter loss of $65.8 million highlights the challenges these companies face in their implicit quest to become the next Oracle or SAP. The new software industry is dominated by free open source software such as Hadoop, which means their customers hold most of the cards andprofits are thin to nil. At the same time, the companies are now getting squeezed by cloud computing companies such as Amazon.com and existing software companies.
There should be plenty of demand. Wrangling huge amounts of data has long been key to businesses such as Yahoo, Google, Facebook, LinkedIn and other Internet companies, which is why they developed many Big Data technologies on their own before making them available via open source software. Now, retailers, manufacturers, advertisers and financial services companies need to corral and analyze data so they can better target customers with offers, track credit card fraud and find the best places to drill for petroleum. “Data is the new oil,” Bearden said in an interview at the company’s recent Hadoop Summit conference (* disclosure below) in San Jose, Calif.
But many customers are finding it tough to put these systems into full production. One problem is, in a sense, too much innovation. That’s partly the nature of open source software, which is developed by decentralized legions of programmers under theApache Software Foundation. And in many ways, that’s a good thing.
Hadoop and dozens of related open source Big Data software projects such as Spark and Kafka have enabled companies to find value in a wealth ofunstructured datasuch as text, web pages and video. That data used to be thrown away because it couldn’t be stored in traditional databases or analyzed with limited computing power. Not anymore.