Standing amid the clamor of the Hadoop Summit trade show in San Jose in June, Rakesh Kant couldn’t conceal his annoyance.
As head of enterprise data management and analytics technology for U.S. Bank, Kant wanted to use some of the many emerging Big Data technologies hawked at the show — such as Hadoop, the open source software used to store huge amounts of data in clusters of computers — to help his employer make better use of all the data it collects. But theconfusing morassof open source software and cloud services leaves him unsure how to do it and worried about buying technologies that may become obsolete before a project even launches.
“The industry is evolving more and more experiments that are confusing the market,” Kant said. “We don’t want to spend time choosing things, we want to deploy them.”
Open-source software, which anyone can modify and improve, has enabled an explosion of innovation throughout information technology in the past couple of decades. In particular, Big Data software developed this way helped Yahoo, Google, Facebook and other companies create services used by billions of people. Indeed, these companies and the startups spun out of them created many of those same open-source Big Data software and services precisely to solve problems, such as indexing the Internet to facilitate search, that traditional IT couldn’t.
The hope was that this data-driven open-source software also could unlock massive value inside more traditional corporations, from banks to retailers to manufacturers. But that hasn’t happened nearly as widely as expected. And it’s not likely to happen anytime soon. Gartner Inc. Research Director Nick Heudecker estimates that through 2018, some70 percentof Hadoop deployments won’t produce hoped-for higher revenues or cost savings.
The situation exposes a downside of open source software, which saw earlier blockbuster successes such as the operating software Linux, sold in an enterprise version by Red Hat Inc. and others. Startups hoped to do the same with Hadoop, the data processing software Spark and other software. But they’ve struggled to persuade many of the largest corporations accustomed to integrated systems and support from the likes of IBM Corp., Hewlett-Packard Enterprise Co. and Oracle Corp. Those enterprises don’t want to have to cobble together software from dozens of startups.
“It’s sort of like putting Frankenstein together,” said George Gilbert, Big Data analyst at Wikibon Research, owned by the same company asSiliconANGLE. As a result, added Peter Burris, chief research officer atSiliconANGLE Media, “Big Data pilot projects often are abandoned.” (The analysts, along with SiliconANGLE Media co-Chief Executive Dave Vellante, will explore this and other issues starting Tuesday at the BigDataNYC conference, which will be broadcast on theCUBE, the video unit of SiliconANGLE Media.)
All that means startups that bet on open source as the path to creating the next great software business are starting to sputter. In early August, Hortonworks Inc., one of the few open source Big Data companies that’s publicly held, saw its sharesplunge 25 percentafter missing second-quarter earnings expectations, losing $64.2 million on revenues of $43.6 million. Its top sales executive, President Herb Cunitz, also left the company.
“The open-source-only model isn’t working,” said John Schroeder, chairman of MapR Technologies Inc., which sells its own versions of Hadoop and other big data software. “It only really worked with Red Hat and I don’t think it’ll work again.”
That may be premature. But there are plenty of problems to overcome, especially for large enterprise buyers — chief among them the dizzying array of overlapping software, a situation endemic to open source.