Hortonworks and Hewlett Packard Enterprise, aka HPE, announced a joint initiative to produce new enhancements to Apache Spark, specifically to handle a new class of analytic workloads from large pools of shared memory.
So far, the collaboration has led to enhanced shuffle engines for faster sorting and in-memory computations, and improved memory utilization.
"We're hoping to enable the Spark community to derive insight more rapidly from much larger data sets without having to change a single line of code," said Martin Fink, EVP and CTO, Hewlett Packard Enterprise, and Hortonworks board member, in the announcement.
"We're very pleased to be able to work with Hortonworks to broaden the range of challenges that Spark can address."
The two companies plan to contribute the resulting tech to the Apache Spark community.
Hortonworks also announced enhancements to its Connected Data Platforms.;
Chief Analytics Officer Europe
15% off with code 7WDCAO17
Chief Analytics Officer Spring 2017
15% off with code MP15
Big Data and Analytics for Healthcare Philadelphia
$200 off with code DATA200
10% off with code 7WDATASMX
Data Science Congress 2017
20% off with code 7wdata_DSC2017