3 key ways Hadoop is evolving

3 key ways Hadoop is evolving

The Strata+Hadoop World 2015 conference in New York this week was subtitled "Make Data Work," but given how Hadoop world's has evolved over the past year (even over the past six months) another apt subtitle might have been "See Hadoop Change."

Here are three of the most significant recent trends in Hadoop, as reflected by the show's roster of breakout sessions, vendors, and technologies.

Spark is so hot it had its own schedule track, labeled "Spark and Beyond," with sessions on everything from using the R language with Spark to running Spark on Mesos.

Some of the enthusiasm comes from Cloudera -- a big fan of Spark  -- and its sponsorship for the show. But Spark's rising popularity is hard to ignore.

Spark's importance stems from how it offers self-service data processing, by way of a common API, no matter where that data is stored. (At least half of the work done with Spark isn't within Hadoop.) Arsalan Tavakoli-Shiraji, vice president of customer engagement for Databricks, Spark's chief commercial proponent, spoke of how those tasked with getting business value out of data "eagerly want data, whether they're using SQL, R, or Python, but hate calling IT."

Read Also:
What Happens When You Combine Artificial Intelligence and Satellite Imagery

Rob Thomas, IBM's vice president of product development for IBM Analytics, cited Spark as a key in the shift away from "a world of infrastructure to a world of insight." Hadoop data lakes often become dumping grounds, he claimed, without much business value that Spark can provide.

"Instead of hiring 500 IT ops people to handle [Hadoop] clusters, I'd rather hire 250 data scientists to get something meaningful out of the data," Thomas said.

The pitch for Hadoop is no longer about it being a data repository -- that's a given -- it's about having skilled people and powerful tools to plug into it in order to get something useful out.

Two years ago, the keynote speeches at Strata+Hadoop were all about creating a single repository for enterprise data. This time around, the words "data lake" were barely mentioned in the keynotes -- and only in a derogatory tone. Talk of "citizen data scientists," "using big data for good," and smart decision making with data was offered instead.

Read Also:
Data Science Talent is Key to Analytical Innovators


Sentiment Analysis Symposium

27
Jun
2017
Sentiment Analysis Symposium

15% off with code 7WDATA

Read Also:
Big Data's Next Big Thing: Sports Training and Personalized Medicine

Data Analytics and Behavioural Science Applied to Retail and Consumer Markets

28
Jun
2017
Data Analytics and Behavioural Science Applied to Retail and Consumer Markets

15% off with code 7WDATA

Read Also:
5 Ways Artificial Intelligence Is Already Changing Government

AI, Machine Learning and Sentiment Analysis Applied to Finance

28
Jun
2017
AI, Machine Learning and Sentiment Analysis Applied to Finance

15% off with code 7WDATA

Read Also:
Lambda Architecture for Big Data Systems

Real Business Intelligence

11
Jul
2017
Real Business Intelligence

25% off with code RBIYM01

Read Also:
Smart drugs: Where IoT meets healthcare, a market snapshot

Advanced Analytics Forum

20
Sep
2017
Advanced Analytics Forum

15% off with code Discount15

Read Also:
What Happens When You Combine Artificial Intelligence and Satellite Imagery

Leave a Reply

Your email address will not be published. Required fields are marked *