BN-IX327_0615_c_G_20150614125141

IBM Wants to Push Spark, Real-Time Big Data Tool, Into Mainstream

IBM Wants to Push Spark, Real-Time Big Data Tool, Into Mainstream

 

International Business Machines Corp. has thrown its weight behind Spark, an increasingly popular tool that is used to analyze large amounts of data in real time. The number of such uses is expected to greatly expand as Internet-connected sensors become pervasive in physical objects, a phenomenon known as the Internet of Things. By working to push Spark into the mainstream now, IBM hopes to catch an emerging market at the very early stages.

IBM isn’t the first company to embrace Spark, and Spark isn’t the only tool that can be used to analyze real-time data. But given the scale of IBM and its customer base, its support for the technology, already one of the fastest-growing open-source software projects, could be significant.

Spark, which emerged from the University of California at Berkeley in 2009, addresses some of the limitations of Hadoop, an older open-source software framework that employs a distributed architecture to analyze large amounts of data. Hadoop is less suited for analyzing data in real time, and it has a reputation for being tricky for developers to use, according to Bob Picciano, senior vice president of IBM Analytics. Gartner Inc. has characterized the adoption of Hadoop as steady but slow, with 26% of surveyed companies saying they had a Hadoop project underway, and 46% saying they expected to within a few years. Some users have found it difficult to scale, as the Wall Street Journal has reported.

Read Also:
Data Insight Expert Interview: Jim Brooks [video]

IBM is expected to  formally announce Monday that it will embed Spark into its analytics and commerce platforms, and to offer Spark as a service on its Bluemix development platform. IBM also is expected to announce that it will assign more than 3,500 engineers and developers to work on Spark-related projects around the world, and that it will donate its IBM SystemML machine learning technology to the Spark open-source ecosystem. It also is expected to launch a Spark technology center in San Francisco and help educate data scientists and engineers in the use of Spark on a mass basis.

Spark is part of a broader group of tools and companies that are pushing the frontiers of data analytics

Several startups are working on analytics technology that approaches real-time, too. “What we can do is know everyone who is on the call right now, and analyze whether they were at Starbucks yesterday,” said J. Andrew Rogers, the founder and CTO of a startup in Seattle called SpaceCurve Inc. Mr. Rogers, a database expert who said he worked on large-scale analytic and geospatial systems with Google Earth, developed new technology for SpaceCurve.

Read Also:
Big Data to accelerate Tour de France coverage

Bottlenose Inc., a startup based in Los Angeles, has developed technology that helps companies analyze what co-founder Nova Spivack calls “streaming data.” It can be used to spot real-time trends on social media, with applications in areas such as business, politics and government.

IBM has bet on Spark from the very beginning. It was one of the founders of the Berkeley lab where Spark was first developed.



Chief Analytics Officer Europe

25
Apr
2017
Chief Analytics Officer Europe

15% off with code 7WDCAO17

Read Also:
Big Data to accelerate Tour de France coverage

Chief Analytics Officer Spring 2017

2
May
2017
Chief Analytics Officer Spring 2017

15% off with code MP15

Read Also:
What are the Challenges of the Analytics of Things?

Big Data and Analytics for Healthcare Philadelphia

17
May
2017
Big Data and Analytics for Healthcare Philadelphia

$200 off with code DATA200

Read Also:
How the Internet of Things is changing healthcare and transportation

SMX London

23
May
2017
SMX London

10% off with code 7WDATASMX

Read Also:
Surprising Data Warehouse Lessons from a Scrabble Genius
Read Also:
How Airbnb Uses Big Data And Machine Learning To Guide Hosts To The Perfect Price

Data Science Congress 2017

5
Jun
2017
Data Science Congress 2017

20% off with code 7wdata_DSC2017

Read Also:
Predicting Patient Experience with Narrative Data: A Healthcare Goldmine

Leave a Reply

Your email address will not be published. Required fields are marked *