Real-Time-Data

Big Data Moves Toward Real-Time Analysis

Big Data Moves Toward Real-Time Analysis

 

It's clear there's a transformation in enterprise data handling underway. This was evident among the big data aficionados attending the Hadoop Summit, in San Jose, Calif., and the Spark Summit in San Francisco earlier this month.

One phase of this transformation is in the scale of the data being accumulated, as valuable "machine data" piles up faster than sawdust in a lumber mill. Another phase, one that's less frequently discussed, is the movement of data toward near real-time use.

The data warehouse, as valuable as it is, is history. The most valuable data will be that which is collected and analyzed during the customer interaction, not the review afterward. The analysis that counts is not the results of the last three months, or even the last three days, but the last 30 seconds -- probably less.

In the digital economy, interactions will occur in near real-time. Data analytics will need to be able to keep up. Hadoop and its early implementers, such as Cloudera and Hortonworks, have risen to prominence based on their mastery of scale. They gobble data at a prodigious rate, one that was inconceivable a few years ago.

Read Also:
Why we need artificial and business intelligence for higher customer satisfaction

Spark is the new kid on the block, an in-memory system that's not exactly unknown, but is still a stranger in data warehouse circles. IBM said it would pour resources into Spark, an Apache Foundation open source project.

Is it wise to focus as much attention and effort on Spark? The big data field is basically in ferment. There's RethinkDB, an ambitious Redis project or, for that matter, commercial in-memory SAP Hana. With so many initiatives underway, was it wise for IBM to announce that Spark is "potentially the most significant open source project of the next decade"?

At Spark Summit, Amazon Web Services announced a free Spark service running on Amazon Elastic Map Reduce, and IBM announced plans for Spark services on BlueMix (currently in private beta) and SoftLayer. These cloud services will open the floodgates to developers, and IBM’s contributions will surely help to harden the Spark Core for enterprise adoption.

Read Also:
Education and Training in Data Science Offered through University & Industry Partnerships


Data Science Congress 2017

5
Jun
2017
Data Science Congress 2017

20% off with code 7wdata_DSC2017

Read Also:
The platform Solving beacon messaging's spam problem

AI Paris

6
Jun
2017
AI Paris

20% off with code AIP17-7WDATA-20

Read Also:
Education and Training in Data Science Offered through University & Industry Partnerships

Chief Data Officer Summit San Francisco

7
Jun
2017
Chief Data Officer Summit San Francisco

$200 off with code DATA200

Read Also:
A Cloud-Native Approach Democratizes Self-Service BI

Customer Analytics Innovation Summit Chicago

7
Jun
2017
Customer Analytics Innovation Summit Chicago

$200 off with code DATA200

Read Also:
Why we need artificial and business intelligence for higher customer satisfaction

HR & Workforce Analytics Innovation Summit 2017 London

12
Jun
2017
HR & Workforce Analytics Innovation Summit 2017 London

$200 off with code DATA200

Read Also:
Open source data visualization tool by Airbnb: benefits and limitations.

Leave a Reply

Your email address will not be published. Required fields are marked *