SQL on Hadoop benchmarks get serious

SQL on Hadoop benchmarks get serious

SQL on Hadoop benchmarks get serious

Unless a "neutral" third party publishes them, we tend to view benchmarks as self-serving exercises that vendors typically stack in their own favor. But recent benchmarks issued by Cloudera and Hortonworks for their SQL on Hadoop engines point to something serious going on. In an era of Spark hype, SQL remains table stakes for Hadoop platforms.

Yes, you can perform machine learning, model customer ecosystems as social graphs, run streaming, and conduct sentiment analysis, but for most organizations, the first question they often ask is how fast is the interactive SQL. Using Hadoop only for SQL query might seem like a waste, given its appeal to R or Python developers. But getting buy-in requires satisfying the BI crowd, because in many organizations, SQL's the gateway drug to Hadoop.

And looking at the benchmarking press releases, you get a sense of who's afraid of whom. For Cloudera, it's Amazon. Competitive benchmarks pitted Impala 2.6, Cloudera's SQL-on-Hadoop MPP engine, against Amazon Redshift columnar analytic database. The results, announced a couple weeks back at Strata, showed Impala performing four to 10x faster on either S3 (which Redshift doesn't use) or EBS (which it does).

Read Also:
How to Write a Data Analysis Report

Cloudera is stating that now even a database that is decoupled from storage (Impala) can perform better than one that followed a traditional tightly coupled data warehouse architecture (Redshift). It's a shot across the bow, given if you want consistent SLAs, high concurrency, or support of very complex SQL syntax, conventional wisdom has been to use a database rather than Hadoop. Cloudera's results don't change that reality, but they do show results in the ballpark with Redshift. And they get the results using AWS's default S3 storage.

But Cloudera's underlying message is not just that Impala has been tuned to go faster. It knows that, while only a minority of customers are deploying to the cloud today, in the long term, the writing is on the wall.

 



Chief Analytics Officer Spring 2017

2
May
2017
Chief Analytics Officer Spring 2017

15% off with code MP15

Read Also:
Can IT keep up with big data?
Read Also:
Can Big Data Increase Voter Turnout?

Big Data and Analytics for Healthcare Philadelphia

17
May
2017
Big Data and Analytics for Healthcare Philadelphia

$200 off with code DATA200

Read Also:
Riding The Competitive Intelligence Automation Wave

SMX London

23
May
2017
SMX London

10% off with code 7WDATASMX

Read Also:
Riding The Competitive Intelligence Automation Wave

Data Science Congress 2017

5
Jun
2017
Data Science Congress 2017

20% off with code 7wdata_DSC2017

Read Also:
EHR Data, Machine Learning Create Cost-Based Clinical Pathways

AI Paris

6
Jun
2017
AI Paris

20% off with code AIP17-7WDATA-20

Read Also:
How Thomson Reuters Improves Customer Experience with Data

Leave a Reply

Your email address will not be published. Required fields are marked *