google-servers-datacenter

Google Launches Cloud Dataproc, A Managed Spark And Hadoop Big Data Service

Google Launches Cloud Dataproc, A Managed Spark And Hadoop Big Data Service

Google is adding another product in its range of big data services on the Google Cloud Platform today. The new Google Cloud Dataproc service, which is now in beta, sits between managing the Spark data processing engine or Hadoop framework directly on virtual machines and a fully managed service like Cloud Dataflow, which lets you orchestrate your data pipelines on Google’s platform.

Greg DeMichillie, director of product management for Google Cloud Platform, says Dataproc users will be able to spin up a Hadoop cluster in under 90 seconds — significantly faster than other services — and Google will only charge 1 cent per virtual CPU/hour in the cluster. That’s on top of the usual cost of running virtual machines and data storage, but as DeMichillie noted, you can add Google’s cheaper preemptible instances to your cluster to save a bit on compute costs. Billing is per-minute, with a 10-minute minimum.

Because Dataproc can spin up clusters this fast, users will be able to set up ad-hoc clusters when needed and because it is managed, Google will handle the administration for them.

Read Also:
Startup Crunches 100 Terabytes of Data in a Record 23 Minutes

Because the service uses the standard Spark and Hadoop distributions (with a few tweaks), it’s compatible with virtually all existing Hadoop-based products, and users should be able to easily port their existing workloads over to Google’s new service.



Data Science Congress 2017

5
Jun
2017
Data Science Congress 2017

20% off with code 7wdata_DSC2017

Read Also:
Analytics for the Masses: Five Things to Consider

AI Paris

6
Jun
2017
AI Paris

20% off with code AIP17-7WDATA-20

Read Also:
4 Business Models for the Data Age

Chief Data Officer Summit San Francisco

7
Jun
2017
Chief Data Officer Summit San Francisco

$200 off with code DATA200

Read Also:
Why predictive analytics will shape the future of every sector

Customer Analytics Innovation Summit Chicago

7
Jun
2017
Customer Analytics Innovation Summit Chicago

$200 off with code DATA200

Read Also:
What does intent data mean for the data-driven marketer?

Big Data and Analytics Marketing Summit London

12
Jun
2017
Big Data and Analytics Marketing Summit London

$200 off with code DATA200

Read Also:
Startup Crunches 100 Terabytes of Data in a Record 23 Minutes
Read Also:
Are You Monetizing Information?

Leave a Reply

Your email address will not be published. Required fields are marked *