Will You Always Save Money with Hadoop?

 

Hadoop-related technologies can save you (lots of) money when it comes to system/infrastructure costs, those cost savings can quickly disappear when you start looking at application/development costs.

It all depends on what analytic data problem youre trying to solve. In this post, well take a look at two Big Data examples and determine, for each example, which platform the enterprise data warehouse (EDW) or Hadoop will be the most cost-effective over time.

These two examples come from WinterCorp’s Big Data – What Does It Really Cost? special report, which introduces the Total Cost of Data (TCOD) framework. The first example is building an enterprise data warehouse, and the second example is building a data refinery. We’ll first look at the requirements for each example, and then compare the costs. Again, the question we want to answer is: Which platform – the EDW or Hadoop – is the most cost-effective over time?

These results may surprise you. Keep in mind that the results are just estimates (because a lot of assumptions have to be made), but these estimates trump anecdotal guesses any day.

The data warehouse platform ($265 million) is far more cost-effective than a Hadoop solution ($740 million). Choosing the data warehouse platform in this case lowers the overall cost by a factor of 2.8. Further analysis shows that you will get essentially the same result for a data warehouse ranging in size from 50 TB to 2 PB.

The development of complex queries and analytics are the dominant cost factors in the example. Of the $44 million estimated for EDW system cost, $10.8 million is the initial acquisition cost – about 4% of the TCOD.

While it is common to focus on the first major outlay in the project—i.e., the acquisition of a platform—the total cost of the project is far more important, and other factors greatly outweigh all the system costs combined.

The purpose of the TCOD framework is to help organizations estimate the total cost of a big data solution for an analytic data problem. It considers two major platforms for implementing big data analytics – the enterprise data warehouse and Hadoop – and helps you understand where each big data platform architecture works best.

The TCOD framework was developed by Richard Winter and his team at WinterCorp, a consultancy focused on large scale data management challenges. WinterCorp introduced the TCOD framework in a 2013 special report called Big Data – What Does It Really Cost?.

So if you’re ready to roll up your sleeves and do the hard work of figuring out what big data really costs, then the TCOD framework is waiting for you.

Share it:
Share it:

[Social9_Share class=”s9-widget-wrapper”]

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

You Might Be Interested In

The Future of Analytics Is Prescriptive, Not Predictive

13 Jul, 2015

  Though predictive analytics and data mining are particularly hot topics at the moment, they’ve been widely used in marketing …

Read more

Data Quality Management Lacking Among Businesses

29 Jan, 2016

Data quality must improve significantly if businesses are to reap benefits, according to a Blazent survey of 200 C-level and …

Read more

Using Data-Driven Intelligence to Stay Ahead in Retail

22 Jul, 2015

What the typical merchant wouldn’t give to be able to read a customer’s mind. The next wave of retail technology …

Read more

Do You Want to Share Your Story?

Bring your insights on Data, Visualization, Innovation or Business Agility to our community. Let them learn from your experience.

Get the 3 STEPS

To Drive Analytics Adoption
And manage change

3-steps-to-drive-analytics-adoption

Get Access to Event Discounts

Switch your 7wData account from Subscriber to Event Discount Member by clicking the button below and get access to event discounts. Learn & Grow together with us in a more profitable way!

Get Access to Event Discounts

Create a 7wData account and get access to event discounts. Learn & Grow together with us in a more profitable way!

Don't miss Out!

Stay in touch and receive in depth articles, guides, news & commentary of all things data.