Why Good Data Scientists are Worth the Big Bucks

Why Good Data Scientists are Worth the Big Bucks

Data scientist is the ‘Sexiest Job of the 21 Century’, so say Thomas Davenport and DJ Patil in their seminal 2012 Harvard Business Review article.

Since then, data scientists have been very much in demand and their wages have soared. Compared to statisticians, computer programmers and AI guys (who are still regarded as geeks), they are extremely well paid.

But are they worth it? Is there a good reason why data scientists should get paid more or are their pay packets over-inflated due to unnecessary hype?

According to academic research by Anders Ericsson and popularised by Malcolm Gladwell in his book ‘Outliers: The Story of Success’, that’s how long it takes to become an expert in something.

Well actually it’s a bit too general. 10,000 hours is an average across lots of disciplines including sport, business, education, music, etc.. Obviously it’s not going to take you 10,000 hours to be an expert in tiddlywinks, snap or noughts and crosses, but if you want to be an expert in 3D chess, quantum physics or neuroscience, then it probably will.

Ericsson also points out that the quality of the practice time is important too. Reading general anatomy books for 10,000 hours will not make you an expert in open heart surgery, but making specific improvements to your knowledge, technique and experience on a daily basis will.

So let’s say – for the sake of argument – that 10,000 hours is what it takes to be an expert in any serious academic pursuit.

In a typical work year of 2,000 hours, then, it’ll take you around 10 years to become an expert.

Yes, my maths is correct(ish) – I’m factoring in how much time you spend on emails, ebay, funny cat videos on TwitFace and the worthless one-hour meetings that interrupt at regular intervals.

So there you have it – dedicate yourself to your chosen discipline and you’ll be an expert in around 10 years.

Well OK, it might take 10 years to become an expert, but it’ll take much less than that to become pretty good at something. But how long?

This is where the Pareto Principle comes in handy.

According to the Pareto Principle, also known as ‘The 80/20 Rule’, you can get 80% of the required results by putting in 20% of the effort, so according to Pareto you can get 80% of the way towards being an expert in 20% of the time. In terms of Ericsson’s 10,000 hours, that’s 2,000 hours or 2 years.

That’s pretty good, really. It means that you only need to work hard for 2 years to be a nearly-expert.

When I look back over my career I sort of recognise this. After a couple of years I’m beginning to get pretty good at something, after which, improvements come much more slowly – according to the flip-side of Pareto’s coin, the final 20% of the way to becoming an expert takes 80% of the effort.

But what does this really mean? If it only takes 20% of the time to attain 80% of the knowledge, then surely the stuff that defines an expert must lie in the remaining 20% that you haven’t reached.

So working for the first 20% of the time gets you strong foundations, but doesn’t qualify you as an expert.

Making good decisions based on your knowledge and experience requires a strong foundation, but making consistently better, more informed decisions requires more than this, it requires expert knowledge and an instinct about your problem domain.

To illustrate this, consider the case – featured in Gladwell’s book – of a lieutenant firefighter who claimed that ESP saved his life and those of his crew. The firefighter’s team was fighting what appeared to be a simple kitchen fire in a one-story house. The lieutenant led his crew into the back of the building to spray water onto the fire, but the fire just kept roaring back. He thought the water should have had more of an impact so he ordered his men back to the living room to regroup. Then, feeling that something wasn’t right, he ordered his men to evacuate the building. As soon as they left, the floor they were standing on collapsed.

 

Share it:
Share it:

[Social9_Share class=”s9-widget-wrapper”]

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

You Might Be Interested In

Machine learning is all the rage with Big Data developers

23 Jul, 2016

Machine learning has advanced to the point where it more or less goes hand-in-hand with Big Data. Indeed, so popular …

Read more

OpenDefra: 8,000+ open datasets released in one year

5 Jul, 2016

On Tuesday the Department for Environment, Food & Rural Affairs (Defra) will announce the results of OpenDefra, its ambitious project …

Read more

5 requirements for success with DataOps strategies

23 Jul, 2019

Spring is traditionally a time to commence Spring cleaning. It’s also a great time for IT teams to look at …

Read more

Recent Jobs

Cyber Security Engineer – P2

Hybrid (Aurora, CO, USA)

5 Mar, 2024

Read More

Sr. Manager – Data and Analytics Technical Lead

Hybrid (Dedham, MA, USA)

5 Mar, 2024

Read More

Manager, Business Data and Analytics

Hybrid (Troy, OH, USA)

5 Mar, 2024

Read More

Do You Want to Share Your Story?

Bring your insights on Data, Visualization, Innovation or Business Agility to our community. Let them learn from your experience.

Get the 3 STEPS

To Drive Analytics Adoption
And manage change

3-steps-to-drive-analytics-adoption

Get Access to Event Discounts

Switch your 7wData account from Subscriber to Event Discount Member by clicking the button below and get access to event discounts. Learn & Grow together with us in a more profitable way!

Get Access to Event Discounts

Create a 7wData account and get access to event discounts. Learn & Grow together with us in a more profitable way!

Don't miss Out!

Stay in touch and receive in depth articles, guides, news & commentary of all things data.