data-scientist-venn-diagram1

Big data, data science and machine learning explained

Big data, data science and machine learning explained

Data are considered the new secret sauce, are everywhere and have been the cornerstone for the success of many high-tech companies, from Google to Facebook.

But we always used data, there are examples from the ancient times dated thousands of years ago. In the latest centuries data started to find more and more practical applications thanks to the emergence of statistics and later by the Business Intelligence. The earliest known use of the term “Business Intelligence” is by Richard Millar Devens in 1865. Devens used the termto describe how a banker gained profit by receiving and acting upon information about his environment, prior to his competitors.

It is after the WWII that the practice of using data-based systems to improve business decision-making – surely driven by advances in automatic computing systems and storage possibilities – started to take off and be used widely. Digital storage becomes more cost-effective for storing data than paper and since then, an unbelievable amount of data have been collected and organised in data warehouses, initially in structured formats. The term Big Data started to be used meaning just a lot of data.

Read Also:
The Enterprise of the Future Will Need Connected Big Data

In a 2001 research report and related lectures, analyst Doug Laney defined data growth challenges and opportunities as being three-dimensional, i.e. increasing

Gartner, and now much of the industry, quickly picked this  “3Vs” model for describing Big Data which, a decade later, has become the generally accepted three defining dimensions of big data.

Especially critical is here that the Big Data focus is not primarily about the size but all 3 aspects, the characteristics of the data nonetheless its variety.

Relational Databases require ‘pristine data’. If the data is in the database then it is accurate, clean and 100% reliable. A huge amount of time, money and accountability is put on to making sure the data is well prepared before loading it in to the database.

Big Data tackles this problem from the other direction. The data are poorly defined, much of it may be inaccurate and much of it may in fact be missing. Big Data has to have enough volume so that the amount of bad data or missing data becomes statistically insignificant.

Read Also:
Why Data Scientists Create Poor Data Products? 5 Humbling Lessons

The abundance of available data means also that the trend was shifting from Business Intelligence (inherently  descriptive statistics ) where data is used to measure things, detect trends, etc.. to the use of  inductive statistics to infer laws from large sets of data  to reveal patterns, relationships and dependencies, or to perform predictions of outcomes and behaviours.

In the world of data this new interdisciplinary field  is called Data Science.

Data Science is all about extracting knowledge from data, either structured or unstructured, and incorporates many diverse skills such as mathematics, statistics, artificial intelligence, computer programming, visualisation, image analysis, and much more.

The term “data science” has existed for over thirty years and was variously used interchangeably for data analysis or data mining (i.e., the process of discovering patterns in large data sets, like when you mine a mountain of data and your goal is to find the nuggets of insight) but gradually started to include more areas.

Don’t be fooled by the many academics and journalists who see no distinction between data science and statistics or even advocate that statistics be renamed data science and statisticians data scientists (C.F. Jeff Wu in 1997).

Read Also:
Invasion of the Data Snatchers: Big Data and the Internet of Things Means the Surveillance of Everything

Data science is an independent discipline, who relies on the shoulders of statistics but it is extending the field to new realms thanks to Big Data, Computer Science and distributed systems.

 



Data Innovation Summit 2017

30
Mar
2017
Data Innovation Summit 2017

30% off with code 7wData

Read Also:
Associative Data Modeling Demystified

Big Data Innovation Summit London

30
Mar
2017
Big Data Innovation Summit London

$200 off with code DATA200

Read Also:
Reverse-engineering artificial intelligence

Enterprise Data World 2017

2
Apr
2017
Enterprise Data World 2017

$200 off with code 7WDATA

Read Also:
Artificial Intelligence Is Setting Up the Internet for a Huge Clash With Europe

Data Visualisation Summit San Francisco

19
Apr
2017
Data Visualisation Summit San Francisco

$200 off with code DATA200

Read Also:
How Sense with Voice will save your sleep with big data

Chief Analytics Officer Europe

25
Apr
2017
Chief Analytics Officer Europe

15% off with code 7WDCAO17

Read Also:
Why your company needs to ask the right questions about big data

Leave a Reply

Your email address will not be published. Required fields are marked *