Big Data – a roadmap for smarter data

Big Data – a roadmap for smarter data

Big Data – a roadmap for smarter data

Big Data is a term that is used in the information technology industry to mean building multiple sources of information together into a data lake, a data repository built on relatively inexpensive high performing computer hardware. The value of your data can be extracted from a data lake through existing reporting and business analytic systems. Furthermore, the advent of machine learning capabilities for Big Data solutions provides additional analytical capabilities. Machine learning has derived meaningful insights which can be used to support business development and organizational growth.

Big Data in its current form will reduce your operational and infrastructure costs, but will not provide you with any additional value for your business over what enterprise data warehouses provides. Why is that? The machine learning of today, that is employed within Big Data solutions, has no more capability than the statistical packages that are already in use within enterprise data warehouses solutions.

This may be true for today’s current big data incarnation. However, the future holds a new set of machine learning tools that integrate with Big Data. These new machine learning tools fall under the heading of neural networks. It might be helpful to first understand what neural networks can and cannot do.

Read Also:
Statistics is Dead – Long Live Data Science…

First and foremost, neural networks cannot think! More about that later. Neural networks have the capability for classification, regression analytics and forecasting. Given those capabilities, here is a small but amazing sample of uses that neural networks excel at:

There are more spectacular uses, but the above list will give you some idea that Big Data platforms are only in their infancy.

It is important to understand that not all neural networks are created equally. Picking a neural network that doesn't align to the specific problem that your trying to solve will result in poor accuracy and performance.

To get a sense of how different neural networks are used, below is a small sampling of uses and the neural networks that best fit the problem space and how they align to their capabilities. Note: It is beyond the scope of this article to go into detail about how each of these networks works.

Neural networks need to be trained. The process for training a neural network is called “back propagation.” Back propagation takes a lot of time to train a network using conventional CPUs. This is why the neural network community has turned to using graphical processor units (GPUs), as they are 250 times faster in training a neural network. That's the difference between one day of training and over eight months using conventional CPUs.

Read Also:
Social media data analytics: How to apply them in education

 



Data Science Congress 2017

5
Jun
2017
Data Science Congress 2017

20% off with code 7wdata_DSC2017

Read Also:
Opening up Big SQL for all: An interview with Paul Yip

AI Paris

6
Jun
2017
AI Paris

20% off with code AIP17-7WDATA-20

Read Also:
Introducing the New SAP Leonardo: Empowering Companies to Digitally Transform at Scale

Chief Data Officer Summit San Francisco

7
Jun
2017
Chief Data Officer Summit San Francisco

$200 off with code DATA200

Read Also:
Google AI Just Beat Human Pathologists at Detecting Cancer -

Customer Analytics Innovation Summit Chicago

7
Jun
2017
Customer Analytics Innovation Summit Chicago

$200 off with code DATA200

Read Also:
10 Big Data Advances That Are Changing the Game

Big Data and Analytics Marketing Summit London

12
Jun
2017
Big Data and Analytics Marketing Summit London

$200 off with code DATA200

Read Also:
More Firms Embracing Streaming Analytics, Machine Learning

Leave a Reply

Your email address will not be published. Required fields are marked *