Some analysts claim that big data hasn’t lived up to its promise. Mind you, nothing could live up to the expectation that was generated. Ironically, now the spotlight may be turned elsewhere, Big Data will deliver big time.
The impact of big data was less than expected, says the September 2016 Gartner Group research note Using Algorithmic Retailing to Drive Competitive Advantage.Gartner says that in future algorithms will cut costs and boost revenue for industry as the Internet of Things (IoT) takes off. I dare say they may do, one day.
In the meantime, as expectation is stocked up for algorithms, the pressure is off the big data camp and they can get on with their work and start delivering the benefits. In the last few years, the volumes of data being generated have boomed. According to Gartner, around 90% of all the data in the world was created in the last two years.
For once, it’s not hyperbole to use the word ‘explosion’ in the context of data, since we really are witnessing a textbook ‘sudden violent change of energy’. For proof, you only need to check the data center, where servers have to be powered and processors have to be cooled in order to accommodate the mass volumes and diversities of information types being produced.
Has big data been underwhelming? It’s fairer to say the task, created by the tsunamis of intelligence flooding into servers in every industry, has been overwhelming. Pharmaceuticals companies have never been able to capture such a variety of information types. As a result of all the new, affordable types of sensors, manufacturers have never had so much intelligence about their machinery and production lines. Retailers have never had so many channels to market and have never been able to capture so much information about their customers.
Those are the scales of the big data challenge that have been created. However, cometh the hour, cometh the man or, more accurately, the plan. The ‘plan’ in this context is in-memory computing, which provided the breakthrough that makes Big Data analysis possible.