Data analytics is composed of data and analytics on the data. Data is varied, generated by internal sources (like transactional applications, sensors etc) and/or sourced from external sources (data aggregators, DMPs, SaaS applications etc). As more internal applications are moving to the cloud, this internal data is invariably already present on the cloud platform. Organizations spend less effort and people managing the data infrastructure in the cloud environment while getting better scalability, if security challenges are addressed properly. For external sources, the process to get into a hosted or cloud environment is essentially the same in terms of complexity. It can be argued that getting data into the cloud environment is a little simpler due to simpler provisioning. Even though computing and storage are decoupled in the cloud, it is very easy to build a solution that uses scalable compute and storage to message and munge this data from different sources and combine into useable stores. Teradata, a stalwart in data management, also has AWS instances to allow it to scale more.
On the analytics front, key activities are data aggregation, machine learning and statistical analysis and visualization. Building machine learning models that can be trained efficiently is dependent on quick access to large compute capacity for short periods of time. Cloud computing is well equipped to provide such capacity models. Many cloud providers provide specific machine types with pre-installed libraries to speed up deployment. Further, services like AWS EMR take the server provisioning completely out of the picture and let the data scientist focus on model training and development tasks.
  ​ ​Many organizations use data only if it supports their gut or intuition. If it doesn’t, then insight from analytics is either ignored or belittled Â
However, deep learning based models have thus far been not very successful in the cloud computing model for two reasons:
(a)Â Availability of powerful GPU based compute infrastructure (b)Â Distributed deep learning software libraries to take advantage of scaling in the cloud
At recent AWS: reinvent, new announcements around new and powerful GPU compute instances along with support for distributed deep learning libraries like MxNet and TensorFlow was announced. Expect to see deep learning work move to the cloud for a number for several companies.
In the visualization area, general web application build outs are well known. Using serverless computation platforms like AWS Quick sight are emerging, which can bring a round of innovation. In summary, a lot of reasons to be excited about use of cloud for data analytics!
There are different types of challenges companies face while deploying data analytics solutions. First challenge is cultural. Having a data driven decision making culture is essential to the success of data analytics initiative. Many organizations use data only if it supports their gut or intuition. If it doesn’t, then insight from analytics is either ignored or belittled. This is a critical question for companies before they embark on their data analytics journey. Organizations must clearly define expected outcomes from data analytics project. Is it speed, quality, or availability that they are after? It is often impossible to do all three at the same time.
