Learn about how to rapidly iterate data applications, while reusing existing code and leveraging open source technologies, brought to you in partnership with Exaptive.
This morning at Hadoop Summit, I gave a talk about how data will transform everything and the need for Connected Data Platforms. Let me explain what I mean.
Until now, enterprise data was largely structured and not particularly diverse. This reality spawned what we know now as traditional IT best practices: Buy standardized packaged solutions or use them to replace custom solutions and implement an EDW based on user requirements.
We have now moved from that old world to billions of connected devices creating and generating more data than ever before—both inside and outside the corporate firewall. All of us as consumers are creating this data every moment of every day with infinite variety. This will only accelerate as the global proliferation of connected devices continues and as consumer and customer expectations for relevant content and social interaction increases.
The new data is very diverse. Whether it’s from a device, a sensor or the web, a car, a phone, its not structured transactional data under an ERP system any more. Analyzable data lives everywhere. In the cloud, at the edge, in the data center, on the device. We can’t put this genie back in the bottle.
It’s beyond big data; CIOs now need to think about all data. The implication is that the technologies and best practices of the past are not sufficient. This is creating upheaval in the IT architectures of businesses and many CIOs are re-platforming their data layers.
We see Connected Data Platforms as the key innovation that is going to drive success in this new world of data.
One big thing driving this re-platforming is cloud versus data center. Now everything is cloud-centric, and internal data silos are being replaced by data clouds that all need to connect.
Another is the need to analyze data where it lives. Data has gravity. Although its getting cheaper all the time, data movement is still relatively expensive, so we need to run analytics as close to where the data lives as possible. It’s paramount to figure out ways to get data out of the old data-center silos and connect it to the data cloud and edge of our networks in order for breakthrough kinds of value creation via analytics to happen.
The data and apps of the past were all about centralization—pull all the data in, normalize it, put it into silos, then figure out what to do with it. In the new world order, that is simply not even imaginably workable, let alone realistic, and it certainly prevents value creation.
Both of these reasons are why see the need for a new kind of Connected technology platform that is all about data versus app-centricity, cloud and data center, and that can connect all data wherever it naturally lives.
Today, we need distributed analytics, distributed data, and distributed applications.
This is a business not just a technology mandate. Amidst all of the new data, it is no longer acceptable to just analyze historical information to report on and understand what happened. You’ve actually got to take decisive action while things are happening.
The technical implication of this is we need to connect streaming data, historical data, and analytics. The only way to have this work is to have all of our data platforms be connected, whether in cloud or in the data center, so they can communicate with each other, connect all the data that is relevant and the analytics that are relevant at a service level that is relevant to the business.