The leading trends in enterprise data management in 2016 were continued from recent years, namely Hadoop adoption and data warehouse modernization. However, the real surprise in 2016 was the data lake, which user organizations are suddenly taking seriously as the preferred design pattern for data set organization in Hadoop.
All these will continue into 2017 and be joined by new activity around the SQL-ization of Hadoop, orchestrated data hubs, and managing sensor data from the industrial Internet of Things (IoT). Let's look at each of these in detail.
TDWI surveys in recent years have shown that Hadoop is making steady progress as a platform well suited to many purposes in data warehousing and analytics. Many early adopters have already integrated Hadoop clusters and tools into the architectures of their data warehouse environments.
Hadoop's massive, cheap storage offloads older systems by taking responsibility for data staging, ELT push down, and archiving of detailed source data (typically in the data lake design pattern). Hadoop also serves as a massively parallel execution engine for a wide variety of set-based and algorithmic analytics methods. These valuable use cases are driving the adoption of Hadoop.
TDWI has seen a giant step forward in adoption starting in late 2015 and continuing into 2016.