Busting 5 Myths about Data Lakes
- by 7wData
The data lake is still quite new, so it's natural that a few myths and misunderstandings have proliferated across the data management community. To set the record straight, I'd like to bust five myths that I keep hearing. First, I need to define the data lake so that we're all on the same page. A data lake is a user-defined method for organizing large volumes of highly diverse data. The data of a lake may be deployed on diverse data management platforms, including Hadoop clusters, relational databases, clouds, or a combination of these. Depending on the platform, a data lake may handle diverse data types, ranging from unstructured to semistructured to structured data. For most organizations, a data lake supports multiple use cases, including broad data exploration, advanced analytics, data warehouse extensions, and data landing and staging. Lakes may also serve specific departments (marketing, supply chain) or industries (healthcare, logistics). Busted! Although it's true that any database can become a dumping ground, this isn't what successful early adopters are doing with their data lakes. Lake owners interviewed by TDWI say that a data lake is a balancing act. Some users are allowed to dump, but others are not. For example, data analysts, data scientists, and some power users need to create "sandboxes" of data in their work; they are allowed to bring data into and out of the lake freely as long as they govern themselves. Most other users must petition the lake steward or curator, who vets incoming data. Myth #2: Data lakes are only for Internet firms. Busted! Hadoop and the data lake were pioneered by Internet firms, and we owe them our thanks for those innovations. However, TDWI has found data lakes in production in several mainstream industries, including finance, insurance, telco, pharma, and healthcare. As noted above, some lakes serve departmental operations or analytics. TDWI has also found multiple forms of analytics operating on a lake data, including data and text mining, clustering, graph, and predictive analytics, and natural language processing. Lake-based analytics supports a number of analytics application types, including risk calculations, customer segmentation, and the detection of fraud, security breaches, and insider trading. Data lakes serve a wide range of enterprises, and each lake is typically multitenant because it serves multiple business units and use cases. Busted! A recent TDWI survey showed that over half of data lakes in production are on Hadoop exclusively (53 percent).
[Social9_Share class=”s9-widget-wrapper”]
Upcoming Events
Strategies for simplifying complex Salesforce data migrations – Free Webinar
27 March 2024
5 PM CET – 6 PM CET
Read MoreCategories
You Might Be Interested In
How artificial intelligence is fuelling engine maintenance innovation at Rolls-Royce
30 May, 2018Engine manufacturer Rolls-Royce has been steadily building up the intelligence capabilities in its aero engines. In April 2018, Rolls-Royce was one …
How Should Organizations Actually Go About Digital Transformation?
5 May, 2017This is now the question that is top of mind in a large number of enterprises today, as speeding up …
Three reasons why you need to modernise your legacy enterprise data architecture
16 Oct, 2018A system must undergo “modernisation” when it can no longer address contemporary problems sufficiently. Many systems that now need overhauling …
Recent Jobs
Do You Want to Share Your Story?
Bring your insights on Data, Visualization, Innovation or Business Agility to our community. Let them learn from your experience.