Integrated vs. specialized: Which shines the brighter light on "dark data?"

Integrated vs. specialized: Which shines the brighter light on “dark data?”

Integrated vs. specialized: Which shines the brighter light on “dark data?”

Ingest, data prep, analysis, visualization and export. They're all part of the Big Data analytics life-cycle. The good news? The market sports multiple product categories that handle each of these. The bad news: there are far fewer products that handle many of these areas together. It's getting more complex too, as other big data lifecycle categories, like data lake management/data cataloging and Big Data operations/DevOps are emerging.

This may be frustrating, but it's also understandable: if there's an area of functionality that the market is neglecting, then funding a company to develop and provide that functionality makes sense. It helps the market, it helps the company and, hopefully, it helps the company's investors.

Does it help the customer, though? That's harder to say. If you're doing your data prep in one tool and your analysis in another, that's a bit regimented, and the context switch may be jarring. Moreover, if your analysis reveals to you more prep work to be done, you are sent back to the data prep tool for a "do-over," rather than embarking on iterative effort.

Read Also:
Go from BI to AI in Minutes

Case in pointRecently, I spoke to Bob Laurent, VP of Product Marketing at Alteryx on a related subject. The discussion began as an investigation into Mr. Laurent's notion of "dark data:" data that you have in your possession but aren't using, or analyzing, for the betterment of the business. As Alteryx has a focus on data preparation, the discussion began with talk of how to avoid dark data, by grabbing, processing and coalescing data sets, be they buried in log files, point-of-sale feeds or elsewhere, to get at intelligence that would otherwise be latent and obscured.

That's all fair game. The nitty-gritty of digging for valuable data -- then scraping the debris away and making it sparkle and shine for observation and analysis -- is the bread and butter of every company in the analytics field.

It wasn't really enough though, not for an interesting conversation and not for a Big on Data post.

Read Also:
One Dataset, Visualized 25 Ways

Lifting the darkness But then, as if sifting for data sets, we stumbled on something really valuable. Laurent described a case study to me -- one where a financial services customer used Alteryx to look at its trading terminal data, to find correlations between certain trading patterns and various abuse incidents (like money laundering, for example).



Data Innovation Summit 2017

30
Mar
2017
Data Innovation Summit 2017

30% off with code 7wData

Read Also:
Venture investors should embrace data over instinct

Big Data Innovation Summit London

30
Mar
2017
Big Data Innovation Summit London

$200 off with code DATA200

Read Also:
States Appear On The Frontier Of Your Car Privacy Protection

Enterprise Data World 2017

2
Apr
2017
Enterprise Data World 2017

$200 off with code 7WDATA

Read Also:
Blog 6: Statistics Denial Myth #2, Traditional Practice Straw Man

Data Visualisation Summit San Francisco

19
Apr
2017
Data Visualisation Summit San Francisco

$200 off with code DATA200

Read Also:
10 Best Practices in Configuring your Agile Project Management Tools
Read Also:
Datera emerges from stealth to offer another take on cloud scale-out storage

Chief Analytics Officer Europe

25
Apr
2017
Chief Analytics Officer Europe

15% off with code 7WDCAO17

Read Also:
What Recruiters and Hiring Managers Are Looking for in a Data Scientist

Leave a Reply

Your email address will not be published. Required fields are marked *