Integrated vs. specialized: Which shines the brighter light on "dark data?"

Integrated vs. specialized: Which shines the brighter light on “dark data?”

Integrated vs. specialized: Which shines the brighter light on “dark data?”

Ingest, data prep, analysis, visualization and export. They're all part of the Big Data analytics life-cycle. The good news? The market sports multiple product categories that handle each of these. The bad news: there are far fewer products that handle many of these areas together. It's getting more complex too, as other big data lifecycle categories, like data lake management/data cataloging and Big Data operations/DevOps are emerging.

This may be frustrating, but it's also understandable: if there's an area of functionality that the market is neglecting, then funding a company to develop and provide that functionality makes sense. It helps the market, it helps the company and, hopefully, it helps the company's investors.

Does it help the customer, though? That's harder to say. If you're doing your data prep in one tool and your analysis in another, that's a bit regimented, and the context switch may be jarring. Moreover, if your analysis reveals to you more prep work to be done, you are sent back to the data prep tool for a "do-over," rather than embarking on iterative effort.

Read Also:
Predictive Analytics: Driving Improvements Using Data

Case in pointRecently, I spoke to Bob Laurent, VP of Product Marketing at Alteryx on a related subject. The discussion began as an investigation into Mr. Laurent's notion of "dark data:" data that you have in your possession but aren't using, or analyzing, for the betterment of the business. As Alteryx has a focus on data preparation, the discussion began with talk of how to avoid dark data, by grabbing, processing and coalescing data sets, be they buried in log files, point-of-sale feeds or elsewhere, to get at intelligence that would otherwise be latent and obscured.

That's all fair game. The nitty-gritty of digging for valuable data -- then scraping the debris away and making it sparkle and shine for observation and analysis -- is the bread and butter of every company in the analytics field.

It wasn't really enough though, not for an interesting conversation and not for a Big on Data post.

Read Also:
Under pressure: 4 main stressors for big data leaders

Lifting the darkness But then, as if sifting for data sets, we stumbled on something really valuable. Laurent described a case study to me -- one where a financial services customer used Alteryx to look at its trading terminal data, to find correlations between certain trading patterns and various abuse incidents (like money laundering, for example).



Read Also:
Generali's Elena Rasa Takes a Data-Driven Approach to Excellence
Read Also:
Business intelligence must look at internal and external data
Read Also:
Car data could be a much bigger business than electric or driverless vehicles
Read Also:
How IoT and AI are driving analytics innovation

Leave a Reply

Your email address will not be published. Required fields are marked *