Solving the data quality conundrum: how to go from big data to trustworthy data

Solving the data quality conundrum: how to go from big data to trustworthy data

Solving the data quality conundrum: how to go from big data to trustworthy data

Businesses often struggle with hard to access data and cumbersome processes, so how can they find quick and simple ways to locate, prepare and verify the data they need?

Lets consider the classic meeting scenario where a Sales Director presents the company’s most profitable customers to the management team.

Before she has got even halfway through her first slide, someone interjects to point out that Cable&Wireless Worldwide is no longer trading and General Electric reduced its regular order significantly last quarter.

People around the table have spotted errors in the data and trust in the analysis is immediately lost - faces around the table tell the story all too clearly. How can that team go on to make an effective decision?

> See also:The key steps to achieving data quality over quantity

Similarly if you’re a data analyst you’ve probably experienced the often tense relationship that can develop with IT colleagues that are integrating and preparing data on your behalf.

Read Also:
Healthcare Big Data Silos Prevent Delivery of Coordinated Care

Comments such as 'I thought we asked for all three customer databases to be brought together for this analysis' or 'sorry for us to extract, transform and load that much data in different formats it’s going to be another three months' are not unusual.

These scenarios highlight two of the most pressing problems preventing companies from making effective, data-driven decisions. Business leaders are witnessing the impact of poor quality data; data analysts and IT managers are struggling with the challenge of pulling disparate data together from multiple sources.

In a world where companies must make increasingly rapid decisions, both of the above can: frustrate, hamper and sabotage even the best-laid plans.

Making sense of a company’s data typically requires individuals in different roles with different skill sets. The business executives that know what questions to ask of the data, data analysts that hunt through data in order to answer those questions and finally there are IT people that ensure data is prepared and ready for analysts to work with.

Read Also:
Data Scientists vs. BI Analysts: What's the Difference?

While everyone has a role when it comes to preparing data, the challenge comes when empowering all groups with the processes and tools they need to collaboratively manage data.

'Big Data' has of course made the opportunities (or the problems) even bigger, with the promise of unprecedented insights into markets, customers and operations.


Leave a Reply

Your email address will not be published. Required fields are marked *