Data integration teams often find themselves in the middle of discussions where the quality of their data outputs are called into question. Without proper governance procedures in place, though, it’s hard to address these accusations in a reasonable way. Here’s why.
First, many operational data integration projects are not configured to include data quality assurance as part of the project. Believe it or not, during a meeting I heard a representative from a project contractor suggest that ensuring quality of the data output was not part of their statement of work. I’ve also reviewed project plans and contracts in which the service provider specifically states that they make no guarantees about the quality or usability of their work.
Second, without defined data assessment processes, it’s challenging to pinpoint the source of a data flaw. That’s especially true in the context of data integration, which typically ingests data from different sources, applies some transformations, and reformulates the data into target data models in preparation for consumption by a downstream user or application. The bad datamight have already been flawed before it was ingested (in which case it’s not the fault of the data integration team). But it might have been corrupted as part of the integration process (in that case, it is their fault!).