The framework relies on 4 distinct stages that in execution all build on the knowledge and infrastructure developed in the previous phase (see below):
From experience, I find that most companies struggle with making their data quality assessment results compelling because they take a data-centric viewpoint. They show stats and metrics that, whilst valuable to the data community, can be dull as dishwater to the business leader who needs to make tough decisions on where to focus limited staff and financial resources.
Make your Data Quality Assessment really compelling
Communication is a big problem.
To get the business onboard, they need to visualise the problem and build a story in their mind of how (and why) they are impacted. They need to ask questions and observe the impact from the different dimensions that matter to them.
This is where the Interrogative Reporting Layer comes into its own.
The final stage in the 4 step data quality framework, interrogative reporting allows the business to “slice and dice” the results of your assessment to visualise data quality from their side of the fence.
A great example of this is the telecoms company that undertook a national data quality assessment of their plant asset data. By creating an interrogative reporting layer on top of their extensive data quality metrics they were able to ask questions such as:
- How much equipment cannot be activated due to poor quality data?
- Which region/team/equipment has the best data quality?
- What is the financial impact of stranded assets due to poor quality data?
- How is data quality trending in each telephone exchange?
They were able to put their “data quality assessment on the map” by overlaying location, equipment, financials and of course data quality metrics, all in one visualisation platform.
So how do you achieve this?
The following diagram explains how I typically set up an interrogative reporting layer to my data quality assessment:
The Information Chain Management component is critical because a lot of people focus on measuring data quality using a column-by-column, table-by-table viewpoint that adds little to the understanding of the business who need to visualise data quality at a function level.
For example, consider the information chain required to support a typical customer order.
The order may be initiated via a web order handling system. It will then flow to a billing system where the financial transaction is executed. 3rd party supplier systems may then be communicated to request real-time supply of the goods. The product may then be dispatched via another 3rd party transport system. The customer receives a final invoice after all these systems have come together seamlessly.
You can now see how business functions rely on connecting data across different departments, spanning multiple attributes and relationships across many systems. To make data quality assessments meaningful you must connect your data to visualise the financial and operational impacts of poor quality data.
This is where the requirement for Data Integration and Movement becomes necessary.
Static data profiling tools that assess data only at a column level simply don’t scale up to the needs of the modern enterprise that deliver customer value through complex information chains.
Track data quality over time – a key aspect of the Interrogative Report Layer
This requires a data repository so that the operational data (and metrics) can exist in snapshots over time for comparison purposes. Without this trending functionality, it becomes extremely difficult for the business to visualise the financial benefit of investing people and capital into data quality improvement.
Another benefit of having a data repository is that you can prototype improvements and assess what kind of impact they have had in terms of operational data quality levels, without impacting the live systems. This can help you dramatically reduce the “lead time to improvement” over more traditional methods that don’t include a repository structure.
Finally, the dashboard and visualisation layer allows you to leverage whatever existing reporting and analytics tools you have within your organisation so that the business feels comfortable visualising data quality in an environment they are comfortable with.
You can connect these tools directly into your data quality environment and create dashboards that are meaningful to the various stakeholders involved.