Data quality management left solely to IT often results in approaches that start from the database layer and work their way out into the business. Organizations where IT is not tightly integrated with other parts of the business tend to stop once the application layer is reached, assuming that problems rooted in people and process are management issues for the business to tackle.
Data profiling by IT is an exercise of evaluating data values in databases relative to pre-determined quality thresholds, which may not focus attention on the data that matters most to the organization. Data quality improvement approaches often include automation (e.g., batch data cleansing, “scripts” that change values across the database, etc.). Error prevention strategies often involve edits and other controls in front-end systems and data validation routines that run in the background.
Contrast that with data quality management being sponsored by a non-IT part of the business. Data quality approaches in this scenario often start from the customer perspective (internal or external) and work inward. These approaches tend to yield data quality issues that customers see, but also tend to only go inward as far as the application layer.
Data profiling by the business is more like a data quality audit in terms of errors exposed to customers—errors in data which are often most important to the organization. Data quality improvement approaches in this scenario include many of the non-IT variables: training, job aids, process reengineering, workflow/procedure adjustments, etc. At times, controls in applications are also considered in improvement strategies, but the business solutions to these issues often rely on changes in culture and the fabric of the organization.
It seems that a full-spectrum approach to data quality improvement is one that focuses on prevention and is bi-directional—that is, it comes at data issues both from the customers’ perspective and from the database layer. These vectors intersect in the front-end systems making the connection between people, process, and technology more solid.
The first step in implementing a full-spectrum data quality improvement approach is to utilize cross-functional teams for root-cause analysis, design of prevention strategies, and their ultimate implementation. Since quality can be influenced by changes to people, process and technology, these teams must include a full-spectrum of individuals (including IT).
Organizations that only make IT responsible for data quality management run the risk of not improving data that is most important to the organization and only going so far as technology systems for improvement. Organizations that only manage data quality outside of IT run the risk of too much reliance on people and processes while not leveraging the heavy-lifting value IT brings to the table. It makes much more sense to continuously improve data from a technology perspective while also improving it from a business perspective…a full-spectrum approach to data quality improvement.
