The value of data quality tooling

Allow me to start this blog post with the side note that some of my blog readers comment that I sometimes sound like a broken record. If that hasn’t been your observation, you may be wondering what I’ve been saying that prompted those comments. Well, simply put: that good data is a strategic business asset for modern organisations, and that data quality should be an integral part of your business strategy. And why do I keep emphasising that over and over, you might wonder? The answer is both simple and surprising. Because most organisations still aren’t sufficiently aware of these basic facts. So here we go again:

High-quality data = Strategic business asset

In other words, data is a high-value asset for organisations. A carefully considered Data Governance framework makes it possible to achieve data quality. Don’t wait too long, though; you should act now! The data that organisations can access is changing at incredible speeds, transforming in type and in sheer volume. This means that the potential costs of not implementing a Data Governance framework will grow higher and higher.

In my blog post on “Why your organisation needs Data Governance”, I presented various reasons why it would be extremely advisable for organisations to implement Data Governance as a matter of real urgency.

In short, it comes down to this. As a result of the growing body of laws and regulations, it is becoming more important than ever for organisations to implement Data Governance. Don’t just sit back and wait to see what happens, but pro-actively anticipate future data issues that will arise from this ever-changing landscape. In addition, organisations operating in less heavily regulated sectors are increasingly realising that they can achieve a competitive advantage if they manage their own data effectively – and understand it thoroughly. In my opinion, organisations that take a holistic approach to Data Governance will benefit most. But please note: Data Governance initiatives will need to be directed by the business and use a structured methodology in order to be successful, cost-effective and above all relevant.

That brings me to my next point. A good Data Governance framework cannot truly exist unless it is supported by technology. The current generation of tooling can provide that support in the following ways.

Tools for data profiling

Research shows that only 3% of the data meets organisations’ own standards of quality (Nagle, 2017). This implies that poorly managed data costs organisations millions of euros in wasted time, money and underutilised potential. Data profiling tooling determines the quality and legitimacy of your data. Using analytical algorithms, data set characteristics are analysed down to the most detailed level. Common database errors are eliminated in this step. (Examples include empty fields, values that should not be included, values that occur with an unusually high or low frequency, and values that do not follow expected patterns.) Data profiling tooling helps organisations structure their own data more effectively and understand the data (and the relationships within the data) more completely. This enables organisations to discover essential information derived from the data already at their disposal – and knowing what data you have, what it means and why it’s significant is obviously elementary in Data Governance. After all, how can you manage your data effectively if you don’t even know what data you have?

Tools for data glossaries and metadata

An important part of Data Governance is identifying the available data and its significance. It also includes defining ‘business rules’ and data quality rules that will ultimately determine what makes the data good enough for the end user. Drafting a ‘data glossary’ can offer a solution. A data glossary describes the characteristics of business-critical data elements in the various systems. This description is important to ensure that everyone knows how these data elements are defined, where they are stored, and how good the quality is. (While implementing projects, the data glossary will be consulted frequently to check how data elements are affected by certain changes (Feltkamp, 2018)). A data glossary can be supplemented with reference data (metadata), search lists and the mapping between reference data stored in different systems. Some tools have a master workflow for directing lower-level workflows. This simplifies the process for data owners and data stewards, enabling them to assess and approve data definitions and metadata.

Tools for data quality monitoring and reporting

These tools are probably the type of tooling that end users will be most familiar with. As the name of the category suggests, these tools focus on controlling and monitoring data quality. They monitor the data quality levels defined using the business rules, generating reports that can reveal any data quality problems. Data owners and data stewards can use those reports to take action, investigating the problems flagged in the previous step and resolving them as needed.

Finally, above all good Data Governance is about assigning the right people to the right roles and setting up effective processes. These processes can be supported by using the right technology. I would like to emphasise that tooling alone is not the answer to data-related issues. Even so, you can successfully use tooling to support and operationalise some of your processes. For example, if you want to start using much larger volumes of data, you’re going to need all the help you can get. Right?

Tentive Solutions offers organisations support in resolving their Data Governance and Data Management issues. Would you like to know more? Feel free to contact our Data Management Team.

Jacco Oudeman

References consulted for this post:
Feltkamp, D. (2018), “Data is een onderwerp voor in de boardroom”
Nagle, T., Redman, T.C. and D. Sammon (2017), “Only 3% of companies’ data meets basic quality standards”, Harvard Business Review, 11 September 20