Establishing Data Governance and Quality Frameworks at a Standards Setting Organization

Jeff Shick:
Well, good afternoon and thank you for joining me here at the Clinical Architecture Data Quality Theater. My name is Jeff Schick. I’m Director of Translational Informatics at the United States Pharmacopeia. Translational Informatics is a group within the Digital and Innovation office, and I’m a pharmacist and clinical informaticist. I’ve been with USP for almost five years now. Next slide. So, USP operates on a five-year cycle and in the most current cycle, the DNI group was formed and has focused on developing digital solutions to advance USP’s public health mission by exploring emerging science and technologies, maturing promising innovations and delivering digital solutions that address stakeholder needs. This includes creating standards for drug information, creating standards for chemical substance information, and creating standards for analytical test methods and parameters that are used in pharmaceutical laboratories to support this mission. The data integrity office was created under the auspices of DNI with the purpose of establishing data governance and data quality frameworks across the organization.

So here today, we’ll discuss the progress that we’ve made and lessons learned. Next slide. So who is USP? The United States Pharmacia, we are a scientific, non-governmental non-profit organization, founded in 1820 by a group of physicians that came together to address the issues of inconsistent and poor quality medical preparations that were available at the time. Their goal was to protect patients by establishing standards of quality, purity, and consistency of medications just like those physicians over 200 years ago. USP Data Integrity Office brings together various individuals from across the organization to focus on poor quality of data that typically results in the evolution of printed content to digital content. Next slide. While previously available in electronic format, CD ROM. USP really began its digital evolution in 2008 with the first online version of our product.

This allowed for more efficient updates and easier access to standards, but it was still just an electronic version of the book. And in 2020, we discontinued the print product and print publication of our standards, and now they’re only available electronically. Next slide. So the next phase of our digital evolution includes both our documentary standards as well as physical reference standards, which are vials of actual physical pharmaceutical product using a digital fingerprint. Laboratories are can now automate identification and quantification methods with databases of quantum mechanical models of simple molecules and complex mixtures. Next slide. So what is data governance? Well, it’s the execution and enforcement of authority over data. Data governance has become increasingly important as we expand the volumes and types of data to make critical decisions about costs, operations, products, service development, etc. It’s responsible for developing and enforcing policies and procedures and it ensures the entire organization is on the same page regarding addressing this vital organizational business asset.

A successful data governance program must be aligned with the business strategy of the organization to ensure that we get the most value from our data. That’s regardless of where it’s generated, where it’s consumed, and where it resides. Governance objectives may vary depending on industry. It can be driven by compliance, regulatory requirements, and business value. It could also focus more on value creation, decision-making, and process optimization. In any case, governance must be owned and led by business executives in partnership with their technology executives. This puts the business owners in the driver’s seat and ensures data governance efforts are tied to business use cases.

Your approach to data governance will depend on various factors including the size of your organization, the culture, data management maturity, and the industry’s regulatory requirements. One option is a decentralized or federated approach to data governance. This distributes responsibility and authority for data governance across multiple groups. Though it can be more challenging to manage and that’s why we at USP decided upon a centralized approach. This involves a central body with the authority to determine how power will be applied throughout the organization. This includes policies and procedures for creating and managing master data, identifying approved technologies, and deciding on governance priorities. This approach is often more appropriate for organizations with strict compliance and regulatory requirements. Next slide. So in establishing a data governance framework, the first thing to do is decide what will be governed. So one of the first steps is to identify the scope of the data that will be handled.

That means identifying the data domains or categories that must be managed and the stakeholders who will provide data stewardship for each part. This might include reference data, transactional data, or master data. Once the domains and stakeholders are established, it’s a good practice to interview stakeholders across business and IT functions to analyze the pain points of poor data governance. This is the time to learn about obstacles that is stand in the way of achieving business objectives. So who will govern it? Organizations are recommended to establish a data governance work group, an executive team, a data council, data steering committee, whatever you call it. This cross-functional leadership team oversees data management, typically led in large organizations by chief data officer. Other key players in data governance typically include business line executives, group project leaders, and others who have accountability for using the data within their domains. They can help ensure that domain data is properly defined and used throughout the enterprise.

Depending on the organization’s size, data owners are generally leaders in their business functions. Business data stewards are responsible for implementing and improving information quality. They’re generally two types of data stewards. One is a business unit steward who is responsible for managing the data for their specific business unit. So it could be marketing, finance, et cetera. And domain stewards who collect data for a particular domain, so like customer data or reference data, in our case, scientific data. And then IT data stewards are subject matter experts for the data source assigned to them, and each IT data steward is responsible for executing initiatives and decisions associated with the functional area or domain of the trusted data source. So how will it be governed? Well, the first step is to define the organization’s data governance work products for each data domain and then analyze the current state of those work products.

Examples of domain level work products include data lineage, policies and procedures, standard business definitions, data quality and compliance metrics and dashboards. The team must build specific capabilities to help the organization address its data governance goals. And this exercise also ensures that the team is focused on higher priority work products and processes. Typically, the highest priority capabilities are impact analysis, common business definitions and policies and procedures. Next slide. The good thing is that you don’t have to go it alone. Some of the most widely adopted data governance frameworks come from the data management association, data management body of knowledge. It’s a comprehensive coverage of all data management aspects. The DGI Data Governance Institute, they focus primarily on decision-making processes. Price Waterhouse Coopers, they emphasize lifecycle management and McKinsey, which takes a business value approach to data governance. And you can draw elements from more than one framework depending on what makes sense in your business.

At USP, we drew heavily on DMAs framework and next step or next slides rather. So the initial step involved forming the executive team or business council. At USP, we call this group dice, which is short for the digital impact committee. It’s led by our senior Vice President of digital and innovation, and it also includes our C-suite, the COO, CFO, Chief Science Officer, Chief Information Officer, and our General Counsel. Each data governance lead reports back to the DICE committee on a quarterly basis. And the functions of dice responsibilities of DICE is to provide strategic direction, allocate resources, resolve executive level issues, assert accountability, and serve as a single point of escalation for all unresolved issues. And then they also monitor adherence to data policies, procedures, and standards. Next, we formed the digital integration office. It was originally called that because one of the things we wanted to accomplish was to integrate our content among various enterprise business solutions.

But now as that has evolved over the last couple of years, it’s been renamed still DIO, but it’s now referred to as the data integrity office. This is led by our senior data quality manager for digital products and the DIO consists of all of the business owners. Some have data quality leads, some do not from each of those domains. So the responsibilities of the DIO provides tactical direction, manages resources and associated activities, resolves leadership issues, asserts accountability aligns on messaging to the business council, responsible for collaboration on execution of requests and developing those policies and procedures. Data governance committee is made up of data content owners and source system owners, which could be it typically are it and data content owners, typically the business source data stewards. These drive the organization’s data governance, strategic vision and evaluate financial considerations.

They establish and manage the data governance policies and procedures and are accountable for data definition, management, usage and quality. There’s a data governance committee for each data domain. At USP, I’m actually the data governance lead for three of those domains, clinical informatics, chemical informatics, and laboratory informatics. We also have other data governance leads for business units such as customer service data for our customer data and financial data as well. So data stewards execute day-to-day affairs in accordance with strategic direction tone and expectations set by the council and committees. SOPs are established to allow employees to clearly understand their roles and responsibilities and to guide all major decision-making processes and actions within predefined limits. Large business process changes need to be captured and SOPs are created in accordance with our board of trustees of the organization. Finally, it’s fundamental that data control processes and enabling IT systems are components of embedding data governance. Next slide.

So data quality is the second area that I wanted to focus on today, and measuring data quality levels help us identify errors that need to be resolved and assess whether data fit its intended purpose. Poor quality data is often pegged as a source of operational snafus and inaccurate and analytics and ill-conceived business strategies. Data quality problems can cause expense when products are shipped to the wrong customer address. For example, lost sales opportunities because of incomplete customer records and fitness and fines rather for improper financial or regulatory compliance reporting. Gartner estimates that every year, poor data quality costs organizations an average of $12.9 million.

The Data Management Association developed a list of over 60 dimensions and sub dimensions of data quality. That’s a lot of data quality metrics. We only focused on seven of the most commonly used dimensions. Accuracy, which is the degree to which data correctly represents what is being described. Completeness percentage of missing data consistency is the absence of differences in data, regardless of the data source. Timeliness is when the data’s actual arrival time as compared to when it was predicted or promised. Validity measures how well data conforms to predefined business rules. Integrity is the validity of relationships across various data sets and uniqueness ensures duplication of data is identified and corrected. And here I’ve identified several metrics that can be used in quality measurements, the number of data errors identified, and the timeframe in which you do these data quality metrics or run these data. Quality metrics can vary depending on how often the data is updated. It could be daily even, but for us, weekly for some datasets, monthly and quarterly for others, the accuracy and error rates that are in data sets. Quantitative measures of data completeness, consistency, integrity and timeliness, calculations of the business impact of data quality problems and assessments of quality levels in definitions and metadata. Next slide.

So the lessons learned, you’re not alone. There’s a lot of resources out there and data governance and quality are concerns of organizations, whether they’re big or small. Secondly, start small. Like I said, there were 60 dimensions of data quality. We chose seven, but we started with one. We started with one data domain and one quality dimension. In small organizations, the same person may be wearing multiple hats, but the thing is these frameworks scale very well. Also, while there are tools out there to help you, don’t get bogged down with evaluating software. Tableau. It’s a great resource, but you can get started right away with Excel and without the added costs and time and money to obtain and learn a new tool. Finally, document your progress as you move forward so that successes can be repeated and roadblocks avoided when scaling up to additional domains and dimensions and stay on top of data quality issues. Like I said, depending on the volume and frequency of updates might be daily, weekly, or quarterly monthly. Next slide. So thank you for your attention on this late Wednesday afternoon. I don’t see anybody, well, I do, yes, actually somewhat has started a happy hour over here. So although they’ve seemed to have disappeared maybe to go get a refill. So anyway, thank you all for your attention today and I’ll open the floor to any questions you might have.