Top 5 Impactful Use Cases for Improving Data Quality

Charlie Harp, CEO (00:03):
Alright, ladies and gentlemen and Joe, welcome to this afternoon session on the top five impactful use cases for improving data quality. I’m Charlie Harp, your host, and with us today we have Shaun. Use the microphone. There you go. Is it working?

Shaun Shakib, MPH, PhD, FAMIA (00:26):
Shaun Shakib, Chief Informatics Officer with clinical architecture.

Bonnie Bruner, MSN, RN-BC (00:30):
Sarah Brummel, I’m a Client Success Manager. Bonnie Bruner, also a Client Success Manager

Charlie Harp, CEO (00:36):
And you might be. And I’m Charlie Harp. I’m the CEO and founder of Clinical Architecture and I have the fine privilege of having these wonderful people work for me. And you might be asking yourself, Charlie, or asking me, Charlie, why are we having this session? And I’ll tell you these fine folks that you see before you have the pleasure of working with our wonderful clients and in the trenches of healthcare, they encounter a lot of different opportunities for doing things to improve the data, make things work better. And we thought we were having all these sessions with these amazing thought leaders and I said, Hey, we have amazing thought leaders right here at Clinical Architecture. Why don’t we hear what they have to say? And so what we’re going to do is we’re going to go through these five use cases and for each one of the use cases, I’ve asked these guys to share a little bit of things that we’ve done with clients that have helped improve data quality. Maybe if you’re watching this video or you’re Joe, you’ll walk away with some insight or you’ll think, you know what? I could use that, I should do that. And whether you use Clinical Architecture to do that or you do it some other way, it might help you to improve data quality which helps patients, which helps healthcare, which makes us happy. So why don’t we start with the first one. Are you guys ready?

Bonnie Bruner, MSN, RN-BC (01:49):
Ready.

Charlie Harp, CEO (01:49):
Alright, you look ready. The first use case is content acquisition. So Sean, why don’t you explain what content acquisition is?

Shaun Shakib, MPH, PhD, FAMIA (01:58):
It’s funny, I thought by sitting on the far end I would be the last one.

Charlie Harp, CEO (02:01):
Ah,

Shaun Shakib, MPH, PhD, FAMIA (02:02):
Content acquisition is just getting all the content that you need to use in your environment and those are disparate assets that come from multiple different standards, bodies are curated in different ways, we’re created for different purposes. For us, content acquisition involves collecting all of those various assets, putting them in one place and making them fit for purpose so you can use them.

Bonnie Bruner, MSN, RN-BC (02:29):
I would just add content acquisition is also, there’s a lot of different frequencies and how often standard terminologies will update RxNorm, for example, updates weekly. So we are able to keep that content updated so that when new medications come out, certainly new lab results come out. We keep that content updated so that when our clients want to normalize the data to that content, it’s available and updated.

Charlie Harp, CEO (03:05):
And what does that have to do with data quality?

Bonnie Bruner, MSN, RN-BC (03:10):
The standardized content should be quality data.

Shaun Shakib, MPH, PhD, FAMIA (03:13):
So a couple of things. So one thing, the reason why we say content and we don’t just say terminology, is that there’s all kinds of content that is terminology related, reference maps, things that you might not even think of as terminology, like zip codes. There’s all kinds of content that’s out there, but if you don’t have a trusted source for that content and that content is not kept up to date, then there’s no way you can have good data quality, right? Because those content, all that content is alive, it changes over time, concepts are deprecated, concepts disappear. And if you don’t have a good way of managing that and the lifecycle of all that content, then you’ll have poor data quality because it’s like the plankton and the ocean’s all dead and then the whole rest of the lifecycle chain. I know it stopped me.

Charlie Harp, CEO (03:59):
That’s pretty dark, man. Thank you. Just saying this is supposed to be a happy occasion. There’s another aspect to content acquisition that I’ll throw out there too. And we deliver content from all these standards and from all these things we scour the internet when a client asks for something, we work with content vendors, but the other way you deal with content acquisition is really content from any external system. So we have clients that have credentialing systems, location management systems. You’re acquiring content from those systems just like you would from a standard source and those things. The thing about content acquisition is you’re getting content from elsewhere, which means you don’t necessarily control how that content is created, managed or changed. Do you want to make sure that you have the most recent content? And to me the impact on data quality is really, I worked at a lot of places where they don’t update the content for like five years and when you’re trying to be compliant or you’re trying to be current, if your content’s not updated, you’re in trouble and that’s really to me with the data quality perspective on that. Ready to move on? Because the natural progression from content acquisition is reference data management. Now let me start out by saying reference data kind of fits into the content acquisition bucket because reference data is data that you get from somewhere else. It’s not yours, it comes from somewhere else. So let’s start with Bonnie, you want to start?

Bonnie Bruner, MSN, RN-BC (05:33):
Yeah. So reference data management, as you said is content you can acquire, but someone else kind of authors that content keeps it updated. So what we see a lot is the need to localize that reference data and extend that reference data so that the people who are using it have a control of what’s in it. A good example is when you are downloading, we have a client for example, who they use the NCI thesaurus and it’s not just a catalog of data, it’s a model of data. And the way they utilize that is with the ontologies and that reference data to be able to monitor of all these drugs, what are the neoplasms? So utilizing that reference data not only to map, to normalize the data to be able to roll it up and make informed decisions based on the data.

Charlie Harp, CEO (06:42):
Sarah, you got any use cases?

Sarah Brumley, MHA (06:46):
Well the one that comes to mind is probably the SNOMED and extending it for the PLR codes. And so we have a client that creates the PLR codes and

Charlie Harp, CEO (06:58):
The PLR codes are?

Sarah Brumley, MHA (07:00):
Sorry, public lab results. And so in the interim between when a new lab result, when you have that, you have to have a code for it, but it’s not in SNOMED yet. So you need something in that interim time. Originally these had been maintained via spreadsheets, but now it is a formal extension of SNOMED. And so once the concept has a SNOMED code for it, people can either use PLR or the SNOMED code, but that is just an example of how they’re extending that content source, SNOMED.

Charlie Harp, CEO (07:34):
Dr. Shakib

Shaun Shakib, MPH, PhD, FAMIA (07:37):
Reference data management’s a huge, my shorthand is making content fit for purpose. SNOMED has a half million concepts. If you don’t do something to create value sets from that that you use downstream, it’s not usable in that particular format. So PLR is an example. A PHL in particular the Association of Public Health Labs says PLR and PLT code. So they extend both SNOMED and LOINC, which LOINC has no formal extension mechanism for it currently, but they’re using our solution to create an extension of LOINC. Actually it’s harder to find a client that is not doing reference data management than to find ones that do. So everybody to some extent takes the standard terminologies and does something with them to make them fit for purpose.

Charlie Harp, CEO (08:34):
One of the things I’ll say is reference data management is one of those things that is, it looks easier than it is in principle. It seems easier, but let me harken back to a time when I worked for First Data Bank and First Data Bank had this whole terminology still does around drugs and a lot of times clients would say, Hey, I want to add a drug, you’re missing a drug. I want to add a drug or I want to localize a drug and the problem we had back then, I don’t know if it’s still a problem today, but the problem we had back then is they couldn’t do anything because we would give them an update every week or every month and it would stomp right on top of anything they did. And so when we rolled out our distribution architecture and we built Symedical, one of the things we did is we said clients use a lot of reference data, they have to be able to safely localize and extend that data without worrying that an update is going to stomp all over everything that did.

(09:31):
And if you’re pulling content down from the cloud and you’re putting in a relational database, the whole idea of updating to the most recent version can be daunting. I think as an industry we are headed towards a much more frequent update cycle than we used to. It used to be once a year or once a quarter or twice a year. But if you think about it the way we’re using data in healthcare, it’s going to become a real time phenomenon at one point or another. And the whole idea of localizing whether you’re translating into another language, whether you’re adding your own attributes, extending subsetting, you have to have a way to do that that doesn’t unravel the universe anytime an update comes out. Fair enough. Okay, master data management, who wants to talk? Explain what master data management is.

Shaun Shakib, MPH, PhD, FAMIA (10:27):
So Master Data Management is there is no third party that is creating and utilizing that content. It’s your own content. So it’s content that you are authoring that you probably want to integrate with the standard terminology that’s out there, but that there is, it’s your own homegrown stuff. Typically things like you might not even think of as classically as terminology like locations like room and bed. That’s an example of something that we have organizations that use Symedical to author and maintain that kind of master data. But there’s also like the DOD and their military readiness codes, that’s data that’s very specific to the DOD, it’s their kind of master data. Again, master data can in some cases be just extensions of standard terminology. Sometimes it’s completely new De Novo kind of content that they want to maintain and track and treat like standard terminology.

Charlie Harp, CEO (11:27):
Does anybody have an example of master data management?

Bonnie Bruner, MSN, RN-BC (11:30):
So some common examples I see are clients and my clients are mostly large health systems and payers, but the common examples I see are management of locations and management of provider, particularly characteristics around providers. So location for example, we have clients utilizing nursing units and seeing how they’re comparing and reporting things to the CDC. At hospitals locations are typically termed as a four A or a six north. And so that type of data has to be managed and it has to be managed by the data steward who is intricately familiar with that data. And that’s an ongoing process. Those locations change, it’s very important to keep them updated so you can have the right kind of reporting. Then providers I see used a lot as well when it comes to specialties, maybe conditions they treat, maybe cohorts that they should be responsible for quality metrics, even something as simple as providers being added to a practice or leaving a practice. And again, the master data management of that lies in the hands of the subject matter experts who are closest to the data. So it’s important to be able to federate out the management of that data. So again, the people who know it best are the ones making those changes and making those updates. And the impact of getting master data management, there’s so many downstream benefits to that. Being able to benchmark how your hospitals are comparing against each other and even clinics as well.

Sarah Brumley, MHA (13:42):
The only thing that I have to add, I mean we have the CDC is creating the location concepts, H-S-L-O-C and using those for antibodies that report up through. So that’s the location codes for the master data management that I’m involved with.

Charlie Harp, CEO (14:01):
The thing to remember about master data management, from my perspective, and this is something we didn’t come down from a mountain with tablets with this stuff written down and we worked with clients and the clients basically said, it’d be great if you could do this. And so one of the things that what we learned from working with clients is what Bonnie touched on and that is that it’s really important who’s managing the data, not just that the data is managed. So we spent a lot of time building out capabilities to kind of federate data management. I call it the democratization of master data management so that you can take concepts and separate out who’s working on different parts of the concept based upon who they are. And in the case of doing the location management, you have people that do marketing information for locations, you have people that do contracting with certain payer plans for locations.

(14:58):
You have people that maintain things like the phone number and things that are happening and those are all very different people. And what you want to try to minimize is kind of the whisper down lane effect of sharing spreadsheets and emails to manage data in a more Rube Goldberg way. So I think people underestimate what it takes to do master data definitely and the impact getting it right. And I think they also, there’s a certain aspect around healthcare data that’s different than what I might consider traditional master data that you’d see in the automotive industry for example. Alright, that brings us to bread and butter normalization. Sean, you want to talk about normalization professor, you’ve been doing the kind of the explanation then you do a good job.

Shaun Shakib, MPH, PhD, FAMIA (15:45):
So normalization is mapping is another way of thinking about it. So standardizing your data and actually the terms are confounded a lot and this happens a lot for us. We try to in some cases create our own nomenclature to just avoid all the confusion. But basically when we see normalization, we’re referring to taking local content, standardizing to appropriate standard terminology. What that does is that actually ends up normalizing your data on the way in, right? Because if you take a typical facilities lab data within that one facility, they’ll have the same lab test coded in multiple different ways. So just in a single facility across facilities, thousands of different local codes for the same underlying concept. When you map it to a standard terminology, you’re normalizing all those variant ways of describing the concept down to one consistent term. And our technology is all about making that, automating that, trying to get consistency and how you achieve that normalization, having good way to manage it over time with changes to the underlying standard terminologies and good ways to publish it out where it needs to go to be used.

Charlie Harp, CEO (17:01):
Anyone else want to chime in?

Sarah Brumley, MHA (17:02):
So I think that some of my customers that are heavily doing some of the normalization just want to tie this back to APHL with the PLR codes. When they had originally done the mapping to SNOMED, they had found I think it was like 500 some duplicate concepts within their own, what they had created. So from a quality perspective, even doing this normalization and mapping exercise is very important to make sure that you’re using the best data that you can for your purposes.

Shaun Shakib, MPH, PhD, FAMIA (17:34):
And that’s a good example because that’s not, so typically you think of normalizations. I got local labs and I want to map into standard particular with APHL, what data is, they had a master data, but it had gone a long time time since they sort of reconciled it with the standard and they found duplicates within their own data by sort of mapping it to itself and mapping it to SNOMED. So this is a more unique use of our mapping engine, but it was exactly to achieve normalization.

Sarah Brumley, MHA (18:05):
And from another perspective, we have some customers that are working on migrations across their sites right now and specifically working on labs procedures and they have subject matter experts participating in the mapping, making the decisions, but then they want to make sure that the local sites agree with those decisions that were made. So not only using some medical to do the mapping and automate that process with the human piece to make decisions, they’re also using waypoint, which is our web-based application for mapping so that the subject matter experts at the site level can get in there and validate that those mappings are the correct target and that is what that they need to be using at the site level. So some interesting nuances there to some of the mapping work.

Shaun Shakib, MPH, PhD, FAMIA (18:54):
Absolutely, Bonnie.

Bonnie Bruner, MSN, RN-BC (18:56):
So I’ll talk about some of the different approaches I’ve seen to data normalization. What is common is doing data normalization for a particular project or use case. And a benefit of that is you can show the benefit of the results pretty quickly, maybe even an ROI. But the next use case that comes down the road, a lot of times you’re having to do the same work over again for a different purpose. So what works well is when you can get buy-in to map everything, all of your, and you can hit the high points, the clinical domains, the labs, the medications, whatnot, and map all of that data with your editorial policy being just equivalency to the standard so it’s not for a particular purpose. And that way down the road when you kind of have this all cleaned up, you can use it for multiple use cases and it’s much more efficient as new use cases come up. Covid is a great example. No one expected it and I have clients who had already done that work and their ability to have the data points they needed was much more efficient than other companies.

Shaun Shakib, MPH, PhD, FAMIA (20:30):
I just want to add one more thing. This is not, so I think a lot of people think of normalization as a one-time effort, like bringing in my data, map it and then I’m good to go from there on. It’s a lifestyle choice. If you head in this direction, there’s ongoing maintenance, there’s the changes to the standard terminologies, there’s a change to your internal terminology. So you can’t think of this as just if I managed to get these into a lookup or a spreadsheet and I do a one-time mapping effort, I’m fine. It’s a decision to change the way you manage your data and you also have to have all the processes. And these guys mentioned some of those things like interrater agreement processes in place. We help with automation, we help the tooling, but editorial policy, these are all important things when you’re thinking about normalizing your data.

Charlie Harp, CEO (21:17):
I mean one of the things when it comes to data normalization, this is something that’s been happening for a while and in the early days of Clinical Architecture, this concept of normalization is one of the things that gave birth to our Symedical product and what happened was we were looking at a normalization project and they were, what we found is that people were normalizing a portion of things like the top 70 or 30% and they didn’t really have rules or they didn’t understand why or they were taking four people, locking them in a room with an Excel spreadsheet and making a sweatshop operation out of it. And so our goal really was we knew that doing this normalization is valuable because when you want to collect data from other places, you need to put it against a normative terminology to be able to reason over it as if it was kind of a homogeneous thing.

(22:10):
And so if you want to be able to do that at scale and get it all done, not the top 10, not the top 100, but get it all done, the data becomes liquid at that point. You could do anything with data that’s mapped and if you do it that way and you make it a lifestyle choice, like Sean says, what ends up happening is you end up getting a lot more consistent than having a project here and a project there because mappers change and humans, the same human can map something differently on Monday and Wednesday. So when you extend that over a big period of time across multiple projects, what you end up with is probably a pretty poor quality map. One of the things that we hear a lot here recently is the concept of artificial intelligence using a large language model to do mapping.

(22:56):
Now I’ll throw something out there, but I’m curious what you guys think. What I thought was interesting is we use a deterministic AI to do semantic normalization on each individual term and then we have a bunch of different types of algorithms that we deploy that do lexile linguistic semantic probabilistic mapping. And what’s kind of funny to me is we’ve been doing that for a decade and people still feel like they want to have a human being check it. So I don’t know how having a large language model do it is going to magically solve this issue. I think there are things we can do with large language models to enhance that and I think just like we have with other types of deterministic Ai, but I’m curious to see where that goes. What do you guys think?

Bonnie Bruner, MSN, RN-BC (23:47):
I mean I can tell you that I think I have four of the five top IDNs. I’ve never come across one that did not want to have human intervention and to validate and sometimes there’s more validation that’s needed on the front end to get some trust in those algorithms, but certainly want a need for human intervention and a lot of times have even multiple people verify that because the end result and what is sent out is affecting decision support and analytics, things that really matter.

Sarah Brumley, MHA (24:41):
Yeah, I don’t think that human component is ever or not for quite some time going to go away. Similar to Bonnie, I don’t have any customers that just say, blindly trust anything and there is no need for validation and oversight. So there’s always the level of trust that you build within some medical and the mapping and the algorithms is certainly there, but then the extra layer of confirming that it is what it is that they’re expecting, I don’t foresee that going away.

Shaun Shakib, MPH, PhD, FAMIA (25:19):
I think this is all really ironic because I’m actually Sean’s AI avatar. So AI, we’ll let you off the blockchain. I don’t know. I think when it comes to LLMs, AI, they have huge potential in our space and we’re recognizing those potentials and we’re going to take advantage of those potentials. Right now the things we are seeing are that AI does not fail gracefully. It fails catastrophically, but it’s very useful as yet another tool when we’re trying to match or when we’re trying to discover concepts and narrative text or it’s so we are going to be adding AI related technology to our matching mapping stack and our overall platform. But the points that these guys are raising around that doesn’t obviate the need to have an editorial policy, to have things like integrator review, to have somebody curate and review these kinds of things because of the fact that right now AI does not fail gracefully. It’s hard to keep it, it gives great answers maybe even nine out of 10 times, but the 10th time is something that no human would ever can fail so catastrophically that no human would’ve gone there. So it’s powerful technology. We’re going to incorporate the technology, we’re just going to incorporate it in a useful and appropriate manner into what we do.

Charlie Harp, CEO (26:47):
Yeah, I agree. I think that there are, the trick is with any new technology is how to figure out what it’s good at, how it can be applied, when it should be applied. Because the other risk we have, and this happened with NLP and healthcare about what five, 10 years ago and NLP wasn’t ready for healthcare and it failed spectacularly and it went through a huge hype curve and I mean people really stopped using it for a while. And so it’s one of those things where you can also undermine the credibility of a technology that’s good because you’re just not ready for it. I think one of the things, I was talking to somebody earlier today and we were talking about our technology using deterministic approaches can get you about 85% of the way there and it’ll find candidates for 10, 12%, but the ones that it can’t even address are not something that AI would be able to solve either unless AI picks up the phone and calls Muriel and says, Hey Muriel, what the heck is A 1 7 8 5 Q 11? I don’t know where that came from. There are always going to be issues where a human has to be involved at least until we standardize everything at the source, which that could happen.

Shaun Shakib, MPH, PhD, FAMIA (28:03):
So I did an exercise with some students and we just used chat tpt, so it was no specially trained model, but it was a treasure hunt for terminology against RxNorm. So we just said find the ingredient for this branded name drug and just basic source of queries. And you thought, well this constrained, we should get good results back with the 10 questions in the treasure hunt. And students did the work and then I did the work and I know that AI experts will say I didn’t prompt right and I didn’t have the parameters right, but it failed every question. And about midway through it even suggested that I go to a tooling specific to RxNorm to get the values back for these responses. So it has, again, it has value, it has tremendous value, you just have to figure out how to use it because you’ll be disappointed if you try to do something that’s not meant to do and you get a bad result.

Charlie Harp, CEO (28:56):
Alright, we’re going to do our last one content authoring.

Sarah Brumley, MHA (29:00):
So I have several customers that are using Symedical for content authoring specifically value set creation and ongoing maintenance and publication. I think that historically what we had seen is that these were maintained via spreadsheets and then the collaboration via spreadsheets and people essentially riding on top of each other, no single source of truth, utilizing Symedical for value set creation. You can create efficiencies there by having your single source of truth, standardized content, updates, all of that. But by creating a rule then you can also, instead of when new updates come out from the source, instead of combing through the entire update, it will provide what new codes fit within the rule that was created for your value set, which also creates more efficiencies for those authors. And the collaboration piece of it is huge because we can use our adaptive workflow for other people within the organization to review those value sets, suggest changes, make any modifications to the definitions, inclusion and exclusion criteria, and then come to a common consensus across. Because what I found is that as siloed as things can be, things like authoring is never a silo. There are always many people in that process that are going to have input that you need to seek a sign off or clarification from and so having a single source of truth but then a tooling that allows you to collaborate in a way that you’re not writing on top of each other and then finishing coming to a consensus and then publishing it out, that’s a huge use case that we’ve seen.

Bonnie Bruner, MSN, RN-BC (30:53):
So I think values that management and authoring of those is also very underestimated. When I think of data normalization, I think of normalizing to an equivalency. And when I think of value set management and the authoring of those, I think of that being more use case appropriate because there’s certainly quality metrics that you can create internally obviously for reporting purposes and you can have a value set for diabetes and your editorial policy matters. That documentation of what that represents really matters. It’s a small thing. How many times have you created a dashboard to show how we’re doing with diabetic management and it looks off and it’s because one includes gestational diabetes, which we may not want to include and the other is just type two. So I think having a description of what those represent is incredibly important. So I think it’s underestimated. And we see with organizations, I have an organization for example that’s created 700 plus value sets and they use them to inform various different use cases including pop health, including quality metric reporting, including point of care decision support.

Shaun Shakib, MPH, PhD, FAMIA (32:47):
I mean authoring is a big space, it’s a big topic. There are different levels of authoring and we have folks that you use protege build ontology. It’s something you’ve got to do in a transparent way, you have to have abilities to federate it out. So you can get a lot of authors involved in the process. But I think you guys have pretty much covered the topic. I mean I think that that’s an area that we continue to explore, continue to create capability in, and it’s in any lifecycle of content authoring to fill in the gaps or make content more appropriate for what you’re trying to do with it is a necessary step.

Charlie Harp, CEO (33:34):
And I think that’s another thing where something like a large language model could be beneficial because people that are authoring content, they might be doing something like categorizing something or coming up with consumer friendly terminology for something. And we’ve experimented with different approaches using things like specialist lexicon and WordNet and MLS Mesosaurus just so that when people are building something, they know all the different ways to say it and that doesn’t stop. There are things like translation, you want to say something in Spanish? Well what about consumer friendly Spanish? So large language models, once again I think managed by a human operator. What we find is a lot of the intensive work, when you take out the busy work, whether you’re mapping or authoring or doing anything, if it involves searching, searching multiple sources is a very intensive and time consuming process for a human being.

(34:32):
But once they find what they’re looking for, making a decision doesn’t take that long, especially if they know what they’re doing. So even having something like a large language model go out and say, well here’s 43 potential synonyms for this thing. A human could look at that very quickly and say, well that’s too broad or that’s not quite right and what it can really be is an accelerant. And so that’s kind of the idea when we look at these use cases for artificial intelligence in all of these things, it’s really how can we accelerate and make an individual that’s doing this kind of work do like the work of 10 people and get rid of all the kind of busy and frankly the kind of work most humans don’t like to do anyways. I mean pharmacists sit in a room mapping acetaminophen to acetaminophen is probably not what they were dreaming of when they went to school. So any last minute thoughts before we turn it over to these guys for questions? These guys being Joe? Alright, any questions from the audience? Joe!

Audience Speaker 1 (35:36):
That was a fabulous panel and Bonnie particularly, you got me thinking about master data and organizational use of it to create integrity and quality. I’m interested in either your thoughts or Charlie’s thoughts on the challenge that several vendors have had in history and some recent history where orders went to queues that weren’t being monitored and as a result orders. And so the thought is if you had master data for all the possible places where an order could go, then as an organization you could use that as a system to make sure that there was accountable parties, there was an accountable schedule for reviewing it, that sort of thing. And I was wondering whether those uses for master data in policy and policy planning is something that you have seen in your install base. Bonnie?

Bonnie Bruner, MSN, RN-BC (36:33):
I have not come across a use case for orders, but I would say the same thing applies. They can be kind of all over the place and a lot of times can be customized by what the provider wants. Certainly order sets can be per provider. So I would say the idea there is I’ve seen organizations try to wrangle that, try to develop kind of a best care path for certain disease cohorts and what not. But again, the idea of having an application to be able to have visibility into all those orders to be able to normalize if you will, or kind of roll up what those orders are and make sure they represent the same thing. And then lastly, having the right people being able to look at those orders, review them and make sure they represent the same thing.

Shaun Shakib, MPH, PhD, FAMIA (37:42):
I just say provenance and metadata.

Charlie Harp, CEO (37:45):
I was going to say Joe didn’t say you could answer.

Shaun Shakib, MPH, PhD, FAMIA (37:49):
I can’t? I’ll just add, I’m just going to add on, I’ll enrich a little bit. So prominence and metadata, so more metadata around. So for example, an order having all the necessary metadata to be able to track it, to make it specific to your organization or how it’s being used, what specialty areas it’s appropriate for. So all that additional method is key. And then provenance. So what’s the history of this? How’s it been deprecated historically? How was this categorized for example? So over time you may make a different decision about where you want, what bucket you want that in over time having some sort of history that you can go back and say at this point in time, this is how we categorize this or this was the specialties, these are the list of specialties associated. So that kind of just basically if you can’t measure it, you can’t manage it if you don’t have the necessary information to be able to do aggregation or query or analysis. So it’s just having allowing master data to be flexible enough to hold all of the additional metadata you need to do a good job of using that content in your organization and then having a good way to track the history of how that has changed over time.

Charlie Harp, CEO (39:03):
So the way I think about it, and I think this will address your question, but I’m not going to be held to that. Historically in healthcare it was like switchboard operators that were connecting calls by moving the wire from place to place. And if you use that as an analogy, that system broke down because the amount of information, the amount of connections necessary just outpace the number of people that could be sitting in chairs and dealing with that volume. I think that’s happening in healthcare and I think a lot of the things we do in healthcare rely on wet wear. It relies on the human brain to remember that I’m supposed to follow this process and I’m supposed to check this queue and I’m supposed to read your allergen list before I prescribe something. And I think what’s happening is with all the information and the aging population and the reduction in the number of providers that we have, we really have to start relying heavily on technology to help offload some of this work.

(40:09):
Technology requires rules, it requires an encoded ontology, and ontology is essentially a system of beliefs. And to me, master data is that systems of beliefs. It is the rules, it is the rails, it is the connectors in my train set of how I want things to operate. And I think that one of the things that I find a little concerning is we as humans are like, and I hate to say this, but we’re kind of lazy and the thing that scares me most about concepts like a large language model is we’re like, oh thank God I can just let the large language model do it. And large language models or machine learning, those things are good at looking at what’s happening, but they’re not necessarily good at determining what should happen. And so us looking at using things like machine learning and large language models to see what’s happening so that we can determine how we can create the rules and the rails with master data or logic that we put into practice so that software can use automation to actually help us from ending up in a situation where in order sitting in a queue and nothing’s happening.

(41:27):
I mean the truth is anybody, I recently went through something with my brother where he surgical procedure and I was his go-to person. I went to all the visits, I talked to the doctors and it’s scary, it’s frustrating. Did I mention that it’s scary when you’re going in and you’re like, wait, wait, wait, wait, wait. Why are you giving him seven diabetes medications to take home? You should only be on two. Those types of situations. There’s so much information, there’s so many people involved, there’s so many things happening for us really to do a better job in healthcare. We’ve got to up our game in terms of defining those rules and creating the master data that gives the software the guardrails. Is that okay for a long-winded answer to your relatively straightforward question? Excellent. Any other questions? We’re a little over time. We might know secrets of the universe, you’re missing your chance, sir. Alright. Hey you guys, thank you so very much. Thanks for coming up today and to the audience, super fan, Joe Bommel. Thanks everybody and have a great rest of the show.