PIQXL Gateway: The Key to Improving Patient Data Quality

Charlie Harp:
Good afternoon. I’m Charlie Harp. I’m the CEO of Clinical Architecture, and today I’m going to talk about the PIQXL Gateway. Now to talk about the PIQXL Gateway, I kind of got to give some background on the PIQI Framework. So if you’ve already sat through a PIQI Framework presentation, feel free to check your email, do some texting, whatever you want to do. So it all started a little over a year ago, actually, almost two years ago when I did a presentation on the taxonomy of healthcare data quality. And I talked about Lord Kelvin and I talked about the HDQT, the Healthcare Data Quality Taxonomy. I talked about using USCDI potentially as a model. And I wrote a whole utility that scored the data and showed what it looked like and talked about evaluation criteria as an idea of how people might want to go about measuring quality. And I was invited to do a presentation at the ONC Data Quality Symposium.

And during that I did some more studies on lab results and medications. And then I had an idea, and the idea turned into this concept that we’re going to call PIQI. The Patient Information Quality Improvement Framework. So the idea was that having a taxonomy wasn’t enough. Talking about it theoretically wasn’t enough, but we need to have a way to score patient data that we’re getting from somewhere else. And I did some research. I looked at ISO 8,000. I looked at a bunch of other things and I didn’t really find anything that the pragmatist in me resonated with in terms of a workable solution for what we were talking about. So my primary objective was to come up with a common approach for measuring, improving, measuring, and improving the quality of patient information in a message regardless of the message format. So whether it’s HL7 V2, whether it’s CCDA, OMOP, X12, FHIR, doesn’t matter.

I want to be able to take the message, I want to be able to assess the payload in that message. So I landed on four core design principles. The first was you need to have a standard model, a least common denominator model that I can build assessments against using a common approach. The second is you need to have some kind of a taxonomy so you can understand kind of the root cause, likely root cause analysis for why things, why you’re having particular issues. You need to create a modular, portable and shareable way of doing assessments so you can establish a community of practice so that instead of us toiling away in our healthcare silos like we do, that we could create something that we could all leverage together. And the last is allow for the concept of a user definable evaluation profile where you can build standard profiles or you can build custom profiles to satisfy unique use cases.

Now to satisfy those design principles, the first thing was the simplified data model. And because I’m not a big fan of reinventing the wheel, I went somewhere where I saw there was a very decent lease common denominator model, and that’s the USCDI or US core. So when you look at this model, it basically says, these are the things that I want you to measure so that you’re in compliance. So what you’re looking at is essentially the core of the PIQI patient model, those things as flat elements. So that’s the PIQI model. The nice thing about this flat model is it’s very easy to extend the model, add new attributes. And as USCDI evolves, as our needs evolve, we can easily leverage the model because it’s not a recursive hierarchical thing. It’s a very flat element, object driven thing that could be easily expressed in something like JSO or XML.

The next thing was the healthcare data quality taxonomy. Now, the healthcare data taxonomy or quality taxonomy, it addresses the core things that plague us when we’re thinking about data quality and healthcare. And it falls into four categories. Now, this is version two. I fully expect that this is going to evolve as we interact with more people in the community of practice. But initially you have availability, accuracy, conformity and plausibility. And what this does is it addresses that things have to be there or we can’t use them, that things have to be accurate. If we’re expecting a date, we should be getting a date. There is a lot of things that we share in patient data that’s codified data and they have to be valid codes and to be interoperable, they have to conform to what we agreed to. And if you do all of these right, then what’s left is plausibility.

I always say that the first three are the vegetables and plausibility is dessert because with plausibility you can say things like this result unit doesn’t match the LOINC code. This value can’t possibly be right. You can’t have a hemoglobin A1C of 10,000. So it lets you take a look a deeper look at the quality of the data beyond just the, is it a date, is it a number type of quality check? But you have to do those first to get to the plausibility. And then when you think about the things these apply to, whether it’s in simple attribute and element or it’s the entire patient, these things can come into play. And once again, this is going to evolve over time and that’s the taxonomy. The third principle is a simple assessment module, A simple assessment module, and I’m going to go into geek speak.

It’s a composable service endpoint. And what I mean by that is it takes in a standard thing and it returns a standard thing. So in the case of the simple assessment modules, a simple assessment module takes an input like a simple attribute or a lab element or a med element. And every SAM returns one of three things pass fail. I couldn’t do the test. The reason for that is at the end of the assessment, you end up with a simple numerator denominator. I tested a thousand things, I passed 900 of them. Your score is 90%. It’s not a crazy weighted thing. It’s a very straightforward 90%, 50%, you be the judge. The other thing is every SAM is assigned into the taxonomy. If the SAM is this field is not populated, then it goes into the availability taxonomy. This allows us to collect statistics to see what is the nature of our main problem in the quality of the data.

And the other thing about Sam’s is Sam’s can be parameterized. So I can pass in a simple string parameter. And the other thing is Sam’s can have prerequisites. For example, for a code to be conformant with L, it has to be a valid code to be a valid code, it has to be complete to be complete, it has to be populated. So all those things have to happen for me to even be able to test if it’s a conformant code. And I’ll get into that. In fact, I’ll get into that right now. Here’s some example. Sam’s the attributes populated. It’s a valid date. The concept is complete. The concept is valid, it’s active, it’s compatible, and the lab result is plausible. Now if we look at those, the input for the first two is just a simple attribute. I pull a piece of text out of a field and I run it through this to see if it’s valid.

These guys are based upon a concept. The thing I’m evaluating starts out as a simple attribute, and then I determine if it’s a concept. And if it’s a concept, I can perform these tests. And then when I get to this level, I’m actually taking all the attributes for a single lab result and I’m feeding them to the sam. And the SAM can look across the attributes for things like concept is compatible. I can pass in a simple list of code systems that I’m evaluating. And then when you think about prerequisites, the concept’s compatible. It has to be valid for the concept to be valid, it’s got to be complete. And just like I said earlier, and then the Sams are bound to the HDQT.

Now, once you have a library of sams, it can be reusable. For example, if I’m assessing a patient message, I’m going to use the is populated a lot. It’s going to be the first gate that every single attribute has to get through to get to the next step or the next sam. So I’m reusing these simple assessments all over the place. I’m really not building a bunch of specialized tests, I’m just reusing the SAMS in my library and I’m using them in something called an evaluation profile. So the evaluation profile establishes the assessment criteria. What is the rubric and how am I scoring? It aligns the SAMS to the PIQI information model and it determines the scoring approach. And lemme give you a for instance. So let’s say I have an evaluation profile of USCDI version three, and let’s say I’m looking at lab tests.

And within the lab test for the lab test, it’s got to be conformant with lo, I am scoring that and it is not conditional. And what that means is every lab test, I’m going to check to see if it’s lo and if it’s not, the denominator gets a point and the numerator does not get a point. If it is, the numerator also gets a point. And I win for lab order, it’s the same thing and I am scoring it. And as I go down the list, I can say the lab result value here is going to be snomed. The lab result unit has to be in the UCUM list. This concept for the specimen type has to be in snomed. But when I get down to this last one where I’m saying the lab test has to be a semantic match, what I’m saying is the original lab test and the LOINC code need to be semantically similar or I fail and that’s informational.

I’m not going to score that, but I want to track it. I want to know if it happens or not, but I’m not incorporating that into my score. It’s just because curious right now, these guys originally I didn’t have the concept of waiting because I don’t like the concept of waiting. Well, not waiting, I don’t like that either. But a weighted score kind of blurs the picture. So what we have is we do have the ability to weight the score. So you can say this is worth three points, this is worth one point and potentially skew your result. But what you can also do is you can say these things are critical. For example, it is critical that the code that the lab is LOINC coded. I cannot use the data if it’s not LOINC coded. And that way you could have a score that’s 90%, but you have critical failures, so the data’s not usable. So it lets you decide how to balance out the score versus criticality.

And so once again, the intent here is to create a scoring rubric that I can use. And so for example, I can create a scoring rubric and I can go in ASTP and I say, I want to work with you to create a standard rubric for USCDI version two. In version three, I can go to NCQA and say, I want to have a standard rubric for scoring the data, whether or not it’s usable for quality measures for HEDIS measures, I can have something that’s for oncology data to say, does the oncology data have all the things that are necessary for me to be able to leverage that data? And there’s a lot of possibility and there’s a lot of ability to share that information across the community as well.

Alright, now this is a little steep. So brace yourself. I want to make an important point about scoring. When you’re collecting data, you’re really doing three things. You’re scoring a message as it passes through the gateway, you’re keeping track of how many entities coming from that source were acceptable and you’re also understanding why they failed. Those are three different things, and I’ll try to explain it. So if I have a message that comes in and let’s say this is the only test I’m doing and it passes all the prerequisites and the assigned sam, so I get one point for my numerator, my denominator. So my message itself gets a point a hundred percent. My lab test statistics are 100%. There’s nothing in my SAM statistics because nothing failed. I send the next lab test through it fails on concept is valid. So my message is zero, I get no points.

My point up here is zero means my cumulative lab test score is 50%. So what I’m saying is of the lab test that I received, 50% were good, 50% were bad. And my SAM statistics for lab test is my failures are 100% because the concept’s invalid. There’s only one more. In this example, the attribute wasn’t populated. So once again, my message doesn’t get a point. My lab stats are now 33%. So only a third of my lab tests passed the test from this data source. And the reasons they failed are 50 50. Does that make sense? It’s important because when I’m looking at the data statistically, and you’ll see this when we get into the PIQXL demo, these things matter because for example, if the data’s not available, I can’t tell you if it’s going to be conformant. So there’s a whole pyramid effect that happens when you look at the stats. So the assessment process is pretty straightforward. I take the message and I align it, which is really a process of extracting the data out of the message syntax and putting it into a simplified model. I align it to the PIQI model. I choose my evaluation criteria, which aligns it to the simple assessment modules. Those are aligned to the HDQT. I run the message through and it generates a report and generates a score that tells me you got a 67% quality score with 17 critical failures.

Now you can learn more about this. I’ve produced documentation, which if you know me, I don’t love producing documentation, but there is a PIQI handbook. There’s a message format guide, there’s a SAM guide which goes into nauseating technical detail on how this thing works. And there’s going to be more the Rubrik Guide and the API guide is coming out. We’re working with Levitt partners to kind of organize the PIQI alliance, which will be the group that helps drive the standard. And we’re also moving PIQI through HL7 so that it can be a recognized standard for doing data quality assessments. And that’s the QR code for the PIQI Framework.com website. If you’re interested, you can go there and say, Hey, keep me involved or I want to be part of the conversation. And we encourage your participation. So the PIQXL gateway is our implementation of the PIQI Framework. So what we’ve done is we’ve taken the PIQI scoring engine, we’ve incorporated into an endpoint, a commercial high performance platform.

Whichever direction the PIQI Framework goes in, the PIQXL gateway will conform with the way that PIQI moves in the future. So it’s kind of like PIQI on steroids if you will. It organizes data into channels. It collects source metadata, things like facilities, formats, use cases, applications. It collects statistics on volume and quality. It provides tools for sampling the data so you can see why it’s wrong and do detailed analysis. And more importantly, remember PIQI is about improvement, not just measurement. So part of what we’re doing with the PIQXL gateway is coming up with ways where we can articulate to the source, here’s your priorities for making this data usable. Here are the things you have to do.

It’s designed to support high scale cloud base and on-premise operations. We are in beta right now. In fact, I’m going to show you guys some beta stuff in a minute and we’re planning on rolling it out and making it commercially available next quarter. So sorry, go ahead Susie. So what are some of the features? Well, there’s three modes. There’s standalone where you could just send in PIQI formatted data. And the way that works is PIQI just measures everything. And you can actually call it and get a detailed audit report for an individual message. You can say, what’s my score and where did I go wrong? And it’ll mark up your original content and give it back to you. We can integrate pivot into Pivot, and I’ll talk more about that. And we’re also looking at doing PIQXL on FHIR. So we can take in native FHIR and do PIQXL against it, but honestly, because we’re pushing this out in the open source, I’m also going to be releasing a guide that says, here’s how you take data out of FHIR and put it into the PIQI format so that anybody can do it. You don’t have to do it through us.

We’re going to produce all the standards, the SAMs, and we’re going to make it available. So you can create your own Sams behind a restful endpoint. We’re also going to create custom content for Sams to do things like disjoint conditions, to do things like plausibility checks for lab results and medications. We’re also going to support any standard rubrics that come out with our collaboration with PIQI Alliance will be supported. And also we’ll have it set up so you can customize your own as well. The concept of channels I’ll talk about. But the idea with a channel is a channel is something where you route messages through PIQXL gateway. So for example, you can have a pre-mediation and post remediation channel, so you can compare how you’re able to improve the quality of the data in flight. You can also have, I’m going to show you guys a version two, version three UCDI channel.

So I can see how my data did against version two, how it will do in version three and potentially how it’ll do in version four when that comes out. And like I just said, channels can be chained together for comparison purposes. And as I said earlier, well I’ll show this to you, the detailed scorecards. And of course you can also monitor an alert. So if all of a sudden something goes terribly wrong and your messages quality starts to drop through the floor, the PIQI gateway can tell you that’s happening. And so the PIQXL gateway is designed to work with Pivot and does the pre and post remediation views, it can quarantine or gate messages. So if it gets a single message that has a lot of quality issues, it can actually stop it from moving into your data repository. And I’m sure as we continue to work with our clients, we’ll create more capabilities along those lines. So how about a demonstration? And this is where something that holds the microphone to be useful. So I’m going to just talk loud.

So what you’re looking at here is a live PIQXL gateway. And this first dashboard is really showing you for this entire channel, what is all the message volume, how many clean messages do we have? Zero how many CRI messages with at least one critical failure? That’s the red line. And then down here is your quality score. So for the last 30 days, I can see that my quality has gone up 8%. Now ideally what would happen is you would give somebody feedback and say, you need to fix these 14 things. And you’d see an increase in quality. But what you’d probably do if you wanted to do that is you go to the data sources tab. Now in the data sources tab, you can see the different data sources, see their message volume and see the nature of their quality. Now it shows you the quality score here it shows you the top 10 reasons why they’re having quality issues.

And for any one of these, I can also drill into the details. So let’s take for university health, what I’m seeing here, let me adjust this a little bit. As I’m seeing the message volume over the last 30 days, the cumulative quality score, the average quality score, the critical failure rate, and the clean rate, I can see that my quality is down a little bit less than 1%. I can see what my top five quality issues are. And if I go down to the model, I can see the performance on each one of these. So allergy substance, my reasons, and this is where the Sams come into play, the reasons for failure, 66% is unpopulated, 9% invalid concept and 25% the display in the coatings unpopulated. And if I want to see this, well, if I want to see the overall pass rate, I can see that actually 57% of these terms passed.

And I can also go into a view where I can look at the detail grid and I can see them in the order in which they occur. So the first SAM is unpopulated display unpopulated invalid concept. Now if I’m saying, well, what the heck’s going on? I’ve turned on logging for this source. So I can actually go in and I’m going through the HIMSS wifi, so it might take a second back to Indiana. So I can see what are the reasons why these things are failing. And so these are just purely uncoated things that are coming through the allergy track. Those things could all be remediated by something like Pivot, for example.

So the other thing I can do, so this allows me to go through and look at the individual entities. So here’s all the demographic entities, here’s the lab result entities. And I can also go into a view from the classes. I can say, well, how am I doing overall my classes? Well, the only place you’re having critical failures is lab. If you look at immunization, I’m at 94% and that means I’m getting lot numbers for 94% and valid CVX or NDC codes for 93% or 94%. So the other thing we can do is if you’re trying to figure out how can I remediate? And by the way, we have things based upon the taxonomical dimensions. We have things based upon the application. If you’re using multiple applications, the formats, you can see what your informational Sams are. And this one is whether the test code is active in LOINC. You can also go in and look at a remediation strategy where you can say, well, what if I were to fix some of these things in remediation? What would my quality be then? So you can kind of assess the things that are remediable and the things that are not remediable.

And the last thing I’ll talk about is, as I said before, I can see that when I run this through a USCDI version two rubric, my scores are much higher. So I can jump over to that and I can look at it in the USCDI version two rubric and see the things that, what my failure rates were there, how my classes are doing there. And as you can see with USCDI version two, I’m doing a lot better than I was with version three because it didn’t have the same requirements. Alright, well that’s pretty much it ladies and gentlemen. I really appreciate your time and attention and as I said before, check out the PIQI alliance or the PIQI Framework.org website if you have any questions. I’m happy to answer them now. Alright, thank you. I appreciate your time.