In October 2015, Orem, Utah-based KLAS and several major health IT stakeholders collaborated toward the creation of a viable measurement tool of interoperability, which was then used as a base for KLAS’ Interoperability 2016 report this past October.
The 2016 report, inclusive of interviews with more than 500 electronic health record (EHR) end-users, focused on the “four bases” of interoperability, which were that patient records must be (1) available, (2) easy to locate, (3) situated within the clinician workflow, and (4) delivered in an effective way that facilitates improvement in patient care. KLAS found that true “home runs” in which all four criteria are met were only reported by 6 percent of providers surveyed. Provider respondents were also asked about their experiences with specific health IT vendors.
Two key members of this work were Bob Cash, vice president of provider relations at KLAS; and Tim Zoph, chair of the KLAS Interoperability Measurement Advisory Team (IMAT). Zoph also spent years as a healthcare CIO and is now client executive and strategist with Naperville, Ill.-based consulting firm Impact Advisors. While the report’s core findings can be read here, and access to the full research can be gotten here, Healthcare Informatics wanted to get a “behind the scenes” look at the process in developing the tool to measure interoperability, what to take away from this year’s report, and what could be in store for future KLAS reports. Below are excerpts of interviews with both Cash and Zoph.
Last fall, KLAS and a bunch of major health IT stakeholders collaborated around a tool to measure interoperability. Then you published a comprehensive report on data sharing. Can you take me through this process?
Cash: Well, it wasn’t easy. I should give credit to four key leaders in the industry on the provider side, at least for the initial draft of the report’s questionnaire—John Halamka, M.D., [CIO of Beth Israel Deaconess Medical Center]; Dan Nigrin, M.D. [CIO of Boston Children's Hospital], Micky Tripathi, Ph.D., [president and CEO of the Massachusetts eHealth Collaborative]; and Stan Huff, M.D. [CMIO of Intermountain Healthcare]. Those four helped us put together the initial draft which we then took to the leaders of the major EHR vendors along with a group of providers last October. And there was back-and-forth battle with that—not as much about the content or context, but about making sure that what we asked would get to the issues and concerns in the industry. There was discussion around if we needed to know about the volume of transactions shared, but we opted not to ask about that. It was more about the perception and experience of people using the information.
As far as the creation of the tool, last year we had about 50 percent vendors and 50 percent providers [at our Keystone Summit]. But we had a broader group this year. We had Carequality, CommonWell, HIE [health information exchange] vendors, and representatives from ONC, CHIME, AMIA, AMDIS and the Center for Medical Interoperability. Our vendor representation this year was people who are responsible for interoperability in their organizations, rather than the CEOs and presidents.
Zoph: This process started over a year ago where we put a number of things into motion at a Keystone Summit in Utah. We thought it was important that if we were to put a measurement tool together from both the provider and vendor sides, they must agree on those measurements. So we wanted a collaborative view of what interoperability is. KLAS became a convener; we saw them as a trusted third party to ultimately carry forward whatever we agreed in the measurement framework to do. They were willing to put resources behind the measurement.
We wanted to start with the interoperability around EHRs by bringing together the major providers of EHRs. They all showed up. We had some leading healthcare informaticists and CIOs put together the initial draft of a measurement framework, so there was some pre-work done leading up to the conference. But the goal of the conference was to see if we could get everyone to show up for a multi-day meeting, and use this draft framework to come up with an initial set of measures that everyone could agree on. And then we would have KLAS go out and do its 500 interviews.
We recognized that any time you lay out a first-year measure, you know you will learn things along the way. We put a measurement advisory team together that had a subset of those leaders, which I chaired. The team would be available to KLAS if they ran into issues with the survey. We knew we would learn a lot from the first year, so we decided to do the measurement first, and go back in year two and do a longer-term roadmap since we were starting with a narrow definition of interoperability. Then we could agree on a performance scoring tool which we did not want to put in the first year since it was more about just getting the data back. We got back together this fall to take a look at what it is we learned from the study, how to modify it, put together performance measurement benchmarks. and then lay out a longer-term roadmap with a broader definition of interoperability—so information exchange that wasn’t solely attributed to EHRs in the acute care environment.
As far as challenges, there was a good consensus from the group that we needed to move beyond the measurement of moving transactions around and have it more about the context of that information— how it was ingested and then presented back to the caregiver, and whether it was in the workflow and made a difference. A key concern, and KLAS has addressed this, was that a lot of their surveys typically go directly to the CIO and then gets parsed out to the director teams. But we thought for interoperability it was important to talk to clinicians and have the perspective of end users. That was probably the hardest thing—are we getting this to the right people? KLAS reached beyond the CIO for this one, as over half of those 500 conversations were with someone in a clinical role or someone who can represent clinicians.
What were your biggest takeaways from the report’s findings?
Zoph: We measured both interoperability and “intra-operability;” so we took a look at how well individual EHRs share amongst like clients. Epic was a leader in that work due to everything they do with Care Everywhere, but for the rest of the industry, it’s challenging to share information even among like records. I thought that might be easier to do but was a big challenge.
We have a long way to go when we look at the industry more broadly. We found that even when transactions are available, they’re often not in the workflow, are not easily retrievable, and people are struggling to talk about the difference it makes in care. That’s no surprise to people who are close to this.
Cash: You hear concern and frustration from clinicians. This was performed in a way to try get at least a majority of our conversations with people who could represent the clinicians’ views best, so a lot of CMIOs and CMOs who could express what their colleagues and they themselves were experiencing. The sentiment out there was that it’s not working, and we found out that to be true. Having the information available in a way that’s easy to find and in the workflow is not that common, so it’s not surprising that the impact on patient care is low, too.
The other perspective is where we are with standards. The infrastructure today that is the base of how we exchange are these HIEs, but if you look at their ratings, they’re the least valuable. I think what’s happened is we are relying on a level of infrastructure based around these regional HIEs, and there is momentum in the industry to go to something more contemporary like FHIR [Fast Healthcare Interoperability Resources] and APIs [application programming interfaces]. We need to move through a generation of exchange and get to something more contemporary. HIEs will likely have a role still but to get where we want, we need to embrace more contemporary standards for how it is we exchange data.
What could we look for in next year’s interoperability report?
Cash: We would like to have more pieces of information that are more actionable. So if people are saying that the data they are getting is not on impactful on patient care as they would like it to be, what would make it more impactful? So we might add some questions and we might remove some that we didn’t learn much from.
Zoph: We will continue to keep the Interoperability Measurement Advisory Team together as we interact with KLAS and meet at least once per quarter. We are also sitting down with ONC to brief them directly on interoperability, as there are a lot of questions at the federal level. We will sit down with the tool every year, and ask: do the questions work, does the context work, and what can we do to improve the baseline information that we’re already collecting?
Also, we have agreed on a performance measurement tool which we think goes beyond the transactional level, and the goal is to understand if this makes a difference in care coordination and the efficiency of healthcare. We additionally want to go beyond acute care. With population health and where we are going with the continuity of care, we are going to look at interoperability challenges into post-acute care.
If you had to predict when “true” interoperability will be achieved, what timeframe would you say sounds best: within three years, three to five years or more than five years?
Zoph: I’d like to say it will come in the three- to five-year span. We will need a sustained commitment by all parties in the industry to say we will continue to make interoperability a priority, not only in the functionality in the products but how providers continue to demand it in the purchasing of the products and actually implement it when the capability is available.
Cash: It’s a five-year journey more than a two-year journey. The numbers are low. If I were to measure optimism, and I can’t, I think it would be reasonably high. Things spurring that are the level of collaboration we are hearing and seeing from vendors and providers, the emphasis from a regulatory standpoint from entities like ONC, and then there is also optimism for these facilitating groups like Carequality and CommonWell. There just is not a lot to measure and validate that’s happening and effective. But in the few instances we were able to do that, people are positive and hopeful for what’s to come.
