On July 1, the Department of Health & Human Services unveiled a patient safety plan in response to calls for health IT to play a greater role in reducing medical errors. Central to the effort is the push to establish mechanisms that facilitate incident reporting among users and developers of health IT.
I just had a great conversation with someone on the front lines of the effort to make health IT safer and put EHRs to use to help clinicians avoid errors. Jeffrey Gold, M.D., a pulmonary critical care physician scientist at Oregon Health & Science University (OHSU) in Portland, is studying the role of simulation in improving usability of EHRs and is working toward integrating EHR simulation into residency and fellowship training.
Gold and his colleagues recently published a paper in the journal BMJ Open on a pilot project they ran in which they created a five-day simulated ICU patient in the Epic EHR including labs, hourly vitals, medication administration, ventilator settings, nursing and notes. Fourteen medical issues requiring recognition and subsequent changes in management were included. Issues were chosen based on their frequency of occurrence within the ICU and their ability to test different aspects of the EHR user interface. ICU residents, blinded to the presence of medical errors within the case, were provided a sign-out and given 10 minutes to review the case in the EHR. They then presented the case with their management suggestions to an attending physician. Participants were graded on the number of issues identified. All participants were provided with immediate feedback upon completion of the simulation.
Of the 38 participants, including 9 interns, 10 residents and 19 fellows, the average error recognition rate was 41%. Over-sedation was the least-recognized error (16%); poor glycemic control was most often recognized (68%). Only 32% of the participants recognized inappropriate antibiotic dosing.
Gold, his colleagues and the participants themselves weren’t prepared for how few errors the participants picked up. “I was shocked that it was this consistently bad,” he said. “We are doing a second round with many more participants and the results are similar to the first, in the 40 to 50 percent range.”
Those missing errors included residents and fellows, so it is not just a knowledge thing, he said. “But whether the problem is an educational one or an EHR user interface one, I don’t know. Probably both. We can look at new ways to organize the data on the screen and study which has a positive impact on recognizing errors.”
OHSU researchers also are going to expand the study by looking at participants’ eye movements to see what they are looking at in the EHR during their 10-minute review.
I asked Gold what had been the response at OHSU to the results of the pilot. “The response at the hospital has been the realization that the EHR is a powerful tool and not enough is being done yet to harness its power," he said. "We are finally seeing the issues and complexity of the EHR and now we can quantify the extent of the issues and the relevance to users.”
OHSU went from paper to using an EHR from Epic in 2009. Gold said that the training for how to use the EHR is largely generic and doesn’t have enough context for the ICU.
“When talking about patient safety, people often make reference to flight simulators. Well, with those you respond to dangerous situations,” he said. Pilots are not just shown obvious things like two engines fell off or you took off with no fuel. It is more subtle things like you are gradually losing altitude at a slow but constant rate, and can you notice that. EHR simulation has to have that same fidelity to the subtle nuances of the clinical setting, he added.
Gold and team created a copy of Epic just for training purposes with five days worth of data following clinicians’ traditional workflow in the ICU. “We want to know whether they can see the forest for the trees. If training modules have only one day’s worth of data, that’s just a few trees, not a forest.”
As the OHSU team’s paper concludes, gaps in identifying dangerous medical management issues suggest that EHR-specific training may be beneficial, and simulation provides a novel tool in order to both identify these gaps as well as foster EHR-specific training.
Simulations of EHRs for medical training would require a lot of resources to maintain, but they might provide some real bang for the buck in terms of patient safety improvements.
