A key area of focus these days for chief medical information officers is deploying ambient AI assistants for clinicians to record patient encounters and cut down on the burden of EHR documentation. Healthcare Innovation recently spoke with Matt Keefer, M.D., chief medical informatics officer at Children’s Hospital Los Angeles about the results of a pilot project involving Nabla Copilot.
Keefer is a member of the Department of General Pediatrics at Children’s Hospital Los Angeles and a clinical professor in the Department of Pediatrics at the Keck School of Medicine at the University of Southern California.
Nabla Copilot is also in use at The Permanente Medical Group, Mankato Clinic, Frontier Psychiatry, and others.
Although based in Paris, Nabla is focused on the U.S. healthcare market. Founded in 2018, the company has proprietary large language models and speech-to-text technologies built by former Facebook AI Research engineers.
HCI: Could you talk a little bit about some of the EHR challenges your organization has had — and that hospitals in general have — that make something like Nabla’s ambient AI assistant attractive? Are there burnout issues related to EHR documentation you wanted to address?
Keefer: I think I can safely say that no one went into medicine with the intent of documenting. That brings very little joy to anybody's work day. With the expansion of the EMR, there are some wonderful things, including 100 percent availability of the chart to those who need it. There are also some downsides, which involve a lot more typing, and especially for clinicians who aren't super tech-savvy, it means a lot of interaction with the system that they think is sometimes a barrier between them and their patients.
HCI: There are several companies entering this ambient AI assistant space. How did your organization decide which one to work with? What drew you to Nabla?
Keefer: We certainly did look around at some of the different options. One of the things that really drew us to Nabla was the responsiveness of the team, the interest in improving the product with us and being able to understand how our clinicians were interacting with the patients. In particular, we have a fairly large percentage of our patients and families who speak languages other than English, Spanish being the predominant one. They were very open to understanding that workflow, and trying to ensure that we could could be as as efficient as possible with that.
HCI: You conducted a pilot with 30 pediatric specialists. What are some things you had to tailor specifically for the pediatric hospital setting?
Keefer: We have surgical subspecialties, medical subspecialties, and general pediatrics, and clearly those all have slightly different variations in terms of how their visits go. We also deliberately chose people in different parts of their career — senior faculty, junior faculty, those who are tech-savvy, those who weren’t, and some who were already using things for dictation. We worked with the general pediatricians to come up with a better template for keeping track of information that you would have in a typical well-child exam. Nabla was able to respond to specific requests for how to format the note in a way that would be germane to a pediatrician.
HCI: What about integration with your Oracle Cerner EHR? Is that something that came right out of the box, or did you have to do some work on that?
Keefer: I would say that no integration comes without some work. Certainly, that was one of the other things that we really enjoyed working with Nabla on. We came up with workflows that worked for integration with Oracle Health, and the team from Nabla had already been connected with them, and we're working on on the next steps to further that integration.
HCI: Were there things that you wanted to measure during the pilot in terms of doctor-patient interactions or clinician time spent documenting to see some improvement before moving on to a larger implementation?
Keefer: We did. We have Oracle Health data to look at time documenting after hours and things like that. We also did structured questionnaires with all the pilot users, and the Nabla team has them rate the notes to give them feedback on on the quality of the notes and what needed to be changed.
HCI: Do you feel positive enough now that you're going to roll it out organization-wide?
Keefer: Yes. We’re hoping to start that up with a group of 27 pediatricians next week. We also want to try use cases outside of just the physicians. So we're working with social workers and some other team members, such as psychologists — anybody who has a narrative experience with the patient.
Also joining the conversation were Nabla CEO and co-founder Alex Lebrun and Chief Medical Officer Ed Lee, M.D.
HCI: Dr. Lee, tell us about your background and how you came to Nabla?
Lee: I am an internal medicine primary care physician. I've been with Nabla as the chief medical officer officially for about three weeks now, but I've actually been working with Nabla for quite some time. I previously served as the chief information officer for the Permanente Federation, which is sort of a consulting advisory body to the Permanente Medical Group. I was in that role for about five years and with Kaiser Permanente for over 20 years as a practicing clinician and I still practice there now.
In the pilot work that we did with Nabla early on at Permanente, we actually did see a decrease in time in charting anywhere from 25 to 30 percent within each particular encounter. If you’ve got 20-plus encounters per day, that adds up to a couple hours per day.
HCI: What are some things that distinguish Nabla from the competitors entering this space?
Lebrun: I think the fact that we start from physicians, from the users, and go bottom up is a very, very big differentiation. We push our product directly to physicians. Actually, there is a free version that they can use for free for up to 30 encounters per month. And then they become our champions. So we start to have users inside the organizations and they become our champions and eventually, we have enterprise contracts.
That has big consequences, because since we designed the product to be self-service, you should understand it in two minutes by yourself without any training. Otherwise these bottom-up approaches don't work. So it is very easy to deploy. This is one of the differences.
Something else is we recognized from the start that every organization has different processes. I've never met two physicians who agree on what is a best note. So we know we need to customize. It's more like a platform that we need to customize for every customer. For instance, pediatrics was very different for us because there are maybe two parents, two children and one interpreter in the room. It's very complicated. We had never seen that before. But we could adapt the product in a few weeks to support that. We don't see this as an off-the-shelf product, but more as a platform that we can very quickly adapt to every customer. So we think this makes us a little bit different due to our ability to do that.
Lee: That's how I originally heard of it at Kaiser Permanente. We had some users try the free version. Then they started contacting me and said, ‘Hey, have you seen that? We need to get this here within our organization.’
One of the things that attracted Kaiser to Naba was their responsiveness, their innovation, their ability to really partner. I love the word partner rather than the word vendor because they really did come in and listen to our needs, adapt to our needs, and then build and customize. Every physician writes notes in the way they want to write it. It's a very personal thing, actually. And so the different type of customization options that are built in weren't initially built in with Nabla, but through the partnership they were built in, and I think that's a similar experience that Dr. Keefer may have had at CHLA as well.
