Research: EHR Predictive Model Outperforms ED Social Needs Questionnaire
A research study supported by the Agency for Healthcare Research & Quality has found that a machine learning predictive model leveraging several EHR data sources outperformed an emergency department screening questionnaire in predicting future need for health-related social services. The study’s senior author, Joshua Vest, Ph.D., M.P.H., a Regenstrief research scientist and professor of health policy and management at the Indiana University Fairbanks School of Public Health, recently spoke with Healthcare Innovation about the implications of the research and next steps.
Healthcare Innovation: Is collecting health-related social needs information becoming a higher priority for healthcare organizations because it's part of some quality reporting requirements or because they are focused on reducing total cost of care?
Vest: The impetus for so many organizations to screen for health-related social needs is coming from two directions. One, you have the institutional players that are using this information as part of quality reporting. CMS has it on the inpatient side as part of their quality metrics. It's also an NCQA metric. The Joint Commission has some language around screening requirements as part of their requirements as well. On the other side, you also have the push from the opportunity to improve cost, quality and care.
HCI: What are some challenges around conducting the patient questionnaires on this in the emergency department? Is it challenging to fit into the workflow when people are in that setting?
Vest: Screening is important. It has a definite role. But as with all screening, there are challenges. There are a couple that have been identified through a lot of work in the literature. One challenge is workflow. It's one more set of questions that somebody has to ask and somebody has to answer. Everybody’s busy. There are some concerns about survey burnout, too, among the clinicians, the staff and the patients.
It is also a cultural shift. Traditionally in the U.S., social care and healthcare have been very separate. They're different organizations or different entities. Now a lot of healthcare organizations have had as part of their mission thinking about people more holistically, and a lot of clinicians are super supportive of addressing the entire needs of people. But as a system, they've never really worked together cohesively and systematically, and it's a cultural shift. It's something that a lot of individuals in healthcare weren't trained to deal with directly, and you have to build the workflows and the processes to not only identify patients with needs, but once you identify somebody with a need, you have to do something. Patients aren't satisfied if nothing happens. And physicians and nurses are not happy asking a question and having no answer.
HCI: This study sought to determine if using patient data from the health system EHR and predictive models could perform as well as the questionnaires at identifying patients with social needs. Can you explain how it was set up to compare those two things?
Vest: We went into our clinical partner’s emergency department and surveyed about 1,100 adults who were visiting the ED for care. We gave them a survey that was outside of their care practice, so it wasn't something that we shared with their clinical providers. We also got consent to link their responses to electronic health records from our clinical provider, as well as our health information exchange. With that combined data, we were able to look on one hand to say, ‘Okay, how did you answer your screening questions?’ And then from the EHR and health information exchange data, we were able to both look forward in time to say, ‘Okay, did you have encounters?’ And then we were also able to look back in time and pull basically everything we could that the health system already knew about the patient to build large models to predict those same outcomes.
HCI: I read that from their address you could get area deprivation index scores as well.
Vest: Yes. there’s been a longstanding interest in how we think about and measure social needs. In the screening questions you ask the individual and you know that individual level characteristic; however, where patients live also matters. It affects their opportunities. It constrains their resources, or gives them access to resources, so with the address information from the EHR, we're able to map to their area characteristics and put those into the models as well.
HCI: Could you talk about the results and whether the predictive model did better at identifying the people than the questionnaire did?
Vest: About four in 10 of the patients had a subsequent need that we could identify after their emergency department visit. We found that the screening questionnaire didn't do particularly well in predicting that. Adding gender and age to the questionnaire model actually did make it better, and that's something we think is kind of important. But in comparison, the EHR models did a slightly better job. We're not going to say they were perfect by any stretch of the imagination, but they were doing a better job of predicting that future health-related social service need than the screening questions by themselves.
HCI: Could you talk about the potential for bias you identified?
Vest: If we think about healthcare access, it is not equitable across the country, right? There are individuals who have more opportunities to access care than others. The biggest and most obvious example involves individuals who are insured vs. those who are not. If you are uninsured, it's much harder to access care. That, of course, is correlated with things like social needs, so that affects the data. If I have insurance, there is more likely to be more data available on me than somebody who does not have insurance.
The purpose of all this screening is to get people the services they need. We want people in need to get to the services that help them. We want to make sure we're being fair. We don't want to penalize anybody, and we don't want to promote anybody at somebody else's expense, simply because we're not able to model it well enough, or we know that there are inherent problems in the data. So that's one of the things we always check for, is trying to make sure that we're not perpetuating those problems in the data.
HCI: I read that you’re hoping to develop tools that can be integrated into the EHR systems to make the process of identifying and addressing health-related social needs easier and more effective for everyone. Would that be something like a clinical decision support pop-up saying this patient might need help with one or more health-related social needs right in the clinician workflow?
Vest: We could do this a couple different ways. One option would be clinical decision support. And there are a lot of places across the country that are trying such things and have been working very hard to build those into workflow. It could be a pop-up or a reminder for the clinician, but it also could be somebody else in the workflow, such as registration. If we built in modeling, we could build that in to say that high-risk individuals, according to the models, are referred to screening at registration. Or it's pushed to their patient portals, or there is some other process that catches individuals. So it could be integrated in a way to help be more efficient in who you're trying to screen.
HCI: Would most large health systems have access to the same types and quality of data that your team working on this research project did in order to create something similar?
Vest: We try to draw on things that health systems have. Now, there are always variations between health systems. I think the biggest concern that we come across is what we discussed earlier — that the data is not the same for everybody right now. Clearly, large integrated systems are going to have a bigger reach and be able to catch instances of care in other locations outside their own facility. If you've got a stand-alone single hospital with no connections to primary care practices, their data is going to be much more sparse, and you would be building different models or trying to think about the data in different ways.
HCI: Are you doing some follow-on research from this work?
Vest: We are building that decision support step we talked about. We are getting ready to roll out a trial where we have built models that indicate whether a person is at risk for each of these social conditions, and we have built them into our existing health information exchange platform that is accessible to providers in the ED, so it's integrated into an existing decision support platform — it is almost like one extra chief complaint, if you will, to help ED providers identify what kind of challenges are going on. We plan to roll that trial out this spring at IU Health.
HCI: You said this is being built into the state’s HIE platform. Does that mean that people could eventually use it at other hospitals around the state?
Vest: Yes, we're building it into the HIE platform so it could be accessible to other individuals or other settings.
