MENDS (Multi-state EHR-based Network for Disease Surveillance) is a 5-year pilot project to implement a national sentinel chronic disease surveillance system, using a distributed analytic network to enable near-real-time surveillance of chronic disease prevalence in the population of patients under care. The project team is building an OMOP-to-FHIR Bulk Data service and will provision the data warehouse via a FHIR Bulk Data client.
Speaking June 7 at HL7’s FHIR DevDays event in Cleveland, Bob Zambarano, Ph.D., vice president of healthcare analytics for Commonwealth Informatics, described the MENDS infrastructure and the work on FHIR Bulk Data.
The MENDS project is funded by the CDC and managed by the National Association of Chronic Disease Directors. It uses the technical elements and design of the successful Massachusetts state-level distributed analytic network for chronic disease surveillance: MDPHNet
ESPHealth is the open-source, free-license product suite used by MDPHNet and MENDS. ESPHealth is an acronym for “EHR Support for Public Health.” It was developed by Harvard Medical School’s Department of Population Medicine and Harvard-Pilgrim Health Care Institute, originally under a CDC Centers for Excellence grant.
Zambarano said that ESPHealth is essentially a sibling to and functions analogously to two other distributed analytic network systems: PCORNet and FDA Sentinel. Multiple data partner EHR systems provision their own ESP DataMart. The PopMedNet Web Portal app is used to distribute custom analyses from a central hub and collect results. Each DataMart pushes a monthly aggregate data structure to the RiskScape visualization tool.
How does it work? “You'll have multiple clinical data partners, each with their own EHR systems, which provision an ESP data mart, which has a standard data model,” Zambarano explained. “It's a limited data set typically, in that it doesn't have all the patient data, but it has patient-level data from the electronic health record system, and that sits behind a firewall. But on a monthly basis, it provisions a visualization tool that is available on the web to authorized users who can see aggregate-level information.”
The PubMedNet tool is used for distributing queries, so users can create queries to run against the data models in each of the data partners’ connected ESP systems, he added. “Those queries are run through a local data mart client that connects to the database, generates aggregate results, and sends those back to the portal for review by the analyst.”
The four initial MENDS data partners using ESPHealth were OneHealthPort with coverage in Washington state, REACHnet (Louisiana Public Health Institute) with coverage in Louisiana and Texas, Alliance Chicago, with coverage primarily in Illinois and Texas, and Regenstrief Institute, with coverage in Indiana.
The MENDS project began looking for a fifth data partner in late 2020, Zambarano said. “In order to promote the transition to the FHIR standard, the CDC was encouraging utilization of FHIR APIs for projects developing data systems that interoperated with EHR data. The MENDS team spent several months looking for a large regional healthcare organization interested in participating in MENDS and that was ready to implement a FHIR bulk data service,” he said. “Health Data Compass, a healthcare data research institute with the University of Colorado, was interested in developing a FHIR bulk data service as part of their data warehouse system. Design work began in early summer 2021.”
Zambarano said the ESP DataMart system is distributed with a number of pre-developed ETL [extract, transform and load] systems:
• PCORI CDM to ESP
• OMOP CDM to ESP
• C-CDA to ESP
• Epic Clarity to ESP
• Epic Cache to ESP
• AthenaIDX (GE Centricity) to ESP
“ETL remains a significant pain point, especially when working with HIEs or other health data systems that rely on bespoke reporting data warehouses,” he said. “Given the current regulatory imperative for the FHIR API, adding the FHIR bulk data client to the ESP repertoire of ETL was a longstanding goal.”
Getting into a little technical detail, he described the data flows for the UC-HDC FHIR Bulk Data Service. Starting from the UCHealth clinical data warehouse, data moves through Cloud Composer and AllSpark to MENDS OMOP data mart. OMOP data is then serialized to JSON using Google Cloud Dataflow. OMOP JSON moves through Whistle Transform to FHIR NDJSON. A request to the FHIR bulk data service triggers the Cloud Dataflow and subsequent Whistle transform process.
He also spoke about how data flows for the ESPHealth Bulk Data Client. The ESP FHIR Bulk Data client is a Python app. Requested resources are: Patient, Encounter, Condition, Observation, Claim, MedicationRequest, Immunization. Authentication uses Google Heathcare API via pre-shared account key. NDJSON from the FHIR bulk data service is loaded to Postgres JSONB. A Python Django app is used to parse NDJSON resource structures to the ESP data model.
Reflecting on lessons learned so far, Zambarano said, “The bulk data and resource APIs are so well specified that the development work was pretty much the easiest part around building the client and services. The significant effort really is understanding the source and target data structures and the transformations that had to occur.”
As always in these projects, the technical folks tend to focus on the things that are technical, Zambarano said, “but the biggest barriers are always coordinating with and obtaining appropriate authorization from the different organizational entities responsible for data privacy and security.”
He also stressed that these files are quite large, because they are provisioning a system that's trying to establish information about patient medical histories and chronic diseases, going back to 2017. The NDJSON file for observations is close to a terabyte in size.
“This has been a significant effort, but we feel like this is a great addition to the ESPHealth system,” Zambarano said in closing. “Subsequently, we hope to add additional partners into the system who can use this and make the process of joining the MENDS project or other ESPHealth projects more straightforward for organizations that want to join.”
