The Agency for Health Research and Quality (AHRQ) and the MITRE Corp. have worked with the American Academy of Family Physicians (AAFP) to pilot a standards-based application programming interface (API) that disseminates findings from multiple research data repositories through a single location, with the goal of making it easier and faster to find and use evidence in research, quality improvement activities, and clinical practice.
The AHRQ Center for Evidence and Practice Improvement (CEPI) maintains public repositories of research evidence and patient-centered outcome research findings. Those repositories include the Systematic Review Data Repository (SRDR), the Effective Health Care Program (EHC), CDS Connect, and recommendations from the U.S. Preventive Services Task Force (USPSTF). AHRQ has a goal of making these repositories more findable, accessible, interoperable, and reusable (FAIR). The CMS Alliance to Modernize Healthcare (Federally Funded Research and Development Center), operated by MITRE, supports the CEPI Evidence Discovery And Retrieval (CEDAR) project toward these goals.
The goal is that health IT developers will use CEDAR to integrate AHRQ CEPI research findings directly into their existing systems, where the findings can then be accessed and used by researchers, clinicians, policymakers, patients, and others.
With more than 130,000 members across 50 states and territories, AAFP was able to recruit end users of AHRQ evidence from diverse backgrounds in healthcare, including guideline developers, practicing family physicians, medical educators, and researchers, for the pilot. AAFP also had the technical capacity in-house to develop an application into which the CEDAR API could be integrated and then tested by the pilot participants. Finally, AAFP staff and members actively use CEPI evidence and expressed familiarity with different repositories indexed in CEDAR. A pilot kickoff meeting was held on Feb. 16, 2022.
As a September 2022 report on the pilot states, CEDAR API enables users to search and access CEPI research evidence that exists in multiple repositories from one place, making it easier and faster to find and use evidence in research, quality improvement activities, and clinical practice. AAFP created and hosted “CEDAR Search,” a client application prototype enabling pilot participants to interact with CEDAR’s features and test the functionality of the CEDAR API.
The pilot assessment found that clinician and researcher participants alike were receptive to using CEDAR. Seven out of nine clinical participants used CEDAR Search in a clinical setting, and five out of those seven found it helpful in that setting. Some clinician participants expressed caution about using CEDAR with patients, as well as enabling patients to use CEDAR Search on their own, suggesting that the information may be too technical and does not address patient needs. When asked whether CEDAR Search had positively impacted their work, six research participants indicated it had positively impacted their work, and eight expressed interest in using CEDAR Search in the future. These findings indicate that it will be feasible to integrate the CEDAR API into either research or clinical settings in the future through a user interface similar to CEDAR Search, the report said.
Clinician participants did not generally report barriers to using CEDAR Search, some using it both in and out of the clinical setting. Clinicians envisioned that CEDAR Search and other applications of the CEDAR API would be useful in a variety of ways, such as:
• Collecting updates on treatments and clinical management.
• Planning for patient sessions.
• Accessing accurate medical information, quality information, and research information.
• Educating others accessing data and developing evidence.
• Finding additional evidence-based platforms or websites. One clinician participant observed that CEDAR Search was geared toward people who want only AHRQ resources.
Three clinician participants used CEDAR in a clinical setting; two of these participants used CEDAR with a patient or in response to a patient question. Six clinical participants reported that they would use CEDAR if made publicly available. Of the research participants, six indicated that using CEDAR positively impacted their work, and eight expressed interest in using CEDAR Search in the future. Researcher participants found benefit from CEDAR Search’s aggregating information from different repositories (leading to time savings) and highlighting high-quality, evidence-based resources. All researcher participants also reported confidence in the search results, noting that the results came from trustworthy sources.
Barriers to Regular Usage of CEDAR
Clinician-identified barriers included the inability to obtain a search result that was helpful in the moment in clinic, as well as results unsuitable for use during a patient visit. Clinicians raised several shortcomings in applying the CEDAR Search application to patient use, including a lack of patient-level medical information, provision of guidelines without sufficient guidance or context, the inclusion of potentially outdated information, and the likelihood of making patient visits longer.
Lessons Learned
Among the lessons learned is that evidence included in the CEPI repositories does not necessarily lend itself to use at the immediate point of care during a clinical encounter; instead, they may be useful for gathering information before or after the session. Reasons for this include the type of evidence artifacts AHRQ offers in the CEPI repositories and the pace with which the evidence is updated. Pilot users reported unexpected results, as well as results that did not address what the pilot participant was looking to find. Initiatives seeking to leverage CEDAR would benefit from ensuring a description of indexed repositories is easily found or available to end users.
Among the recommendations for future work:
• AHRQ should devote additional resources for indexing data into CEDAR. While participants noted the benefits of the limited results and expressed interest in using a CEDAR application in the future, they also consistently shared a desire for more AHRQ content in its search results.
• In future pilots or when developing demonstration user interfaces, the CEDAR team should partner with health science librarians and quality improvement researchers at academic medical centers to develop applications that would be useful and accessible to clinicians, researchers, students, and faculty alike.
• MITRE and any future API steward should provide guidance and recommendations to external developers for building a user interface that includes varying levels of CEDAR functionality, based on end-user needs. For example, a mobile interface might allow clinicians to use CEDAR in point-of-care settings or integration into the electronic health record (EHR) as a tool for planning patient encounters, communication, and follow-up. Developers may leverage insights derived from this pilot, including functionalities that pilot users frequently requested.
• Developers seeking to integrate the CEDAR API into an application may benefit from drawing on lessons learned when training anticipated end users about using CEDAR (e.g., reference information focused on searching, Boolean logic, and filtering to improve the relevancy of search results).
