CHAI Releases Best Practice Guides for 4 Use Cases

The nonprofit Coalition for Health AI (CHAI) has released a series of best practice guides (BPGs) and testing and evaluation (T&E) frameworks for four new use cases: General Health Advice Chatbot, Prior Authorization, EHR Information Retrieval, and Clinical Decision Support. 

Founded in 2021, CHAI is focused on developing a set of consensus-driven guidelines and best practices for responsible AI in Health, as well as supporting the ability to independently test and validate AI for safety and effectiveness.

The guides and frameworks, developed through CHAI’s collaborative work group engagement, are designed to help healthcare organizations safely evaluate, deploy, and scale responsible AI. CHAI said the resources will be living documents, and are accessible and freely available to the public. 

CHAI’s BPGs and T&E frameworks reflect a consensus-driven development process made possible by the coalition’s use case-specific work groups. More than 100 industry experts, including representation from clinicians, health systems, startups, academia, industry, advocacy, policy, and more contributed to their design. Through collaborative and consistent work group meetings, CHAI was able to gather consensus across participants by analyzing real-world challenges, aligning evaluation methods, and dissecting processes that health systems can operationalize to safely adopt AI.

“CHAI remains committed to our mission in building the broadest possible consensus across the healthcare ecosystem to help ensure AI is trusted and safe,” said Brian Anderson, M.D., CEO of CHAI, in a statement. “Work groups are the crux of this mission, as they allow us to convene experts, gather feedback, and create dialogue to inform our BPGs and T&E frameworks. I’m thrilled to see the product of this collaboration with these new resources released today.”

The BPGs surface challenges, insights, discussions, and current consensus-defined practices for developers and implementers, giving organizations a look into what is happening in the rapidly evolving field of health AI. The T&E frameworks provide options for how developers and implementers can operationalize evaluation and monitoring of responsible AI principles as they relate to specific use cases. 

“We recognize that new evidence and experiences will always arise,” said Merage Ghane, Ph.D., director of responsible AI at CHAI, in a statement.  “As such, CHAI has made the feedback process for these documents a continuous one. We wanted to ensure that anyone could submit an issue form with specific change recommendations for the BPGs and T&E frameworks. These suggestions then contribute to our new version updates, along with a summary of changes and feedback responses – all keeping the process entirely collaborative.”