Vanderbilt, Duke to Lead Work on AI Maturity Model Framework

Vanderbilt University Medical Center (VUMC) and Duke University School of Medicine are leading an effort to develop an artificial intelligence maturity model framework.

The health systems have been awarded a $1.25 million grant from the Gordon and Betty Moore Foundation for the project “Measuring Artificial Intelligence (AI) Maturity in Healthcare Organizations.”

Working with the Coalition for Health AI (CHAI) and the University of Iowa, a team of experts will leverage the grant to develop a maturity model framework. The project leads are Peter Embí, M.D., M.S., and Laurie Novak, Ph.D., M.H.S.A., from VUMC and Michael Pencina, Ph.D., and Nicoleta Economou, Ph.D., from Duke.

In April 2023 CHAI published a blueprint for trustworthy artificial intelligence deployment in healthcare. The group said the 24-page guide reflects an effort among subject matter experts from leading academic medical centers and the healthcare, technology, and other industry sectors, who collaborated under the observation of several federal agencies over the previous year. 


The framework envisioned by Vanderbilt and Duke will outline the essential capabilities that health systems must establish to ensure they are well prepared for the trustworthy utilization of AI models.

“The promise of AI for improving health and healthcare is great, but there is currently a wide gap between promise and reality,” said Embí, professor and chair of the Department of Biomedical Informatics and senior vice president for research and Innovation at VUMC, in a statement.

“This work will produce new tools and capabilities that our health system needs to ensure that we select, deploy and monitor health AI to make healthcare more safe, effective, ethical and equitable for all,” he added.

Health systems are actively developing and employing algorithmic technologies; however, the research team notes that there is a significant gap in terms of oversight, resources and organizational infrastructure. The gap hinders the ability to comprehensively document which algorithms are currently deployed, what their value is, who is monitoring them and who is accountable for their use. This gap compromises the safety, fairness and quality of these technologies. 

“If we are to realize the full potential of AI technologies, health systems must develop a more mature process for implementing these tools. Improving oversight of AI technology in healthcare systems is crucial for ensuring the safety and efficacy of patient care,” said Pencina, chief data scientist at Duke Health and the Duke School of Medicine’s vice dean for data science, in a statement.

“With this grant, we’re taking a significant step toward ensuring safe, accountable, standards-based deployment of AI in health systems,” Pencina said. 

“Our team at the Vanderbilt Department of Biomedical Informatics (DBMI) has a long tradition of scholarship on technology and organizations, starting with Dr. Nancy Lorenzi, who paved the way for an informatics focus on workflow and management. We’re very excited to work with our partners at Duke, the University of Iowa, the Coalition for Health AI and the Moore Foundation to build an empirically supported maturity model for healthcare AI,” said Novak, associate professor in DBMI and director of Vanderbilt’s Center of Excellence in Applied AI, in a statement.

Over the next year, VUMC and Duke teams will engage different stakeholders from CHAI and various health systems to outline the significant components health systems should have in place for the trustworthy implementation of AI. 

“Creating a maturity model framework for health AI will enable health systems to identify their strengths and weaknesses when procuring and deploying AI solutions, ultimately driving the transformation of healthcare for the better,” said Economou, director of Algorithm-Based Clinical Decision Support Oversight at Duke AI Health.