Could an Effective AI Sepsis Tool Soon Be Widely Commercially Available?

I read with considerable interest an article published on November 27 online in NEJM AI, the journal supplemental to The New England Journal of Medicine that addresses artificial intelligence (AI) development. The article, entitled “FDA-Authorized AI/ML Tool for Sepsis Prediction: Development and Validation,” was authored by a very large contingent of researchers—52, to be precise; but the lead five named authors are Akhil Bhargava, M.S., Carlos López-Espina, M.S., Lee Schmalz, B.S., and Shah Khan, Ph.D.

And what the 52 researchers describe is an observational clinical study involving analysis of outcomes for patients at five major medical centers, with the goal of determining whether the first comprehensive AI tool approved by the Food and Drug Administration (FDA) that has been designed to predict the early emergence of sepsis among hospital patients, was differentially more effective. The article’s authors write that “The Sepsis ImmunoScore demonstrated high accuracy for the identification and prediction of sepsis and critical illness metrics that could enable prompt identification of patients at high risk of sepsis and adverse outcomes, potentially improving clinical decision-making and patient outcomes.” And they note that the study received partial funding from the Defense Threat Reduction Agency.

The article’s authors write that, “To address the need for a diagnostic and risk assessment tool in the hospital setting, we developed the Sepsis ImmunoScore, intended for integration with an electronic medical record (EMR), which uses machine learning (ML) to identify patients likely to have or progress to sepsis within 24 hours of patient assessment. It was granted marketing authorization (de novo pathway) by the U.S. Food and Drug Administration (FDA) in April 2024 as the first-ever artificial intelligence (AI) diagnostic tool authorized for sepsis.”

And, per that, they write, “The objective of this investigation was to evaluate the performance of the Sepsis ImmunoScore and its ability to risk-stratify patients for the presence or development of sepsis (as defined by Sepsis-3) within 24 hours, and for secondary end points of in-hospital mortality, length of hospital stay, intensive care unit (ICU) admission, mechanical ventilation, and vasopressor medication use.”

The researchers write in the article that “Study inclusion criteria consisted of hospitalized adult patients (18 years of age or older) who had a suspected infection, as defined by the clinical decision to obtain a blood culture, and who had a lithium-heparin (Li-Hep) plasma sample drawn within 6 hours of the first blood culture order that was available for collection. There were no exclusion criteria. Participants were assigned to one of three cohorts: a derivation cohort (n=2366) where the algorithm was derived, an internal validation cohort (n=393) that assessed algorithm performance on a second set of participants from the same hospitals used in the derivation, and a final external validation cohort (n=698) of participants from hospitals not involved in the algorithm derivation.”

Importantly, they write, “The primary end point was the presence of sepsis at presentation or within 24 hours of study inclusion using the Sepsis-3 criteria: suspected infection and a sequential organ failure assessment score of 2 or greater from baseline. The derivation cohort used a Sepsis-3 outcome derived from the medical record in an automated fashion,9,10 while the internal and external validation cohorts used expert clinical adjudication to apply the definitions and determine the Sepsis-3 outcome. The clinical adjudication occurred in a retrospective fashion and was carried out by clinicians who likely did not treat the patient but had access to the entire hospital chart and utilized information including laboratory testing, radiology testing, and clinical assessment and decision-making documentation. The secondary end points consisted of sepsis-related metrics of critical illness: in-hospital mortality, length of hospital stay, ICU admission, use of mechanical ventilator, and use of vasopressors.”

The article’s authors note that “The Sepsis ImmunoScore algorithm analyzes up to 22 input parameters to generate a risk score and place patients in one of four discrete risk stratification categories. The parameters consist of demographic data, vital sign measurements, comprehensive metabolic panel measurements, complete blood count panel measurements, lactate levels, and sepsis biomarkers PCT and CRP [procalcitonin and C-reactive protein are biomarkers used to detect systemic inflammation; PCT is generally considered more specific for bacterial infection and is often preferred for monitoring sepsis]. The interventional SHAP values indicated that the three most influential parameters of the model were PCT, respiratory rate, and systolic blood pressure. The AUC in the derivation set was 0.85 (95-percent confidence interval: 0.83 to 0.87) for the medical record–derived sepsis outcome. Additionally, the Sepsis ImmunoScore risk categories were associated with increasing risk of sepsis in the derivation set.”

They note that “The Sepsis ImmunoScore demonstrated high overall diagnostic accuracy for predicting sepsis, with an AUC in the derivation set of 0.85 (95 percent confidence interval: 0.83 to 0.87) for the medical record–derived sepsis outcome, and 0.80 (0.74 to 0.86) in the internal validation and 0.81 (0.77 to 0.86) in the external validation for the adjudicated sepsis outcome.”

The researchers write that “The Sepsis ImmunoScore is a comprehensive, multidimensional AI/ML tool that combines demographics, vital signs, clinical laboratory tests, and sepsis-related laboratory tests to assess risk of sepsis and risk of adverse outcomes. In this study, we developed the Sepsis ImmunoScore and evaluated its ability to serve as a risk-stratification tool for patients with suspected infection and to predict the diagnosis of sepsis and adverse clinical outcomes. We found the Sepsis ImmunoScore highly predictive of sepsis and the secondary outcomes of in-hospital mortality, length of hospital stay, ICU admission, mechanical ventilation, and vasopressor administration within 24 hours. We also found that the Sepsis ImmunoScore was predictive in diagnostic (predicting if a patient has sepsis at initial evaluation) and prognostic (predicting if a patient without sepsis at initial evaluation will develop sepsis within 24 hours) approaches.” They acknowledge that “Several FDA-approved tools for infection diagnosis are available, but they are often limited to detecting one or more blood biomarkers.”

The researchers emphasize that this study was “observational,” and “did not assess the impact of the Sepsis ImmunoScore on clinical decision-making or changes in therapeutic approaches. So yes, it’s early days. But I am eager to hear what clinician leaders working in this space have to say about this study, and whether the study points to what might amount to a major breakthrough in early sepsis detection using a tool that the study indicates will be generalizable to heterogeneous hospital organizations nationwide. At the very least, this article points up the fact that things are moving far more quickly now than many might have predicted, in a key area of applying AI to patient care.