Study: Inaccuracies in EHR Problem Lists Pose Problems for Risk Adjustment
Inaccuracies in EHR (electronic health record) problem list–based comorbidity data can lead to incorrect determinations of case mix, according to a study that took place at two southern California Veterans Affairs (VA) medical centers.
For the study, researchers compared EHR problem list–based comorbidity assessment with manual review of EHR free-text notes in terms of sensitivity and specificity for identification of major comorbidities and Charlson Comorbidity Index (CCI) scores. The CCI contains 19 categories of comorbidity and predicts the 10-year mortality for patients who may have a range of co-morbid conditions.
The researchers then compared EHR-based CCI scores with free-text–based CCI scores in prediction of long-term mortality. The overall goal of the study, which was recently published in the American Journal of Managed Care, set out to determine whether comorbidity information derived from EHR problem lists is accurate enough.
Whereas comorbidity data from inpatient medical records are reviewed by trained coders, outpatient records may be less reliable, as they often rely on “problem lists”— a compilation of patient diagnoses entered by clinicians during patient encounters and updated at varying intervals—in the EHR to identify the index condition. With increasing numbers of EHRs being used to store patient data across healthcare organizations, interest has grown in utilizing these lists as a source of comorbidity data. However, it is unclear whether the data in these lists are sufficiently accurate to assess patients’ total comorbid disease burden, the researchers hypothesized.
As the researchers noted, EHR-based data is increasingly being used for purposes of risk adjustment. But the findings of the study revealed that the Veterans Affairs EHR problem list led to incorrect determinations of case mix and did not predict survival, in contrast to chart-based comorbidity assessment.
Speaking to the importance of accurate EHR problem list-based comorbidity data, the study’s authors pointed out that that other recent research has highlighted the inconsistency of different sources of data (eg, registries, claims, the EHR) for identifying basic health information, such as major comorbidities. For individual physicians, these inconsistencies are less relevant because they have the opportunity to confirm this information directly with the patient. “However, when used for risk adjustment for purposes of performance assessment, incorrect data may lead to misclassification and unfair comparisons. This is a major concern for health systems participating in alternative payment models, which base some portion of reimbursement on risk-adjusted quality outcomes,” the researchers stated.
Specifically, the study found that EHR problem list–based comorbidity assessment had poor sensitivity for detecting major comorbidities: myocardial infarction (8 percent), cerebrovascular disease (32 percent), diabetes (46 percent), chronic obstructive pulmonary disease (42 percent), peripheral vascular disease (31 percent), liver disease (1 percent), and congestive heart failure (23 percent). Specificity was above 94 percent for all comorbidities. Free-text–based CCI scores were predictive of long-term other-cause mortality, whereas EHR problem list–based scores were not, the researchers said.
As such, the researchers concluded, “Inaccuracies in EHR problem list–based comorbidity data can lead to incorrect determinations of case mix. Such data should be validated prior to application to risk adjustment.” They added, “Other sources of comorbidity information, such as patient-reported measures of health status or natural language processing–derived data, may be considered for risk adjustment in comparisons of performance.”
