Last week, the Centers for Medicare & Medicaid Services (CMS) published the first release of its overall hospital quality star ratings that the government says reflects comprehensive quality information about the care provided at U.S. hospitals.
Indeed, the new overall hospital quality star rating methodology takes 64 existing quality measures already reported on the agency’s Hospital Compare website and summarizes them into a unified rating of one to five stars. As written by Kate Goodrich, M.D., director of center for clinical standards and quality at CMS, at the time of the release of the ratings, “The rating includes quality measures for routine care that the average individual receives, such as care received when being treated for heart attacks and pneumonia, to quality measures that focus on hospital-acquired infections, such as catheter-associated urinary tract infections. Specialized and cutting edge care that certain hospitals provide such as specialized cancer care, are not reflected in these quality ratings.”
Earlier this year, the American Hospital Association (AHA) and other industry stakeholders wrote a letter to CMS pushing the agency to delay the release of the new ratings system, originally scheduled for April. The AHA has long opposed the star ratings system, arguing that it doesn’t display an accurate portrayal of quality or patient experience. As such, CMS said it listened to stakeholder feedback, delaying the release of the ratings until now “to give hospitals additional time to better understand our methodology and data.”
Despite the delay, in a July 27 statement, AHA said, “We are further disappointed that CMS moved forward with release of its star ratings, which clearly are not ready for prime time. As written, they fall short of meeting principles that the AHA has embraced for quality report cards and rating systems. We want to work with CMS and the Congress to fix the hospital star ratings so that it is helpful and useful to both patients and the hospitals that treat them.”
Similarly, the Association of American Medical Colleges (AAMC) also released a statement criticizing the quality ratings, attesting that “They are based on a deeply flawed methodology that does not take into account important differences in the patient populations and the complexity of conditions that teaching hospitals treat.” The organization further took issue that facilities are being measured on an equal basis, which should not be the case at all since that’s not the reality that hospitals face.
What’s more, a recent Kaiser Health News report dug further into the ratings, revealing that just 102 of the more than 3,600 hospitals that CMS rated received the top rating of five stars. If that doesn’t jump out to you, maybe this does: Medicare gave its below average score of two-star ratings to 707 hospitals, including some of the most prominent and well-renowned organizations in the country such as Geisinger Medical Center in Danville, Pa., Beth Israel Medical Center in New York City, Tufts Medical Center in Boston, MedStar Washington Hospital Center in D.C., and many others, according to Kaiser Health News.
Due to these bizarre results, I dug deeper into Medicare’s ratings, which include 64 of the more than 100 measures reported on Hospital Compare, divided into seven measure groups or categories: mortality, safety of care, readmission, patient experience, effectiveness of care, timeliness of care, and efficient use of medical imaging. CMS collects the information on these measures through the Hospital Inpatient Quality Reporting (IQR) Program and Hospital Outpatient Quality Reporting (OQR) Program, noting that “a hospital’s overall rating is calculated using only those measures for which data are available.”
According to CMS, for each hospital, a hospital summary score is calculated by taking the weighted average of the hospital’s scores for each measure group or category. The table below shows the weight applied to each measure category. The hospital summary score is then used to calculate the overall rating.
Source: CMS
To this end, AAMC said that CMS used more than 60 measures to calculate ratings for teaching hospitals and as few as nine measures on some hospitals that treat patients with less complex conditions or that treat a limited number of conditions. AAMC also said that its analysis of the ratings “has confirmed that the lower the number of measures a hospital reported, the more likely a hospital was to receive a higher star rating. In fact, hospitals that reported on only 60 percent of the metrics or less received almost half of the five-star ratings.”
In the aforementioned Kaiser Health News report, Elizabeth Mort, M.D., chief quality officer at Massachusetts General Hospital in Boston, which received four stars from CMS, was quoted as saying, “I don’t put any credence in this. Don’t clutter it up with measures that have no place being there,” such as infection and readmission measures that she said were not well designed to compare dissimilar hospitals.
To get a better sense of these measures and how accurately they might reflect a hospital’s quality, I chatted with Rita Numerof, Ph.D., co-founder and president of St. Louis-based consulting firm Numerof, where she has 25 years of experience in helping healthcare organizations deliver better care at a lower cost through a variety of strategic approaches. Numerof is a fan of CMS publishing these ratings when it did, noting the agency’s overarching desire to connect payment to outcomes. A big proponent of greater transparency and accountability for healthcare cost and quality, Numerof understands that there will always be some resistance to change, but feels that these ratings—even if not perfect—are a significant step towards that goal.
“A lot of this has to do with risk adjustment and if an organization has not done its coding appropriately internally, which translates to how they get paid. So they can be indirectly self-penalizing themselves, if you will, since they don’t get ‘credit’ for taking care of sicker patients,” Numerof explains. “The way in which reimbursement is structured today, if you code properly, you do take into account sicker patients. When some of our clients got data back this year, it led them down a path in which they got lower scores than they would have liked, so they had to come to terms with doing better coding. That’s not new,” she says. Numerof feels that the greatest mistake from CMS regarding the ratings is that combining patient experience with clinical outcomes for an overall star rating “is a bit problematic.”
Responding to complaints from hospitals that academic medical centers take care of the sickest of the sick patients, many with significant socioeconomic challenges who transfer from other facilities, thus “skewing” the ratings even more, Numerof again said that this is not a new argument, though she agrees that CMS is essentially comparing apples and oranges. “The argument for academic medical centers historically has been that their patients are sicker, and the argument gets played out at the level of individual doctors who also say that ‘my patients are different from yours.’ So they don’t want to compare themselves to their peers,” she says. “At end of day, part of this is a reflection of the angst on the part of industry that forces them to address issues of accountability and outcome where they don’t feel they have control necessarily. This is brand new territory for these organizations.”
Nonetheless, the AHA and other groups that represent U.S. hospitals still point to the various flaws they see in CMS’ ratings. And as a result, they fear that consumers who rely on them will make uninformed decisions on where to get their care. To that point, Numerof says that these ratings are a first step in seeing that the consumer is informed, has questions, and feels entitled to ask the questions and demand good answers. That’s a good thing all around,” she says. Numerof also notes that “prominent” doesn’t always equal “best,” based on what was marketed through the press. “Historically, maybe you made the assumption that prominence led to better outcomes, but we know that’s not true,” she attests.
Based on the data and reaction that we have heard in recent months, it almost seems impossible to fairly assess the quality of care at hospitals with such different patient populations. But I agree with Numerof in that CMS has made a necessary first step in its goal to greater accountability and transparency in healthcare. Now, it’s time for CMS to back up its words that it will continuously “analyze the star rating data and consider public feedback to make enhancements to the scoring methodology as needed.” And, as AHA said, they need to work hand-in-hand with the government in making these measures as accurate and fair as possible.
For consumers who have the right to shop for the providers that will make sense for meeting their specific medical needs, that day needs to come sooner than later.
