Not everyone’s study gets attention from Farzad Mostashari, M.D., the national coordinator for health IT. However, that’s exactly the situation Danny McCormick, M.D., an assistant professor of medicine at Harvard Medical School, and his team of researchers from the Cambridge Health Alliance and the CUNY School of Public Health found themselves in recently. The study appeared in the most recent edition of the journal Health Affairs last week and provoked a blog post response from Mostashari shortly after its release.
The study, with samplings from the National Ambulatory Medical Care Survey (NAMCS), examined 28,741 patient visits to 1,187 office-based physicians and concluded electronic access to computerized imaging results did not result in a decrease in the amount of imaging tests ordered nor the costs associated with those tests. In fact, the study found that physicians having access to those computerized imaging results may actually have lead to an increase in tests ordered and associated costs.
In his blog post, Mostashari said the authors of the study used data that was passé, from 2008, before the passage of the American Recovery and Reinvestment Act/ Health Information Technology for Economic and Clinical Health (ARRA/HITECH). He also said the authors did not “consider clinical decision support, which helps give providers the data tools they need to make appropriate care recommendations and the ability to exchange information electronically.”
In addition, Mostashari said the study was not designed to answer the question of cost, or associations between EHRs and quality. He added it did not consider variables like sicker patients and included a note about how a reduction in test orders is not the way to reduce costs regardless.
McCormick and his co-authors offered a rebuttal to Mostashari’s response, which they posted in a blog of their own. McCormick said the 2008 data is legitimate because the EHR vendor market is largely unchanged and the systems in place now are widely the ones used back then. He also said a lot of Mostashari’s suggestions to explain the study’s results, such as doctors who are inclined to order more tests are also inclined to purchase health IT for viewing test results electronically, are merely hypotheses, without proof.
Additionally, McCormick rejects Mostashari’s claim that the study failed to assess whether health IT improved the quality or appropriateness of care, noting that that wasn’t the purpose of the study. McCormick also noted Mostashari selectively touted a study that supported his claim, which he said was authored by members of the ONC, “without regard to study quality or statistical niceties.” There were several other refutations of Mostashari in the thoroughly-written blog post.
Few recent studies in healthcare IT that have driven this much controversy. HCI Assistant Editor Gabriel Perna recently got a chance to talk with McCormick about his findings. Here are excerpts from their conversation.
Explain the study’s background and what you were trying to accomplish.
I’m a primary care physician (PCP) myself, and back in 2005, I was on the Senate Health, Labor, Education & Pensions Committee under Senator [Edward] Kennedy and worked a fair amount of my time working on one of the precursor bills to the HITECH Act. Through that experience and our [McCormick and his team] experience as practicing clinicians, we had skepticism about the legitimacy of the long-made claim on the ability of health information technology to decrease test ordering and therefore costs throughout the U.S. Part of our question had to do with the fact most people put health IT forward as something that will bail out the spiraling cost spending crisis. We wondered if this was true.
We’ve used this federal database (NAMCS) for other research, not related to health IT. Several years ago, it started collecting data on physicians use of health IT and so it was a natural idea to examine a large number of patient encounters across the U.S. and to ask the question, ‘Was the ability to see imaging results or the image itself associated with decreased test ordering?’ We thought most people would expect that.
What did you find?
Rather than finding there was a decrease or no change, we found in physicians’ offices that had the ability to see those electronic images or get the image reports, that image and blood test ordering was higher. If you included all imaging studies, having that electronic capacity to see the image results was associated with a 40 percent increase in test ordering. If you look specifically at the advanced imaging, i.e. CT and MRI scans, it was about a 70 percent increase.
One thing to note is this was not an experimental study where you take two populations and do one thing to one population and another to the other, and see the results. This was a cross-sectional study, where you’re collecting data and not treating one differently. Thus, you can’t say having the electronic capability caused the increase definitively. We avoided making any erroneous associations, and 90 percent of our analysis was doing multi-variant modeling to take patient-level differences, doctor differences, and practice differences into consideration. For instance, maybe a patient needs more imaging tests. Ninety percent of our analysis was to avoid making an erroneous conclusion based on something like that. We controlled for illnesses, characteristics, factors related to the doctors’ specialty, how they were paid, etc.
We think that the results are not due to any systematic difference in the need for the studies, but what we’re actually seeing is maybe the health IT was driving the test ordering.
Why do you think this was the case?
The claim has been made since the early 1960s that health IT will lead to decreased test ordering. The main claims have been, if I’m sitting in front of a patient and he had an MRI in another town yesterday and I could see that on my computer screen, then I wouldn’t have to order a second test. It’s this idea of interoperability. What we think we may be seeing partly is what many people would tell you. We don’t have interoperability yet. It’s in pockets here and there within healthcare systems, but not across the spectrum.
The other thing is, in certain cutting-edge institutions, people have come up with health IT systems that have decision aids, which can decrease needless test ordering. There is some evidence that has been achieved in cutting-edge institutions. But that’s not what we were studying. We were studying what’s actually happening out there, if you take a snapshot. It may be that there’s a big difference between what’s possible and what’s happening.
In terms of unintended consequences, why would test ordering numbers go up? We’re just speculating, but we think if you make something easier, just like in any human endeavor, people will do it more often. In borderline cases where a physician is wondering if they should order a test or not, if they have a computer in front of them, it may shift them in favor of doing it because it’s just much easier. The study didn’t look at that; but that may be what’s going on.
What, if anything, surprised you about the results?
I think what we expected, our hypothesis based our own experience, was there’d be no difference. We didn’t think it would decrease test ordering. But we certainly didn’t think it would increase it. That was a surprise. I think we really haven’t seen much in the literature where people talk about this phenomenon possibly existing where the ease-of-use may drive utilization. For us, this is an area that needs to be thought about as people implement systems in outpatient practices, hospitals, etc.
How come you excluded outpatient departments, radiologists, anesthesiologists and pathologists?
They generally don’t order tests. There’s not typically much of a reason for them to be ordering tests for patients and it would be for really unusual reasons if they did. It’s a very limited number. We thought it would be more general if we only included the types of doctors that would be ordering the tests and that do order the tests.
What did you make of the fact that specialists ordered more tests than PCPs?
That’s not too surprising to us because in general specialists are sent more complex patients and sicker patients than PCPs. Patients go through the filter of PCPs and it’s the ones who have unusual questions or who are sicker that get sent on to specialists, so it wouldn’t be surprising that they order more tests. It’s explainable by that.
What follow up research would you like to see happen?
I think certainly the whole question of what’s going on [should be studied]. If it’s true in common practice that health IT drives up test ordering, why is that? And are there things that could be done to engineer it differently? Is there actually a phenomenon that making the retrieval of test results easier is driving up costs? If you found it that it does, are there ways around that? These are avenues we hope to explore in the future.
