As certain clinical IT product categories come to represent increasingly mature markets, user evaluations of products within those categories are becoming less positive over time, signaling more critical mindsets on the part of sophisticated users. That trend appears to be one of the more important ones coming out of the 2008 Top 20 Best in KLAS Awards: Software & Professional Services report from Orem, Utah-based KLAS. The dipping of most products in the more mature product categories, along with continuing wide variation across products and categories, speaks to ongoing fluctuations in perception of clinical IT products currently available in the HIT market, 10 years after KLAS' first year-end summary report was released.
Wide variation seen overall
One has only to look at Figure 1 to get a first glimpse of the broad range of perceptions toward clinical software products on the market. At the high end of product categories were enterprise scheduling products, which scored an average of 84.79 on a 100-point scale, whereas the acute-care EMR sector scored the lowest overall, at 72.26. In between those two fenceposts, such clinical product categories as PACS (78.82), Laboratory (77.68), Radiology (75.73), Cardiology (73.87), and Pharmacy (72.64) filled in spaces on a wide spectrum.
What is the significance of such wide variations among the categories ranked in its survey? KLAS leaders ascribe the differences partly to the maturity of solutions on the market within each category. Products in categories that tend to be going into their second and even third generations, such as acute-care EMR, tend to be rated lower overall, likely because familiarity over time has led to a more skeptical view of capabilities, says Jason Hess, KLAS general manager, clinical research.
Similarly, says Ben Brown, general manager of imaging informatics at KLAS, “At a high level, from a PACS perspective, when you look at the maturity of their implementation in hospitals, most facilities have had a PACS for five or seven years, even nine years in the case of the early adopters.” As a result, Brown says, there is a “maturation factor” influencing perceptions of PACS products at this point in the evolution of that market segment. And those perceptions are in turn putting more pressure on PACS vendors “to become more competitive both in terms of product quality and pricing.”
In fact, Brown says, “If you look at two or three years ago, PACS was one of the highest-rated categories, in the low 80s; now it's in the high-70s.” Expectations for faster turnaround times and similar requirements, and regulatory demands from the federal government have “set expectations higher than ever” for the PACS category, he emphasizes.
There's an additional reason that EMR products are the lowest-ranked in the current survey, says Erica Drazen, a partner in the Lexington, Mass.-based Emerging Practices Group at the Falls Church, Va.-based CSC. “Fundamentally, it's because EMR is really hard to do,” says Drazen, who points out that most of the people KLAS surveyors speak to are hospital CIOs. “It's their biggest headache. And it's also a starting point. So if your EMR isn't feeding the data you need for your revenue cycle system, you blame the EMR, right? And also,” she adds, “the simpler systems in smaller environments get much higher ratings than the more complicated systems in larger environments.”
The shifting sands of PACS
If any area has seen considerable change in the past few years, it's the imaging management area, particularly PACS. Various acquisitions, sell-offs and other business moves, combined with a recession-induced series of changes at a number of vendor companies, have made this a relatively unstable category.
As can be seen in Figure 2, the PACS product category spans a wide range of ratings, from a high of 89.98 to a low of 64.38 among those vendors with sufficient volume of customer evaluations to be ranked. At the top of the list is the Dynamic Imaging product, now owned and marketed by GE Healthcare, with Siemens syngo imaging at the bottom of the list, and a variety of vendors' products in between.
The meaning of these rankings is clear if one looks at the developments and trends that have occurred within the last couple of years, says Scott Grier, president of the Sarasota, Fla.-based Scott Grier & Associates, a consulting firm specializing in PACS and imaging management solutions.
Addressing some of the specific survey results, Grier says that Dynamic Imaging, which GE purchased two years ago, “was a good product that hadn't previously been marketed well. So the branding of the product as a GE product has been phenomenal.”
Grier also says that he isn't surprised that the Fuji and Philips PACS products are highly ranked as well, since they are “the two top Web-based solutions.” In contrast, he says, Siemens' syngo solution has had a very rough start in terms of reliability.
The bottom line with PACS, Grier says, is that “If you need a PACS system, you must have a much broader, more encompassing plan than just buying a PACS system from a company.” Focusing on evolving towards enterprise-wide imaging management and EMR interoperability will be critical for customer organizations going forward, he insists. And the instability in the vendor market makes long-term, strategically focused planning all the more important.
When it comes to cardiology and pharmacy systems, CSC's Drazen looks at the relatively closer point spread in terms of cardiology product evaluations (see Figure 3). “They're simpler products operating within one department, and most vendors have a long history with their customers,” she says. “It's a very fluid market with lots of participants, and to some extent, pretty similar products.”
Pharmacy, like cardiology, has a relatively small number of vendors and some fluidity, though the point spread is wider with regard to customer evaluations (Figure 4). Still, Drazen says, “We encourage our clients to look at vendors that have both clinical and non-clinical products. Everything should be coming out of the clinical system, but the documentation and reporting will also support non-clinical systems. So you really should have both, as things like pay-for-performance go forward.”
Drazen notes that hospitals are struggling to help their practices adopt information systems. “If you look across the board at which vendors have scored highest among small, medium-sized, and large practices, no one scores great across all of them,” she says.
Skepticism over EMRs?
What is interesting, all those interviewed agree, is the relative stability (along with the wide range of point scores) in the all-important acute-care EMR category (Figure 5). Epic Systems' EpicCare Inpatient has maintained its number-one position for a few years, and is significantly more highly rated (85.24 score) than the second highest scoring acute-care EMR product, Eclipsys' Sunrise Clinical Manager (74.56).
So why does one vendor continue to score consistently higher than any of its rivals? Speaking of Epic, Drazen says, “There is a very different relationship with the customer.” She cites the unique policy of Epic executives to limit the number of customers they contract with, and set parameters around the vendor-customer interaction. Some in the industry might be taken aback by such practices. “But,” she says, “there is an aura of authenticity around them; there's just a really strong trust factor.” Drazen says that vendors such as Epic - and she cites athenahealth as another example - have unusually close working relationships with their customers, which sometimes helps raise their products' KLAS scores.
Meanwhile, overall in the acute-care EMR category, KLAS' Hess notes that “every single vendor has dropped” over the last year. There are several elements to look for, he says, one of which is complexity. “This is the core clinical infrastructure that ties all the ancillary products together. What's happened historically is that there has been a rush to get clinical information systems up and running. And CPOE is another frontier, where we need to get physicians engaged to order electronically. So what's happened is that for years, you've had all these organizations rush out to implement. It's like when my family bought our first VCR, and at first all you could do was play videos. Then you learned you could record television programs, and your expectations increased from there.” Now, he says, the expectations are far higher for EMRs.
And what of the near future with regard to customers' evaluations of clinical IT products? “One constant that I think we're seeing - and this is true for pharmacy and certainly for ED and some of these ancillaries if you talk to people using these products - is that functionality and usability are pretty important,” says Hess. “But if you talk to the CIOs, they'll be focused on interoperability. Is the pendulum swinging towards enterprise-wide and single-source vendors? Absolutely. But if you talk to the end-users, they'll have a different perspective.”
Healthcare Informatics 2009 March;26(3):3-7
