Lions and Tigers and Meaningful Use Betas, Oh My!

Every vendor involved in ARRA certification, inpatient and eligible ambulatory care provider, has rolled its newly certified code to a beta client or two, on the path to demonstrate Stage 1 Meaningful Use. This will be the first relevant experience for implementation teams to bring Stage 1 functionality live across the install bases.

What is a beta test? Definition

Second level, external pilot-test of a product (usually a software) before commercial quantity production. At the beta test stage, the product has already passed through the first-level, internal pilot-test (alpha test) and glaring defects have been removed. But (since the product may still have some minor problems that require user participation) it is released to selected customers for testing under normal, everyday conditions of use to spot the remaining flaws.

http://www.businessdictionary.com/definition/beta-test.html

Because of the incentive revenue implications for the provider organizations, and the threat of having their attestation contested by the government, and because of the complexity of dozens of measures (Core, Menu, and requisite capability), we have never before seen betas of this complexity in HCIT. And just as we begin to tie a bow around our Stage 1 accomplishments, an uncertain Stage 2 looms on the electronic horizon.

Stage 2 Meaningful Use is shaping up to be more complicated with a broad spectrum of escalating requirements. Already many on both sides of the vendor-provider partnership are pushing back against what they’ve determined is an overly challenging set of requirements with an equally unrealistic deadline. Stage 2 certification will be a difficult and expensive process, but with commensurate improvements for more efficient, safer and better care delivery.

Pilot testing (see definitions of alpha and beta stages above) for both Stage 1 and 2 requirements are, of course, extremely important. There have been many past pilots from many very competent IT companies that ended in disaster even though the controlled testing results were positive. In this blog, I hope to help those of you developing, managing, participating in, and interpreting pilots to avoid having to deal with a disaster by implementing a crisis management plan focused on not rolling out a pilot program designed solely to succeed.

A short time ago, I ran across a section of an article on pilot roll outs. The article was titled, “ Strategies for Learning from Failure ,” by Amy C. Edmondson. Read the whole article. It’s a treat, and is typical of her other research. There is a companion video interview with the author here .

The article closes with an interesting twist on pilot testing that’s worth your attention:

Designing Successful Failures

Perhaps unsurprisingly, pilot projects are usually designed to succeed rather than to produce intelligent failures—those that generate valuable information. To know if you’ve designed a genuinely useful pilot, consider whether your managers can answer yes to the following questions:

• Is the pilot being tested under typical circumstances (rather than optimal conditions)?

• Do the employees, customers, and resources represent the firm’s real operating environment?

• Is the goal of the pilot to learn as much as possible (rather than to demonstrate the value of the proposed offering)?

• Is the goal of learning well understood by all employees and managers?

• Is it clear that compensation and performance reviews are not based on a successful outcome for the pilot?

• Were explicit changes made as a result of the pilot test?

We, the vendors and healthcare providers, might want to rethink our mantra, “Failure is Not an Option,” under certain circumstances. It does appear that at the pilot testing level we should be willing to accept some degree of failure in order to acquire the knowledge to succeed when it really counts, i.e. MU Stage 1 certification and beta roll out to real world users. By knowing what we should know, we might even be able to better control system development costs, as well as the time necessary to make our systems compliant with certification criteria. And, of course, manage and achieve this consistent with and managed to the reporting year chosen.

What do you think? Is the concept of designing successful failures on the path to Meaningful Use relevant to HCIT?

Joe Bormel, M.D., MPH
CMO & VP, QuadraMed
This Post: http://bit.ly/LionsTigersBetas
All posts here

The will to succeed is important,

but what's more important is the will to prepare.
Bobby Knight