On Shaky Ground

The threat of earthquakes is ever-present along the West Coast of North America, from Alaska to British Columbia and south to Washington, Oregon and California. Not surprisingly, the California state legislature passed a statewide law in 1994 following the Northridge earthquake earlier that year that mandated seismic retrofit requirements of all hospitals, in order to prepare for future temblors. Indeed, a 2001 study found that 37 percent of the state's hospital buildings still posed a significant risk of collapse in an earthquake.

For those hospitals that will be core providers in the next natural disaster, much is hanging on their disaster recovery preparation. So it's good news to hear that the two-campus, 888-bed UCLA Health System in Los Angeles, one of the region's premier patient care organizations, has created a comprehensive and intensive DR and business continuity plan that encompasses a broad number of elements, and whose anchor is the construction of a brand-new patient care facility on its Westside Los Angeles campus, set to open its doors sometime next spring, as well as the construction of a new facility for its nearby Santa Monica campus.

"We're building a very IT-capable new hospital facility, the Ronald Reagan UCLA Medical Center, as well as the 271-bed Santa Monica replacement hospital (replacing the current 200-bed Santa Monica facility), which will be the Santa Monica UCLA Rehabilitation Hospital," notes James Atkinson, M.D., senior medical director for transition and a professor of surgery at UCLA.

"Part of our redundancy will be to be able to communicate between those two facilities should there be any interruption in service. We've also made some strategic decisions, including the decision to go with voice-over IP as a core day-to-day communications technology." Atkinson says

Significantly, the new 520-bed Ronald Reagan facility has been built to withstand a magnitude 8.3 earthquake (a very strong temblor), and all DR plans anticipate that UCLA will become a magnet provider for the metropolitan area in the event of an earthquake or other natural disaster (which, in that region, also includes wildfires and mudslides).

"We have an active plan, there's been a working group for the last several months (focused) on addressing each and every segment of the medical record to assure there's a backup plan, a paper recovery plan, starting with what happens if the system is out for five minutes, all the way up to if it's out indefinitely. In that case, we would move to an entirely paper-based process, and it's unit by unit and segment by segment of the population," reports Thomas Rosenthal, M.D., UCLA Health System's chief medical officer.

The fact that the UCLA Health System is spending over $1 billion on its DR initiatives, including all the construction, speaks to the organization's seriousness, says CIO Rodney Dykehouse, who joined UCLA a year ago, after most recently having been CIO at Froedtert Community Health in Milwaukee, Wis.

"I think you step back and look at the spectrum of hospital disasters from fire in a portion of a facility all the way to the extreme of a regional earthquake, it's very difficult to plan for all situations," says Dykehouse. "So it's important for the executives to work together to envision all the scenarios. In the case of an earthquake, we can expect all of the lesser disasters to occur as part of that disaster. Disaster planning is a moving target," he emphasizes.

With regard to core clinical information systems in a post-automated world, two key lessons become clear, he says. The first requirement is careful process analysis. "Al Bookbinder is leading a business impact analysis process now to look at what the most critical applications are," he says, referring to the organization's assistant director of data center services.

Similarly, Mark Rose, UCLA's manager of network services, emphasizes that planning has to get down to a very detailed technical level, including such preparations as "multiple commun ication closets on every floor, so you don't have everything in one basket; and multiple sources of power, both redundant and backup."

The second key to success Dykehouse says, has come out of UCLA's experience is the very extensive disaster drilling the organization has done every year, including one with federal authorities last year. A great deal has been learned from those exercises, he says, that can be gained no other way except through experience. In fact, he notes, numerous issues came up that no one had anticipated and that were corrected as a result.

Most of all, he says, CIOs must be involved with other senior executives to analyze possible disaster scenarios and to determine what their organization's priorities are in the event of a catastrophe. "As a CIO, we should challenge the organization's executive leadership to overtly decide where to invest in redundancy and disaster planning from an IT standpoint," he says. "We can never achieve 100 percent survival of all systems and so forth, but some organizations can move further down that spectrum. And it is through business continuity planning and impact analysis that we can determine where to invest the dollars and the planning."

Building DR into every application

In the Rocky Mountains, several hundred miles to the east of Los Angeles, Denver-based Centura Health system has taken a somewhat different approach to disaster recovery planning, confirms Frank Biondolillo, the 12-hospital system's vice president and CTO.

As a result, Biondolillo says, Centura managers have built DR capability into their new clinical applications. "Everything has to be highly redundant," he emphasizes, "with several layers of fail-over, from the server itself, to hot swappable parts, multiple power suppliers in every piece of equipment, and so on."

In addition, the system has taken one data center facility from one of the 12 hospitals and turned it into the entire system's mirrored data site in Colorado Springs.

Comprehensive technological disaster recovery planning is expensive, Biondolillo concedes; but the alternative — the long-term disabling of a core clinical or operational information system for an entire health system — is one most IT executives will not want to imagine.

He offers the following advice: "This is a business decision," he says. "And the way we presented it to our CEO was in terms of options," between a comprehensive IT replication and a cheaper step-down version. The CEO and other C-suite executives did not hesitate, he says. "The decision seemed clear to them; they took the first option, because they didn't want to introduce the risks involved in preparing less well. That's not an IT decision, it's a business decision."

Sidebar

Underlying UCLA's DR plan are the followingassumptions:

• UCLA will become a magnet healthcare provider location;

• the Ronald Reagan replacement facility's earthquake-proof construction means that data center redundancy (two data centers) has been built inside that facility;

• protecting the organization's EMR, its network, and its infrastructure, is a core priority.