For a health system, establishing secure links between hospitals can be a key component of any disaster recovery strategy, says Aaron Marsden, network and database administrator at Singing River Health System.
A not-for-profit community health system, Singing River is composed of the 435-bed Singing River Hospital in Pascagoula, Miss., and the 136-bed Ocean Springs Hospital in nearby Ocean Springs, Miss., as well as medical parks at each location.
Serving patients in southern Mississippi and Alabama, Singing River was lucky in that it did not sustain a direct hit from Hurricane Katrina in 2005. While there was a power outage in the area, the lines of communication between the two hospitals remained intact and there was no loss of data, Marsden says.
That was not the case, however, in 2003, when a “parity bleed” caused a hard drive to fail. Since this drive was an integral part of the health system’s electronic health record (EHR), and since there was no real backup system in place, all the images that had been scanned into the system were lost, Marsden reports.
By the time Katrina struck the Gulf Coast, Singing River had already begun to implement a disaster recovery plan. “We focused a lot more post-Katrina,” he notes. “Post-Katrina, we wrote it in stone.”
Working with the Hopkinton, Mass.-based EMC Corp. and the San Jose, Calif.-based Brocade, Singing River built a state-of-the-art data center at Ocean Springs Hospital to complement the one at Singing River Hospital.
Having already used Brocade’s networking technology for its storage area network (SAN), Singing River executives called the company back to set up an “active-active data center,” according to a Brocade spokesman. By using Brocade’s ethernet fabric with its high-speed switches and routers, Singing River leaders were able to install the necessary network infrastructure and link the data centers of both hospitals using two, 10-gigabyte-per-second circuits.
Marsden says the importance of this connection is based on the reliance of electronic data and the need to access that data whenever necessary. “Things aren’t printed anymore,” he emphasizes. “Everything is electronic. And since we’re on an EHR, physicians need that data.”
But, he adds, “It’s the presentation of data, or access, that is of critical importance. We’ve got to have it and present it.”
Not having access to data for even a day is too long now, he notes.
For Marsden, safeguarding clinical data is most important. But there also has to be in place a business continuity plan that will safeguard all financial data as well.
Since the lion’s share of this disaster recovery plan was rolled out post-Katrina, Marsden says that everybody on staff was onboard. But there were some challenges. “There was an overall lack of knowledge, so we had a lot to learn. Then there were the hard-headed vendors,” he recalls. “They want to hide behind their systems, saying this is the only way to do it.”
Persistence ultimately paid off for Singing River’s IT leaders, who were able to put into place what Marsden admits is a “home-grown solution.”
Because all systems are run in-house, he says the decision was made to install vSphere, from Palo Alto, Calif.-based VMware, to create a virtualization platform. “We run approximately 850 servers and about 80 percent are virtual,” he notes.
And while some data are now being backed up to disc, Marsden says he still prefers tape for clinical data—given the size of most diagnostic images.
It’s much slower, he adds, but it’s also a less expensive back-up solution.
As for offering advice to other CIOs who are planning to implement a disaster recovery plan, Marsden says, “Presentation of data is key. It’s great to have a disaster recovery plan, but if you can’t access the data, what good is it?”
