When The EMR Goes Down, Doctors Freak Out

Earlier this month, health IT superstar John Halamka, MD, MS posted a story talking about how network downtime within a hospital has changed over the past 10 years or so. I thought I’d share some of it with you, because he makes some interesting points about end user perceptions and sensitivities.

First, he tells the tale of a 2002 network core failure of Beth Israel Deaconess Medical Center, where he serves chief information officer. For two days, he reports, the hospital’s users lost access to all applications, including e-mail, lab results, PACS images and order entry, along with all storage. Or as he puts it, “For two days, the hospital of 2002 became the hospital of 1972.”

He then contrasts that failure with a recent one  (July 25 of this year) in which a storage virtualization appliance at BIDMC failed.  Because the hospital was loathe to risk losing data, he and his team chose a slower path to uptime — reindexing the data — which allowed them to avoid data loss. The bottom line was an outage of a few hours.

This outage was a different ballgame entirely, Halamka says. For example:

* In 2002, staff and doctors weren’t incredibly upset, but this time physicians were angry and frantic, with some noting that they couldn’t take care of patients without EMR access.  Here in 2013, end users expect network access to be like electricity, always there short of an act of God. Worse, though downtime simply isn’t acceptable, but procedures for dealing with it aren’t up to that standard yet, he says.

* Doctors are under an incredible set of regulatory burdens, including but not limited to Meaningful U se, health reform, ICD-10 and the Physician Quality Reporting System. They fear they can’t keep up unless IT functions work perfectly, he observes.

* Technology failures of 2013 are tricky and harder to anticipate. As he notes, back in 2002 servers were physical and storage was physical, but today networks are multi-layered and virtualized. While these things may add capability, they also crank up the complexity of diagnosing system failures, Halamka notes.

Halamka says he learned a lesson from the recent failure:

Expectations are higher, tolerance is lower, and clinician stress is overwhelming. No data was lost, no patient harm occurred, and the entire event lasted a few hours, not a few days. However, it will take months of perfection to regain the trust of my stakeholders.

This story does have one ray of sunshine in it — it demonstrates that increasing numbers of doctors depend completely on their EMR, a state devoutly to be wished for by many health IT leaders. But the price of having doctors throw themselves into EMR use is having them riot when they can’t get to the system. Clearly, hospitals are going to have to find some new way of coping with downtime.