Free EMR Newsletter Want to receive the latest news on EMR, Meaningful Use, ARRA and Healthcare IT sent straight to your email? Join thousands of healthcare pros who subscribe to EMR and HIPAA for FREE!!

Nuance Takes Page from Healthcare Clients in Petya Outage Aftermath

Posted on November 6, 2017 I Written By

Colin Hung is the co-founder of the #hcldr (healthcare leadership) tweetchat one of the most popular and active healthcare social media communities on Twitter. Colin speaks, tweets and blogs regularly about healthcare, technology, marketing and leadership. He is currently an independent marketing consultant working with leading healthIT companies. Colin is a member of #TheWalkingGallery. His Twitter handle is: @Colin_Hung.

On June 27th the Petya Malware (or NotPetya or ExPteya) struck Nuance Communications (NASDAQ: NUAN). For days the company’s eScription speech-recognition platform were unavailable, forcing thousands of healthcare clients to find alternatives for their medical transcription. During the crisis and in the weeks that followed, Nuance borrowed a page from their healthcare clients: not offering false hope and deconstructing the incident to learn from it.

At the recent CHIME Fall Forum in San Antonio Texas, I had the opportunity to sit down with Brenda Hodge, Chief Marketing Officer – Healthcare and Ed Rucinski, Senior Vice President of World Wide Healthcare Sales of Nuance to talk about the Petya outage and where the company is headed.

“The challenge we faced with Petya brought us all together as a company,” explained Ed. “When our systems went offline, the entire organization rallied together. We had engineers and support staff who slept at the office on couches and cots. We had developers who went with less than 2hrs of sleep for 4 days straight because they wanted to help clients and bring our systems back online as quickly as possible. We became a nameless and rank-less organization working towards a common goal.”

As the outage went from minutes to hours to days, Nuance resisted the temptation to offer false hope to its clients. Instead, the company opted to be truthful and transparent. Nuance sent emails and directly called clients to let them know they had suffered a cyber attack, that the full extent of the damage was not known and that they did not know when their systems would be back online. The company did, however, commit to providing regular updates and being available to answer questions and address concerns.

The following is an abbreviated excerpt from a Nuance communication posted online by one of its clients:

Nuance corporate systems were unfortunately affected by a global cyber attack today. We went into immediate security protocol by shutting down our hosted production systems and platforms. There is no update at this time as to when the accounts will be back online but we will be holding regular calls throughout the day and night to gain insight into the timeline for resolution and I will update you again when I have more info. We are sorry for the inconvenience this outage has caused and we are working diligently to get things back online.

Clinicians are coached never to give patients in crisis or their families false hope. They calmly explain what happened, state the facts and talk about potential next steps. They do not, however, say that “things will be alright”, even though they know that is what everyone desperately wants to hear. Nuance used this same protocol during the Petya outage.

The company also used protocols similar to those used following an adverse event.

Healthcare is complex and despite the best efforts and best intentions of care teams, errors occur. These errors are referred to as adverse events. Adverse events that impact patient safety or that cause actual harm to patients are thoroughly documented, deconstructed and analyzed by clinical leaders as well as risk managers. The lessons gleaned from these unfortunate events are captured and used to improve operations. The goal is to prevent or mitigate the impact of similar events in the future.

After their systems were fully restored, the Nuance team embarked on a thorough review of the incident – from technical procedures to client communication protocols.

“We learned a lot through this incident” says Hodge. “We got a first-hand education on how sophisticated malware has become. We’ve gone from viruses to malware to ransomware to coordinated nation-state attacks. That’s what Petya really is – a coordinated attack on company infrastructure. Now that we have been through this type of attack, we have put in new processes and technologies to prevent similar attacks in the future. Most importantly we have made investments in improving our response to these types of attacks.”

Nuance has gone one step further. They have committed to sharing their painful lessons learned with other companies and healthcare institutions. “Like it or not, we are all in this together”, continued Hodge. “The Petya attack came on the heels of the WannaCry ransomware attack that impacted many of our healthcare clients – so there was a lot of empathy from our clients. In fact this whole incident has created a sense of solidarity in the healthcare technology community. Cyber attacks are not going to stop and we need to come together as an industry so that we are as prepared as we can be for the next one.”

“It’s unfortunate that it took an incident like this to show us what we are made of,” says Rucinski. “We had executives making coffee and fetching lunch for the support teams. We had leaders offering to run errands for staff because they knew they were too tired to keep up with those types of things. In the end we found out we truly embody the values and principles that we have hanging on posters around the office.”

One Hospital Faces Rebuild After Brutal Cyberattack

Posted on July 20, 2017 I Written By

Anne Zieger is a healthcare journalist who has written about the industry for 30 years. Her work has appeared in all of the leading healthcare industry publications, and she’s served as editor in chief of several healthcare B2B sites.

Countless businesses were hit hard by the recent Petya ransomware attack, but few as hard as Princeton, West Virginia-based Princeton Community Hospital. After struggling with the aftermath of the Petya attack, the hospital had to rebuild its entire network and reinstall its core systems.

The Petya assault, which hit in late June, pounded large firms across the globe, including Nuance, Merck, advertiser WPP, Danish shipping and transport firm Maersk and legal firm DLA Piper.  The list of Petya victims also includes PCH, a 267-bed facility based in the southern part of the state.

After the attack, IT staffers first concluded that the hospital had emerged from the attack relatively unscathed. Hospital leaders noted that they are continuing to provide all inpatient care and services, as well as all other patient care services such as surgeries, therapeutics, diagnostics, lab and radiology, but was experiencing some delays in processing radiology information for non-emergent patients. Also, for a while the hospital diverted all non-emergency ambulance visits away from its emergency department.

However, within a few days executives found that its IT troubles weren’t over. “Our data appears secure, intact, and not hacked into; yet we are unable to access the data from the old devices in the network,” said the hospital in a post on Facebook.

To recover from the Petya attack, PCH decided that it had to install 53 new computers throughout the hospital offering clean access to its Meditech EMR system, as well as installing new hard drives on all devices throughout the system and building out an entirely new network.

When you consider how much time its IT staff must’ve logged bringing basic systems online, rebuilding computers and network infrastructure, it seems clear that the hospital took a major financial blow when Petya hit.

Not only that, I have little doubt that PCH faces doubts in the community about its security.  Few patients understand much, if anything, about cyberattacks, but they do want to feel that their hospital has things under control. Having to admit that your network has been compromised isn’t good for business, even if much bigger companies in and outside the healthcare business were brought to the knees by the same attack. It may not be fair, but that’s the way it is.

That being said, PCH seems to have done a good job keeping the community it serves aware what was going on after the Petya dust settled. It also made the almost certainly painful decision to rebuild key IT assets relatively quickly, which might not have been feasible for a bigger organization.

All told, it seems that PCH survived Petya successfully as any other business might have, and better than some. Let’s hope the pace of global cyberattacks doesn’t speed up further. While PCH might have rebounded successfully after Petya, there’s only so much any hospital can take.

The Petya Global Malware Incident Hitting Nuance, Merck, and Many Others

Posted on July 3, 2017 I Written By

John Lynn is the Founder of the HealthcareScene.com blog network which currently consists of 10 blogs containing over 8000 articles with John having written over 4000 of the articles himself. These EMR and Healthcare IT related articles have been viewed over 16 million times. John also manages Healthcare IT Central and Healthcare IT Today, the leading career Health IT job board and blog. John is co-founder of InfluentialNetworks.com and Physia.com. John is highly involved in social media, and in addition to his blogs can also be found on Twitter: @techguy and @ehrandhit and LinkedIn.

The Petya Malware (or NotPetya or ExPetya) has really hit healthcare in a big way. The biggest impact on the healthcare IT world was the damage it caused to Nuance, but it also hit Merck and some other healthcare systems. After a shaky start to their communication strategy, Nuance seems to finally at least be updating their customers who saw a lot of downtime from when it first started on June 28 until now. This rogue Nuance employee account has been pretty interesting to watch as well. There’s a lesson there about corporate social media policies during a crisis.

Petya was originally classified as ransomware, but experts are now suggesting that it’s not ransomware since it has no way to recover from the damage it’s doing. It’s amazing to think how pernicious a piece of malware is that just destroys whatever it can access. That’s pretty scary as a CIO and it’s no surprise that Petya, WannaCry, and other malware/ransomware is making CIOs “cry.”

It’s been eye opening to see how many healthcare organizations have depended on Nuance’s services and quite frankly the vast number of services they offer healthcare. It’s been extremely damaging for many healthcare organizations and has them rethinking their cloud strategy and even leaving Nuance for competitors like MModal. I’m surprised MModal’s social team hasn’t at least tweeted something about their services still being available online and not affected by Petya.

I’ll be interested to see how this impacts Nuance’s business. Nuance is giving away free versions of their Dragon Medical voice recognition software to customers who can’t use Nuance’s transcription business. Long term I wonder if this will actually help Nuance convert more customers from transcription to voice recognition. In the past 5 days, Nuance’s stock price has droppped $1.54 per share. Considering the lack of effective alternatives and the near monopoly they have in many areas, I’ll be surprised if their business is severely damaged.

As I do with most ransomware and malware incidents, I try not to be too harsh on those experiencing these incidents. The reality is that it can and will happen to all of us. It’s just a question of when and how hard we’ll be hit. It’s the new reality of this hyper connected world. Adding to the intrigue of Petya is that it seems to have been targeted mostly at the Ukraine and companies like Nuance and Merck were just collateral damage. Yet, what damage it’s done.

Earlier today David Chou offered some suggestions on how to prevent ransomware attacks that are worth considering at every organization. The one that stands out most to me with these most recent attacks is proper backups. Here is my simple 3 keys to effective backups:

Layers – Given all the various forms of ransomware, malware, natural disasters, etc, it’s important that you incorporate layers of backups. A real time backup of your systems is great until it replicates the malware in real time to your backup server. Then you’re up a creek without a paddle. An off site backup is great until your off site location has an issue. You need to have layers of backup that take into account all of the ways your data could go bad, be compromised, etc.

Simple – This may seem like a contradiction to the first point, but it’s not. You can have layers of backups and still keep the approach simple and straightforward. Far too often I see organizations with complex backup schemes which are impossible to monitor and therefore stop working effectively. The KISS principle is a good one with backups. If you make it too complex then you’ll never realize that it’s actually failing on you. There’s nothing worse than a failed backup when you think it’s running fine.

Test – If you’ve never tested your backups by actually restoring them, then you’re playing russian roulette with your data. It’s well known that many backups complete without actually backing up the data properly. The only way to know if your backup really worked is to do a test restore of the data. Make sure you have regularly scheduled tests that actually restore your data to a backup server. Otherwise, don’t be surprised if and when your backup doesn’t restore properly when it’s really needed. Malware events are stressful enough. Knowing you have a good backup that can be restored can soften the blow.

Backups won’t solve all of your problems related to malware, but it’s one extremely important step in the process and a great place to start. Now I’m going to go and run some backups on my own systems and test the restore.