Free EMR Newsletter Want to receive the latest news on EMR, Meaningful Use, ARRA and Healthcare IT sent straight to your email? Join thousands of healthcare pros who subscribe to EMR and HIPAA for FREE!!

The NIST Workshop on EHR Usability

Posted on June 14, 2011 I Written By

John Lynn is the Founder of the HealthcareScene.com blog network which currently consists of 10 blogs containing over 8000 articles with John having written over 4000 of the articles himself. These EMR and Healthcare IT related articles have been viewed over 16 million times. John also manages Healthcare IT Central and Healthcare IT Today, the leading career Health IT job board and blog. John is co-founder of InfluentialNetworks.com and Physia.com. John is highly involved in social media, and in addition to his blogs can also be found on Twitter: @techguy and @ehrandhit and LinkedIn.

As much as I’d like to visit DC (I’ve never been), I wasn’t able to make it out there to attend the NIST workshop on EHR usability. However, Carl Bergman from EHR Selector did make it to the event and sent the following notes on EHR usability according to NIST.  Most of the speakers name link to their slides in PDF format.

National Institute of Standard and Technology’s Workshop on EHR Usability

This week I went to a NIST workshop examining the state of EHR usability. The workshop was at its administrative headquarters, a large 60s building on its sprawling Gaithersburg, MD campus about 20 miles outside Washington.

You might wonder what NIST is doing in the EHR business? I certainly did. NIST’s mission is to promote commerce and technical innovation including methods to determine, independently, the safety and security of a broad range of technologies including software. (It’s part of the Department of Commerce.) Since WW II, this has involved looking at the human factors involved in operation of every thing from nuclear plants to robotics. Interestingly, it’s not a regulatory agency, such as, the FDA or FCC. NIST’s standards work is through consensus building among manufacturers, consumers, regulators, etc.

The workshop, attended by about 200 persons, had two parts:

•      A review of the state of EHR usability studies by academics, practioners and system administrators and,

•      Introduction of NIST’s draft for a usability standard.

Part I. EHR Usability Today. There were many speakers, here’re the ones that had the most new information for me:

•      Mat Quinn of NIST covered its approach and work with ONC on the issue. Notably, NIST has published several documents in the area such as, NIST Guide to the Processes Approach for Improving the Usability of Electronic Health Records, (NISTIR 7741) which promotes a user centric approach to design and development.

•      I was really taken by Muhammad Walji’s study using a unified framework for EHR testing. The study compared user experience with the VA’s Vista program and a prototype system. It looked at:

o   What percent of an operation was substantive and what was overhead?

o   How long it took users to reach various performance levels.

o   How much memorization tasks took.

o   How many steps tasks required.

o   Error and recovery occurrence.

o   Time to complete defined tasks.

The study then applied its findings to rework the EHRs’ structure and workflow showing potential time and effort savings.

•      Anjum Chagpar of Toronto’s University Health Network. A human factors manager for this large healthcare network, she discussed the problems of integrating various vendor products into their system and their approach to usability and user satisfaction.

•      Buckminster Fuller famously declared, “I am a verb.” Dr. Lyle Berkowitz may not be a verb, but he is at least a gerund. His presentation swiftly covered several topics from HIMSS’ EHR Usability Task Force to usability definitions to stakeholder roles, and applying metrics to see how much of the problem was the system and how much the user.

•      The VA’s Dr. Jorge Ferrer provided several key references on usability studies.

Part II. NIST’s Proposed Protocol. If the first part took a broad and free ranging approach to usability, NIST’s staff approach was more focused. After an outline of the study’s setting and approach, the study director, Lana Lowery, outlined the protocol’s goal: prevention of unacceptable medical errors. These include errors of both omission and commission, for example:

•      Writing an order for the wrong patient.

•      Prescribing the wrong dosage.

•      Omitted information causing an error.

•      Critical delays in delivery due to system design errors.

•      Errors due to incorrect sequencing of actions.

Next, came examples of EHRs allowing errors. Unfortunately, several of the examples weren’t well thought out. For example, a patient ID error showed two patient records on the screen. One had the first patient’s x-ray, but the second patient’s name. Most likely, this would be a database problem or an x-ray production error not an EHR problem.

Robert Schumacher of User Centric, outlined how the protocol would be tested. For example, review and update of a problem list or replacement of one medication with another. The plan included testing several of ONC’s meaningful use functions that had usability factors.

Part III. Workshop Reactions. The workshop finally broke into two discussion groups: one for the draft protocol and the other on consensus building. In both cases, the discussion quickly went off script. Participants were quick to criticize the staff’s error oriented protocol as too narrow. Why, for example, did the protocol focus on internal EHR processes to the exclusion of workflow generated errors?

I understand NIST has a high interest in eliminating catastrophic errors, but I think there is not enough solid evidence on the kind and extent of the problem. No one discounts the need to prevent catastrophic errors, however, much of the EHR error focus is due to anecdotal reports of computer prescribing errors. From what I read, many of these reports are both old and recycled. Does anyone know the actual extent of major errors?

The FDA has developed several systems for dealing with medical device errors. These now include the software that the devices use. Even if the FDA does not regulate EHRs, it may step up its efforts to record important errors. I’d sure like to know FDA’s findings before I started an effort to shape EHRs.

This is not to say that safety is not important in EHRs, obviously the types of errors that are outlined by the staff are major. However, I think there are three points that are missing in the NIST approach:

•      Design for Success. You can’t design for failure. You have to design for success. The object of EHRs, as with any system, must be to accomplish certain ends. If you loose sight of that, you may not make mistakes, but you also will fail your objective.

•      Risk Analysis. Risk analysis measures the impact on a given population of an action, its potential and costs broadly defined. It also specifies mitigation efforts. I’d be far more comfortable about the protocol if there were a risk analysis behind it.

•      Error Handling. There should be more thought to error handling. For example, when the stall warning alarm goes off on a plane, it doesn’t grab the stick and take control. It’s a warning, just that. Physicians should be warned if they are about to prescribe beyond the recommended dose, but they may have good clinical reason to do it.

NIST put on a worthwhile workshop. My guess is that the draft protocol is not going to survive without modifications that take into account a broader range of usability issues and approaches.

CCHIT EHR Certification Enters EMR Usability World

Posted on October 28, 2009 I Written By

John Lynn is the Founder of the HealthcareScene.com blog network which currently consists of 10 blogs containing over 8000 articles with John having written over 4000 of the articles himself. These EMR and Healthcare IT related articles have been viewed over 16 million times. John also manages Healthcare IT Central and Healthcare IT Today, the leading career Health IT job board and blog. John is co-founder of InfluentialNetworks.com and Physia.com. John is highly involved in social media, and in addition to his blogs can also be found on Twitter: @techguy and @ehrandhit and LinkedIn.

I’ve been sitting on this post for a while. I figured it was finally time that we talked an interesting development in the CCHIT EHR certification: EMR usability testing. They first presented some of the details of this testing during the CCHIT training meeting. However, they also partnered with User Centric to formulate their EHR user testing and EMR and HIPAA has had a nice connection with User Centric for a few months now.

First the good. I’m glad that CCHIT is venturing into the realm of EHR usability testing. I’ve often talked about CCHIT Certification being rather useless since just because a piece of software does a certain function doesn’t mean that it does it well or that the EMR is usable. In fact, some of the most “feature rich” EMR software is completely unusable by the majority of people. Kind of reminds me of the days of terminals. If you knew the key strokes, it was incredibly efficient. However, learning the keystrokes was so much harder than a nice graphical interface which could do the same things. Not a perfect comparison, but interesting to consider.

So, the biggest problem with CCHIT measuring an EMR’s usability is that the EMR usability rating does NOT affect the certification outcome. Also, it appears that it will be up to the EMR vendor whether they want this result published or not. I wonder if we’ll get to a place where a few EMR vendors show their usability rating and others don’t. Those that don’t we’ll have to assume scored poorly? We’ll see how all that plays out.

I admit I haven’t looked over the entire EMR usability rating process. So, I can’t say if the process is complete or effective in and of itself. Although, I do have some confidence in User Centric as a company even if they’re trying to bite off the very difficult task of measuring EMR usability.

It does look like they’ll give the EMR software a usability rating that is not just a pass fail score. A rating is a much better thing when we’re talking about a somewhat abstract concept of software usability.

I’m also concerned about the quality of the jurors that they’ll use to try and measure usability. I’m sure they’re great people with great intentions. Honestly, that’s one of the most redeeming qualities of CCHIT. They have a large base of volunteers that are very well meaning. However, I’m not sure how much confidence I have in their ability to rate a software’s usability. For that matter, I’m not sure how well I’d be able to do it and I think I’m pretty familiar with the subject.

In a related issue, when you look at the way their putting together the score, it seems pretty complicated at first look. Like I said, I don’t know the details of the methodology. However, that’s kind of the point. Even if CCHIT does post an EMR vendor’s usability score, will the listening public (Translation: doctors) be able to quickly and easily understand what that score means? Maybe it’s a simple thing to figure out. We’ll see, but the devils always in the details and if I’m selecting an EMR I want it to be usable. So, I’ll be very interested in an EMR’s usability score.

Those are just a few things I noticed with the new CCHIT EHR Usability additions. Is there some other parts of it I missed? Anything else we should know about it? Will this be a valuable addition to the CCHIT Certification? Will EMR vendors revolt against it?

The Challenge of Creating a Good EMR User Interface

Posted on April 1, 2009 I Written By

John Lynn is the Founder of the HealthcareScene.com blog network which currently consists of 10 blogs containing over 8000 articles with John having written over 4000 of the articles himself. These EMR and Healthcare IT related articles have been viewed over 16 million times. John also manages Healthcare IT Central and Healthcare IT Today, the leading career Health IT job board and blog. John is co-founder of InfluentialNetworks.com and Physia.com. John is highly involved in social media, and in addition to his blogs can also be found on Twitter: @techguy and @ehrandhit and LinkedIn.

I recently saw a newsletter done by User Centric that had a couple interesting pieces.

First, Dr. Wendy Yee writes, “Most HCPs will bluntly tell you that they did not go through medical training to become data entry clerks. They also are highly sensitive to time requirements because their days can be brutally compressed.” Sounds a lot like my post EHR Software Makes Doctors Secretaries.

Dr. Yee also provides an interesting list of multiple items that are simultaneously going through a doctor’s mind while using an EHR:

  • There must be a way to make order entry faster…
  • What was the procedure code for that variation of a genetic test?
  • Let’s see, should I order Test ABC or Test DEF (or both)?
  • I need to check for Jane Doe’s lab results from yesterday.
  • Why isn’t that med listed under antivirals?
  • What does that obscure lab reading mean?
  • Has the patient’s problem list changed in the last day or so?
  • Why do I have to enter patient notes *this* way when the EHR at the teaching hospital has me enter it the other way?
  • Are there additional contraindications?
  • When was the patient’s last MRI?
  • I’m running behind, but I still need to enter this script.

Is it any wonder that it’s not a simple task to make an EHR that a doctor will consider usable? Not impossible, but certainly a challenge.

User Centric will be at HiMSS, Chicago! Booth #3382 They’ll be demonstrating how eye tracking can be utilized to uncover opportunities to improve user interfaces. No, I don’t have any financial connection with them, but eye tracking is really cool technology. Plus, I’m happy to support anyone that I think can make a difference when it comes to making EMR software more usable.

The Health IT Stimulus Package… for 2011?

Posted on February 27, 2009 I Written By

John Lynn is the Founder of the HealthcareScene.com blog network which currently consists of 10 blogs containing over 8000 articles with John having written over 4000 of the articles himself. These EMR and Healthcare IT related articles have been viewed over 16 million times. John also manages Healthcare IT Central and Healthcare IT Today, the leading career Health IT job board and blog. John is co-founder of InfluentialNetworks.com and Physia.com. John is highly involved in social media, and in addition to his blogs can also be found on Twitter: @techguy and @ehrandhit and LinkedIn.

I’m always happy to have people smarter than me do a guest post on EMR and HIPAA.  There’s far too much going on with Health Care IT for me to be able to cover everything that’s going on.  So, I’d like to thank Randy Pickard for sending in the following guest post about the HITECH stimulus act.

There is almost a Kafkaesque quality to the likely short term impact of the stimulus package upon adoption of Electronic Health Records (EHR) systems. The passage of the stimulus package will probably serve as a speed bump to EHR adoption until the details of the act have been spelled out. Up until the passage of the stimulus package, adoption of EHR systems has been proceeding slowly but steadily. However, the vaguely defined promise of $17 billion in reimbursements for EHR if unknown criteria are met could result in gridlock among purchasers in the short term while they wait for finalization of the provisions of the stimulus package’s Health Information Technology for Economic and Clinical Health Act (HITECH Act).  

A quick glance at the income statements of four publicly traded vendors that receive a significant portion of their revenues from EHR systems provides an indication of steady revenue growth from EHR sales. Income has been increasing by 10% or more per year for these four vendors, Allscripts, Cerner Corp., Eclipse, and NextGen.  (Although the increases in income is not simply due to EHR related sales. Acquisitions of other vendors and sales of other software products has also contributed to the revenue totals).

Company Symbol Period Ending Annual Revenue in ‘000’s Increase Vs. Previous Year
Allscripts MDRX Dec ’07 $281,908 24%
Cerner Corp CERN Dec ’07 $1,519,877 10%
Eclipse ECLP Dec ’07 $477,533 12%
NextGen Healthcare QSII Mar ’08 $186,500 19%

It seems likely that the revenue for these firms from new EHR sales will be greatly reduced in the near term, as purchasers sit on their hands waiting for answers to questions about how they can obtain reimbursement for their EHR spending. The HITECH Act designated that reimbursement would only be provided if a certified EHR was implemented. However, the certification standard is to be developed by an office (ONCHIT) that has not been staffed yet, with a coordinator that has not been named yet by the Secretary of HHS, who has not been appointed yet. Further, the bill indicates that reimbursement will go to establishments that show “meaningful use” of health IT, an undefined description that will likely deter healthcare organizations from rushing to purchase an EHR system. Given that the details of the plan to stimulate the adoption of EHR’s are far from being flushed out, is it any wonder that the Congressional Budget Office has estimated that a mere 2.3 percent of the health IT funds would be distributed in fiscal years 2009 and 2010?

About the Author – Randy Pickard is Vice President of Product Innovation for User Centric, Inc. a user experience research firm. User Centric recently released EHR and PHR white papers: How to Select an Electronic Health Record System that Healthcare Professionals Can Use and Google Health vs. Microsoft HealthVault: Consumers Compare Online Personal Health Record (PHR) Applications

Thanks Randy! If you’re interested in doing a guest post, feel free to Contact EMR and HIPAA.