The 4 Learning Metrics Linked to Successful EHR Adoption – Breakaway Thinking

Posted on June 16, 2016 I Written By

John Lynn is the Founder of the blog network which currently consists of 10 blogs containing over 8000 articles with John having written over 4000 of the articles himself. These EMR and Healthcare IT related articles have been viewed over 16 million times. John also manages Healthcare IT Central and Healthcare IT Today, the leading career Health IT job board and blog. John is co-founder of and John is highly involved in social media, and in addition to his blogs can also be found on Twitter: @techguy and @ehrandhit and LinkedIn.

The following is a guest blog post by Shawn Mazur, Instructional Writer at The Breakaway Group (A Xerox Company). Check out all of the blog posts in the Breakaway Thinking series.
Shawn Mazur - The Breakaway Group
There seems to be a trend in the education processes of a go-live for large EHR implementations: they’re scary. For large hospitals, the task of providing learning to hundreds, if not thousands, of employees for a go-live is daunting, and no matter how much time and resources you pour into designing the perfect curriculum and planning out a detailed schedule, you may quickly end up feeling like your learning effort is falling short. Learning metrics can play a vital role in making the task of creating and managing learning for a big go-live a little less scary.

Despite high levels of EHR implementations since the HITECH Act, many organizations still have significant go-live events in their future. A majority of learners are at least somewhat familiar with EHR systems, so education needs to be focused on making learners comfortable with a new, or advanced, EHR rather than teaching all there is to know about the systems. Since 2014, the number of buyers replacing existing EHR software has increased 59%, according to a 2015 EHR BuyerView report. It was also reported that challenges facing an organization were not overcome by the implementation of a new EHR. A lack of education for any go-live event will discount the value behind a new EHR.

Having the perfect plan for EHR education from the beginning is not the only key to successfully preparing your employees for go-live. Additionally, you should implement a plan to monitor the training process, completing learning metrics as you go, and then be flexible in how you carry out the remainder of your learning. So, you decide to be flexible in the information you provide to learners, but when do you know it’s time for a change in direction? Going beyond the summary of what your users should learn if they complete all of their learning, the following four metrics tell you how learners are reacting to the content.

1. Completion Summary
A simple but effective metric that lets you know how much progress your users have made in their learning objectives. This metric is especially important with e-learning and with self-paced learning. Collecting this data will also help you identify problems with different learning roles throughout your organization. Flagler Hospital, a regional hospital, kept completion summary metrics throughout their large switch from Meditech to Allscripts. They reported that their completion metrics began to show users were completing their learning much faster than expected. This data allowed Flagler to actualize their education plan to make remarkable reductions in training schedule, time, and cost from their original plan. Had Flagler’s completion summary shown less than satisfactory numbers, it would have also provided an opportunity for changes to be made. Low completion rates may mean that one role’s users are getting stuck at a certain point of their learning or struggling to even begin. In these cases, use completion metrics to push learning requirements along in time for go-live.

2. Assessment Summary
If your organization isn’t planning on testing users on the education they’ve received, it may be time to consider doing so. Using a step-by-step simulated assessment is the easiest way to put a solid number on how prepared your users are for navigating workflows in the live system. After implementing tests, compile metrics on them at a high level, including how many learners took their test, how many times each user attempted a test, and of course, the percentage of assigned learners who successfully passed their test. Flagler hospital also used assessment metrics alongside their completion summary. As a result, they saw that that their completion summary aligned with their assessment summary. Along with the fast pace at which they were completing learning, Flagler’s learners had average testing scores of 94 percent. The high test scores solidified their decision to make changes to the original learning schedule.

3. Assessment Audits
After implementing step-by-step testing of your user’s knowledge, dig deeper into your testing scores to pinpoint exactly where users are falling short. You will often find that a deficiency in learning curriculum leads to users missing the same steps during their test. For example, let’s say you break down your scores by step and see that over 60 percent of users clicked the incorrect button for documenting current vitals. This is an advantage over less effective traditional testing methods, like multiple choice formats. From this metric, it is clear that you should delegate additional learning resources on best practices for entering vitals before your go-live approaches.

When you test users without using the metrics to facilitate better learning, your learners will feel frustrated with their lack of proficiency. In his book, Why High Tech Products Drive us Crazy, Alan Cooper defines two types of learners. He says, “Learners either feel frustrated and stupid for failing, or giddy with power at overcoming the extreme difficulty. These powerful emotions force people into being either an ‘apologist’ or a ‘survivor.’ They either adopt cognitive friction as a lifestyle, or they go underground and accept it as a necessary evil.” Auditing your tests by step gives you the opportunity to return to your curriculum to elaborate on topics with low testing proficiency. Pinning down topics that require additional learning will eliminate the frustration and feeling of defeat among learners failing their assessments.

4. Knowledge and Confidence Level
Confident learners are a good thing, but not always the best come go-live. It is important that your learners not only have confidence, but also the knowledge to back it up. When knowledge and confidence are not aligned, the user is in a bad place for not only lacking proficiency in the system, but for their education going forward. Users who are pushed to use the live system before they feel confident enough will be far from proficient in the system, and will feel a resentment against the organization moving forward. Equally so, users confident to get in the system but lacking the knowledge to be proficient will also fail, and be quick to blame it on poor learning. In his book, Cooper also says, “Users only care about achieving their goals.” When learners can’t achieve their goals for the learning, they are quick to find a way to reach their goal, defining their own workflows and workarounds instead of sticking to best practices outlined by your organization. Collecting data from your learners, usually through a survey-like format, on how confident they are to start working in the live system and how knowledgeable they feel about the information taught, will help you gauge how ready users are for go-live. When aligning this with your other learning metrics, you will quickly see how ready your users are to proficiently use the live system.

It is often the case that the education plans you spent countless amounts of time and resources on leaves learners feeling distant with the EHR. Think about how you can use metrics to track your learning and be flexible to make changes using those metrics to benefit your learners in the long run.

Xerox is a sponsor of the Breakaway Thinking series of blog posts. The Breakaway Group is a leader in EHR and Health IT training.