COVID-19 prompted many of us to develop new skills or new ways of working. Additionally, the learning programs to teach those skills are being redesigned for a virtual environment. As with any learning program, a carefully crafted method best evaluates the success of virtual training.
Fortunately, we can measure the success of online training in the same ways we measure classroom training. More than 50 years ago, author and training consultant Donald L. Kirkpatrick created a method to measure learning effectiveness. His “Four Levels of Learning” model guides course developers as they create programs that deliver long-term value. It can also adapt to a virtual training environment to help create lasting results.
The Four Levels of Learning
Level 1: Measure Participants’ Reactions
The participant’s reaction is the extent to which they find the course engaging and enjoyable. In a face-to-face setting, instructors typically take the pulse of the class through an end-of-course survey. They ask basic questions to get feedback about the material, format, and instructor.
Virtual Training Tip: As training moves to a virtual environment, it is important to ensure participants get the most out of the class. In addition, consider how to measure their reactions. How will you change training material to work on a screen instead of a printed manual? How will you ensure the virtual course is interactive and flows well? Some ways to accomplish are to measure learners’ reactions using a virtual poll and require them to tune in via video to get a sense of engagement.
Level 2: Gauge Learning and Retention
The most common way to measure learning in a face-to-face environment is by testing. For this reason, many courses include building toward certification exams. In a physical classroom setting, tests for learning are usually given immediately following the training. They include questions that measure the absorption of facts and principles. They can also require participants to use the course material to analyze problems and find solutions.
Virtual Training Tip: A virtual training program offers the flexibility to provide information in different formats to accommodate multiple learning types. Redesign the course to use as many different types of learning as possible. Consider new and engaging technologies, such as online quizzing programs, that allow remote learners to test their knowledge. The Association for Talent Development also suggests utilizing a virtual whiteboard for brainstorming and hosting online breakout and discussion rooms.
Level 3: Track Behavioral Change
Now we reach the purpose of training. Training that is great at levels 1 and 2 only achieves value if it changes how the participant thinks or acts. Learning programs are meant drive actions. To measure behavioral change in a face-to-face setting, follow up with participants to gauge progress and determine if they’re applying the course content.
Virtual Training Tip: One way to help ensure behavior change is to create online communities to share information and discussions. This provides learners a chance to refine and expand their knowledge even after the course is complete. Instructors who cannot follow up with participants in person can also reach out via email to evaluate behavior change.
Level 4: Measure Results
In the end, the goal of most trainings is to achieve results. One of the best ways to measure results is through a project or event where the employee puts their newfound skills into action. Keep in mind that results do not always equate to monetary value. Increasing customer satisfaction or compliance to a government regulation may not result in immediate bottom line savings, but are still critical to any organization.
Virtual Training Tip: In a virtual environment, ensure a project or opportunity to use the new knowledge is built into the virtual training course. Make sure the remote learner has access to the resources they need to complete this project and report results.
Measuring A Successful Training: Miguel’s Learning Journey
Whether you’re conducting a face-to-face class or a virtual training, it’s critical to work through all four steps of the learning model. Below is an example for Miguel, a participant in one of my problem-solving classes.
- After class, he mentioned after how relevant the content was to his work (Level 1: Measuring Reactions).
- Miguel was from Latin America and Spanish is his native language. Taking a class in a non-native language can be difficult. Despite the additional challenge, results from a short quiz at the end of the class (Level 2: Gauge Learning) showed he did very well.
- Later, Miguel emailed to share that he went back to the office and started thinking about what he could change to make improvements. (Level 3: Tracking Change).
- Several months later, Miguel emailed again to explain that he started looking into his company’s import duties and found many of incorrect product classifications. He corrected the issues, resulting in a government refund of $750,000 for that year’s overpayments (Level 4: Measure results).
All four levels of learning were in place, helping ensure an optimal outcome. Miguel found the class exciting and retained the knowledge of its content. He was moved to change his behavior and utilize his knowledge. As a result, he recovered the overpayment.
Develop and measure for success
Whether you are an instructor, instructional designer, or a learner, virtual courses have become a big part of our lives. By taking the time to develop and measure virtual training, we can ensure we are getting the most out of invested time and funds.