Because of COVID-19, many of us are finding we need to develop new skills or new ways of working. This may also mean that the learning programs created to teach us those skills must be redesigned for a virtual environment. As with face-to-face learning programs, you’ll need a carefully crafted method to evaluate the success of virtual training.
Fortunately, the same ways we measure classroom training can be used to measure the success of online training. More than 50 years ago, author and training consultant Donald L. Kirkpatrick created a method to measure learning effectiveness. His “Four levels of Learning” model is used to guide course developers as they create programs that deliver long-term value, and it can be adapted to a virtual training environment to help create lasting results.
Level 1: Measure Participants’ Reactions
The participant’s reaction is the extent to which they find the course engaging and enjoyable. In a face-to-face setting, instructors typically take the pulse of the class through an end-of-course survey that asks basic questions, like how they liked the material, format, and instructor.
Virtual Training Tip: As training is moved from the classroom to a virtual environment, it is important to think about how to ensure participants get the most out of the class, as well as how to measure their reactions. How will training material be changed to work on a screen instead of a printed manual? How will you ensure the virtual course is interactive and flows well? You can measure learners’ reactions using a virtual poll, and even consider requiring participants to tune in via video conference to can get a sense of their engagement.
Level 2: Gauge Learning and Retention
The most common way to measure learning in a face-to-face environment is to test the participant on the material, which is why many courses build toward certification exams. In a physical classroom setting, tests for learning are usually given immediately following the training and include questions that measure the absorption of facts and principles as well as the ability to use the material to analyze problems and find solutions.
Virtual Training Tip: When creating a virtual training program, you may have the flexibility to provide information in different formats to accommodate multiple learning types. Redesign the course to use as many different types of learning as possible and consider new and engaging technologies, such as online quizzing programs, that allow remote learners to test their knowledge. Other ways to increase learning and retention in a virtual setting, according to the Association for Talent Development, include utilizing a virtual whiteboard for brainstorming and hosting online breakout and discussion rooms.
Level 3: Track Behavioral Change
Now we come to the penultimate purpose of training. Even if training is great at levels 1 and 2, no real value is achieved unless it changes how the participant thinks or acts. Learning programs should drive actions. To measure behavioral change in a face-to-face setting, you can follow up with participants to see how they are making progress and whether they’re applying the course content.
Virtual Training Tip: One way to help ensure behavior is changed is to create online communities where information can be shared and discussions can occur, providing learners a chance to refine and expand their knowledge even after the course has concluded. Instructors who cannot follow up with participants in person can also reach out via email to determine how behaviors have changed.
Level 4: Measure Results
In the end, the goal of most trainings is to achieve results. One of the best ways to measure results is through a project or event where the employee is asked to put their newfound skills into action. Keep in mind that results do not always need to be measured in terms of monetary value. Increasing customer satisfaction or compliance to a government regulation may not result in immediate bottom line savings, but are still critical to any organization.
Virtual Training Tip: In a virtual environment, ensure a project or opportunity to use the new knowledge is built into the virtual training course. Make sure the remote learner has access to the resources they need to complete this project and report results.
Measuring A Successful Training: Miguel’s Learning Journey
Whether you’re conducting a face-to-face class or a virtual training, it’s critical to work through all four steps of the learning model. For example, Miguel, a participant in a problem-solving class I taught, mentioned after class how relevant he thought the content was to his work (Level 1: Measuring Reactions). Miguel was from Latin America and spoke Spanish as his native language, and I knew that taking a class in a non-native language can be difficult. Despite the added challenge, results from a short quiz at the end of the class (Level 2: Gauge Learning) showed he’d done very well. I later received an email from Miguel that said he went back to the office and started thinking about what he could change to make improvements. (Level 3: Tracking Change). Several months later, I received another email from Miguel along with a copy of a certified letter from his local government. He explained that after the class, he started looking into the import duties his company was paying to bring products into the country and found that many of the product classifications were incorrect. He corrected the issues, and the government refunded $750,000 for the overpayments for that year (Level 4: Measure results).
All four levels of learning were in place, helping ensure an optimal outcome. If Miguel had not found the class exciting or did not retain the knowledge of its content, he never would have been moved to change his behavior or utilize his knowledge and would not have recovered the overpayment.
Whether you are an instructor, instructional designer, or a learner, virtual courses have become a bigger part of our lives. By taking the time to develop and measure virtual training, we can ensure we are getting the most out of the time and funds invested.