Stakeholders expect to see a return on their investment in training. In some cases though, they struggle to conceptualize the best way to evaluate the effectiveness of their security awareness training. They are in good company. Training evaluations can be complex, expensive, elusive, and baffles even seasoned pros.
Many busy program leaders instinctively reach for the knowledge check at the end of training. A standardized, graded test is an easy way to measure learning and compare performance, right? Maybe so, but at PhishLabs, we argue against only relying on knowledge checks for a couple of key reasons.
First, knowledge checks can collide with key learning principles. As adults, we are goal-driven and focused on practicality and relevance. These knowledge checks can feel rote and tedious. With all of the competition for your employees' time, we can't afford to waste it by inviting them to phone it in during training.
Without delving too deeply into learning theory, here's the takeaway: measuring knowledge with a test immediately after training is one of the least impactful forms of training evaluation. This is because it only measures how much information the learner has absorbed and can recall immediately. So, let me ask the following: Are you worried about immediate or lasting results? Are you concerned with knowledge or behavior?
The purpose of phishing training or any other security training, is to change behavior. We want to see employees practicing good security behaviors more frequently. It stands to reason then that a more meaningful way to measure the effectiveness of these trainings is with behavioral data over time rather than a knowledge check immediately after a module is complete.
Consider ways to measure the effectiveness of your other security training programs. Are there metrics around data security, password vigilance, or other key behaviors that you can gather to measure the effectiveness of your training programs?
In the context of phishing training, we recommend focusing on phishing simulation results. Has the click rate decreased? Has the report rate increased? The results of your regular phishing simulations offer the best insight into program effectiveness.