Consider the following two scenarios. Which one would you guess represents a security awareness program (SAP) that’s geared towards success?
Scenario No. 1
You provide employees with access to the organization’s LMS (learning management system), which hosts a variety of modules, videos, quizzes and games. The SAP manager records the progress of each employee to ensure they complete required training in the allotted time, thereby checking the awareness and compliance box.
Scenario No. 2
You provide employees with access to the organization’s LMS, which hosts a variety of modules, videos, quizzes and games. The SAP manager records progress, and regularly supplements materials with phishing campaigns, random ID checks, and a variety of in-real-life awareness assessments.
Obviously, the second scenario is the one every SAP should align themselves with. The key difference between the two is that SAP No. 1 measures input (progress as a result of completing tasks), while SAP No. 2 measures output (progress as a result of learning and applied knowledge).
Scenario No. 1 is easy to accomplish and it’s where most SAPs end their efforts. Scenario No. 2 is much more difficult, but it’s an essential part of any truly successful program.
Metrics are the Only Way to Truly Measure the Success of Your SAP
If you can’t measure it, you can’t improve it. Think about your new year’s resolution to lose weight or read more books – how do you know if you’ve met your goal if you don’t regularly weigh in or track the books you’ve finished (i.e. measure)?
How do you know if your phishing course is preventing clicks if you don’t measure the current rate of clicks? How do you know if your employees are bridging the gap between training and awareness if you don’t measure their behavior in some tangible way?
Metrics benefit your program in two extraordinary ways: 1) you gauge output as well as input and effectively measure success, and 2) you can identify the weaknesses of your program—such as which training materials resonate with your users and which don’t.
The data will tell you what works and what doesn’t, what issues need added reinforcement, and how to update your strategies as the threat landscape evolves. Furthermore, if you can provide hard numbers to your superiors, they will be much more inclined to address budget concerns, and might even get involved! Remember that a successful SAP requires buy-in from every level of your organization, from the top all the way down to the bottom.
The 2 Categories of Metrics
One of these metrics you’re probably already measuring. The other is the one you should be measuring.
Deployment: related to participation and initial engagement after deploying your SAP.
Impact: related to behavior changes within your organization based on the impact of your SAP.
Deployment is the natural process of releasing materials and tracking progress. Every LMS comes loaded with tracking capabilities. Measuring who has completed training is an example of deployment.
Impact, the more important of the two types of metrics, is all about effects of training—how employees react, respond and behave after learning something new. Impact is a measurement of behavior, from click rates to the likelihood of reporting incidents. By logging both deployment and impact, you infuse a continuous learning cycle into the administrative side of managing a successful program.
SAP Metric Chart
Remember: Failing to plan is planning to fail. So don’t set your SAP up for failure by not planning which deployment and impact metrics you will track. We’ve even developed a helpful chart to get you started!
Obviously, there is no one-size-fits-all security solution and there never will be, but the chart below lays out the fundamentals and provides a roadmap that will fit almost any program. You can download a PDF of the security awareness program metrics chart here.
As always, we are interested in how other people run their security awareness programs, so we welcome any and all feedback; take our survey about that here! And be sure to checkout our resource center for tons of information on planning, launching, and managing your program.