Lack of engaging and appropriate materials
As previously mentioned, many or most awareness programs rely on computer-based training carried out once a year. CBT can vary greatly in quality. Sometimes an organization acquires posters and newsletters. When there is a check the box mentality, lowest cost is frequently the deciding factor in determining which program to use, and the low cost option is not always very good. Additionally, the materials might not be appropriate for the organization.
Even when low cost is not the deciding factor, you need to ensure that the materials are appropriate for the culture of your organization. Sometimes the person acquiring the materials has a bias for a particular presentation style, which is only engaging to a small segment of the organizations employees. For example, awareness materials appropriate for an Internet company will not be well received by investment bankers.
More important, it is critical that multiple versions of security awareness materials be implemented, as there are generational issues to consider. Research shows that younger employees respond better to blogs and twitter feeds, while older employees respond better to traditional materials like newsletters and posters.
Not collecting metrics
Without metrics, there is no way to know whether or not a program is truly successful in achieving its goals. You do not know whether you are wasting money or proving value. You do not know whether you are decreasing the number of losses.
By collecting regular metrics, you can adjust your program to the measured effectiveness. By determining what is working and what is not, you can tailor future programs based upon lessons learned. Without such data, you are acting blindly and potentially proliferating failure.
The appropriate metrics also allow for the determination of which components are having the desired impact. They should be taken prior to starting any engagement effort, at least once during the engagement, and also post-engagement. Without such metrics, you will waste time, effort and money. For example, if no one is reading your newsletters, there is no need to continue to create them.
Every time there is a security awareness failing, people bemoan the value of security awareness as a whole. While it would be great if security awareness could prevent all incidents arising from the exploitation of humans, it is not realistic. No security countermeasure will ever be completely successful at mitigating all incidents. There will always be a failure.
With the collection of metrics, you can prove the effectiveness of the program, and determine the most important aspect of the awareness program; whether the program is saving more money than it costs.
Relying upon a single training exercise
Similarly to relying upon the once a year CBT, many companies have begun to incorporate social engineering or phishing simulations to their awareness programs. While there is nothing wrong with these simulations as a form of training exercise, they only address a single awareness concern.
Sign up for CIO Asia eNewsletters.