The two primary attack vectors for malware is phishing messages sent to users, or unsafe web browsing. Both attack vectors target poor awareness. The losses related to malware are a better measure of phishing awareness than simulated phishing messages.
In the ideal world, you know the cost per malware incident. Assuming that the malware incidents on the network have decreased, you can attribute a legitimate value to the reduction of malware incidents. The reduced loss can be attributed to your awareness efforts, which might or might not be due to phishing simulations.
If the number of malware incidents increases or remains the same, or the severity of malware incidents increases, you then need to determine why that might be the case. It is conceivable that simulated phishing attacks will have no impact on reducing actual losses.
If your organization has other ways of tracking losses that can be specifically attributed to user related actions, or phishing, you can consider incorporating them.
Other benefits of phishing simulations
While I admit that I do not see a direct correlation for phishing simulations to decrease actual phishing susceptibility, there can still be benefits from the simulations. They do however have to be executed correctly to have an impact.
Phishing simulations can get people talking about phishing and security in general. They are made aware of the fact that they can be tricked, so they are more aware of the fact that they can fall victim to an attack. Most people think it will never happen to them. Simulations can create a teachable moment. How the awareness program uses the teachable moment then becomes critical.
If you have samples of actual phishing messages that were used to attack your organization or other organizations in your industry, it could be very beneficial to use those messages in the simulations. If users take an inappropriate action, they can receive the appropriate training, and hopefully decrease susceptibility to similar phishing messages in the future.
Phishing simulations also allow organizations to see how people react to potential phishing messages. Depending upon organizational policies, the desired reaction is to report the phishing message, so that the security team can triage the message, prevent other users from taking an undesirable action, and respond if it is determined that another employee responded to the message. In order for a security team to respond appropriately, they must know about potential attacks in progress, and a simulation might give the team an idea of the overall exposure and improvements that need to be made in reporting awareness.
It can be argued that success with phishing simulations has little to do with the actual number of people who do not respond to a message, but the number of people who properly report the messages. Phishing simulation success could be determined by the number of people who not only do not respond to the message, but also properly alert the appropriate authorities to the existence of the message.
Sign up for CIO Asia eNewsletters.