Traditional Phishing Simulations Are Misleading You: Why Click Rates Are Insufficient

Traditional phishing simulations do not reveal your company’s true level of preparedness. Discover why click counting is insufficient and how PhishOS helps measure employees’ real ability to identify and respond to social engineering attacks.

Whether for compliance or genuine cybersecurity concerns, we’ve all come across a traditional phishing simulation. That classic email announcing a promotion or an alarming message can bring serious consequences to companies and institutions.

However, what few realize is that traditional phishing simulations are no longer effective and rarely measure what really matters. Instead of raising awareness, they deliver irrelevant metrics while creating fear, demotivation, and even reputational damage for organizations.

When “training” becomes a problem

A striking example of how traditional phishing simulations can negatively impact institutions occurred in 2023. A university in California ran a phishing test with the subject line: “Emergency Notification: Ebola Case on Campus.” The email—just a test—caused panic among students and staff. At what cost?

The problem with traditional metrics

In traditional phishing tests, metrics only show who clicked and who didn’t click on the phishing email. But this metric alone leaves room for countless doubts and interpretations.

A high percentage of employees not clicking does not necessarily mean the organization is safe. Employees may have been too busy to check the email, some may have been tipped off that the test was coming, or the phishing theme may have simply not been relevant to them.

As an example, Google has stopped running traditional phishing tests in recent years. In its official blog, the company explained: “Educating employees on how to alert security teams about active attacks remains a valuable and essential complement to a holistic security posture. However, there’s no need to make it confrontational, and we gain nothing by ‘catching’ people ‘failing’ the task.” (from the post “On Fire Drills and Phishing Tests”).

The root of the problem

According to Hacker Rangers founder and CEO Vinícius Perallis, the root issue lies in straying from the original purpose of phishing simulations: to measure an organization’s protection level against social engineering attacks. Companies began using them for awareness training instead. “The big misunderstanding is that the phishing simulation email is being used for a completely different purpose than the one it was created for and should fit into,” Vinícius explained on the Hacker Rangers Podcast.

The evolution: NIST Phish Scale

Acknowledging these shortcomings, in 2023 the National Institute of Standards and Technology (NIST) published the NIST Phish Scale, a guide on how to recognize phishing indicators. The document lists 23 types of cues to look for in suspicious emails.

PhishOS: the next generation of phishing simulations

With this in mind, Hacker Rangers developed PhishOS, a phishing simulator designed entirely around the NIST Phish Scale. PhishOS goes beyond mere click counting: players must explain why an email is or isn’t phishing, with 34 possible scenarios to identify.

With PhishOS, every simulation becomes a moment of practical education. Employees learn to spot danger signals in a safe environment, while managers gain accurate metrics on team behavior and progress.

It’s time to leave outdated simulations behind and adopt an approach that truly drives behavior change. With PhishOS, you move from simple “click counting” to a smarter, educational, and engaging strategy. Try PhishOS free for 30 days and see how to turn simulations into real results for your company.