CASE STUDY · MANAGED SECURITY CULTURE
Nine hundred thank-yous
Every time someone at a UK professional services group reports a phishing email, they get a thank-you in Teams within seconds. No waiting for someone to review a queue. Nobody decides who deserves a reply. It fires on every report, automatically, and it has fired more than nine hundred times.
That is not a nicety. Reporting is the behaviour the entire programme depends on, and unacknowledged behaviour decays.
1,500
people in the programme, across learning and phishing
900+
phishing reports acknowledged automatically
36%
reported a simulated phish, against a 20% target
45
people failing on both axes, clicked, and training overdue
Two reports that never meet
Phishing behaviour and learning completion live in the same platform, and are reported separately, as two numbers that never resolve to one person.
Each view answers its own question well. Neither answers the one that matters operationally: who is demonstrating risky behaviour?
Culture360° resolves them into a single culture score, calculated across the 1,200 people who appear in both. Clicks and credential submissions raise it. Reporting, reporting speed and on-time completion pull it back down, with mitigation capped so good behaviour can never cancel out a credential submission.
The joining is done in Report360, our own reporting platform. The phishing and learning exports go in; Report360 reconciles them person by person, applies the weights, and produces the score, the quadrant and the chase lists. Every table it draws has a CSV download beside it, so nothing is trapped in a slide.
The current score is 44 out of 100, moderate, where lower is better. Phishing behaviour alone scores 15, very low. The gap between those two numbers is the finding: the phishing programme is working, and training is dragging the organisation down. You cannot see that at all while the two are reported separately. It is also why click rate on its own tells you very little.
THE MEASURE
One score, and the gap inside it
Culture Risk Score. Lower is better. The combined score sits at 44, moderate. Phishing behaviour on its own sits at 15, very low. The distance between the two markers is the training problem, and it is invisible in either report alone.
The bands are set, not inferred. Very low to 20, low to 40, moderate to 60, high to 80. A score means the same thing this month as it did last month and the same thing for this organisation as for any other we run.
Against targets defined with the client
Two measures beating target, one just missing, three missing badly, and the three are all training.
REPORT RATE
36%
Target ≥20% · beating
REPORT AFTER CLICKING
40%
Target ≥50% · missing
CLICK RATE
7%
Target ≤10% · beating
TRAINING COMPLETED ON TIME
32%
Target ≥90% · missing
CREDENTIAL RATE
2.4%
Target ≤2% · just missing
TRAINING OVERDUE
43%
Target ≤5% · missing
The grid that produces a chase list
Phishing behaviour on one axis, training currency on the other. Three cells are operationally useful.
620
Clean on both — no click, training up to date
50
Clicked only — training is current
485
Overdue only — has never clicked a phish
45
Failing both — clicked, and training overdue
Not a chart to read: one hundred and five names, with a reason attached to each, and the 485 are a different problem. Most have never logged in at all, which could be an access problem, while the rest have logged in and started nothing, which is an engagement problem. Different owners, different fixes, and a single completion percentage hides both.
AUTOMATION
Proportionate remediation
Clicking a link and handing over your password are not the same mistake, so they do not get the same response.
CLICKING A LINK
Three Strikes
01
First Click. A Teams notification, reminding them to stay vigilant. One mistake, not a pattern.
02
Second click. Another notification, this time pointing at a relevant CybSafe article. Short, specific, read in a minute.
03
Third click. Enrolled in full training, which stays assigned until completed, with automatic Teams chasers until it is. Then they leave the group on their own.
SUBMITTING CREDENTIALS
One Strike
01
Straight into training. No escalation ladder, because there is nothing to establish.
The two never collide. The click workflow re-checks at every stage whether the person has since entered data and landed in the credential group. If they have, it steps aside. Automation that fires independently on every event produces people receiving three messages about one mistake, which teaches them the messages are noise.
One loud month, eleven quiet ones?
Not on our watch. Whilst October is managed end to end, we maintain awareness throughout the year.
Third parties are engaged and coordinated: cyber escape rooms, external speakers, activities that put people in a room together. These complement on-platform content, nudges, goals and the Security Heroes programme, rather than competing with them. Off-platform content is authored too, including blogs published in the client's own voice.
The reason to go beyond the platform is simple. Everybody ignores the annual awareness email, and nobody ignores an escape room. Awareness month is the one point in the year when attention is genuinely available, and spending it on a content library wastes it.
A leaderboard and rewards carry the engagement. Points for completing bite-sized modules, taking part in awareness month activities and reporting phish, surfaced as a visible leaderboard with recognition for the people at the top. It works because security training competes with everything else in someone's day and has nothing intrinsic to offer them, so give it something. Team-level standings do more than individual ones: nobody wants to be the department that hasn't bothered.
But a campaign is not a culture. The month exists to spend attention the other eleven have earned, and to feed them. The quarterly modules carry the themes forward, the nudges keep them alive between campaigns, the phishing simulations test whether any of it stuck, and the culture score measures all of it every month rather than once a year.
However, an organisation that does October well and nothing else does not maintain a security culture.
Rebuilding the learning journey
The measurement showed training was the weak axis, so the journey was redesigned around why.
Joins
Triggered from the identity feed the moment the account exists.
Buffer
Nothing fires before they have started, leaving room for any onboarding.
Starts
Welcome nudge, and the compulsory introductory module becomes available.
The Gate
No quarterly content is presented until the compulsory module is complete.
Ongoing
Moved across automatically, with a Teams notification on the move.
Admin and excluded accounts are filtered at entry rather than chased later. Ongoing content then runs at two modules per quarter, one training module with a quiz, one video, because training fatigue is the reason completion sat at 55%.
QUARTER ONE
Spotting fake emails
Ransomware
QUARTER TWO
Business email compromise
Agentic AI
QUARTER THREE
Preventing identity theft
Romance fraud
QUARTER FOUR
Real incidents
Deepfakes
New starters joining mid-year complete the compulsory module and then join whichever quarter is running. They do not backfill.
Looking ahead
Nudges reach the individual. Where completion has stalled for long enough, the next phase of the journey escalates to their line manager, because the person who can actually create the half hour to do the training is rarely the person being nudged about it.
It is deliberately the last step rather than the first. Escalate too early and the programme becomes something managers dread being copied into; escalate once the automated chasers have genuinely been exhausted, and it carries weight.
Then we tested our own configuration
Fifteen test cases across two streams, with an evidence pack, exit review and sign-off before go-live.
Seven cases on the learning journey, eight on the phishing workflows, each checked on both sides, the group transition in CybSafe, and what the learner actually sees when they log in. A state change that happens in one and not the other is exactly the failure nobody notices until a person complains.
The principle: test what has been configured, not the functionality the vendor provides. Module players, progress saving and completion recording are CybSafe's problem. Workflow triggers, audience filters, group transitions, nudge firing and what the learner actually sees at each stage are ours.
Testing was scheduled to finish before Cyber Awareness Month, so the security team is not reviewing test evidence in its busiest month — and campaign one opens into a population already warmed up by it.
The score is auditable
Every weight, target and threshold is published in the report and editable in the dashboard. Credential submission carries a weight of 8 against a 2% target; reporting carries 6 against 20%; mitigation is capped at 45 points. Small groups are smoothed towards the mean, so a team of three cannot top the table on a single completion.
These are settings in Report360, not assumptions buried in a spreadsheet, which is why they can be published in the report and changed when the organisation's targets change.
A risk score you cannot interrogate is a number, not a measure.