Investigate disputed phishing simulation clicks using event reconciliation, preflight tests and clear reporting that preserves original totals and unresolved cases.
You share the results of a phishing simulation, and an employee says they never clicked the link. Or you notice that many recipients have click events immediately after delivery. Treating the report as proof of carelessness, or cancelling a result solely on the basis of a denial, leaves the same question unresolved for the next campaign.
The useful distinction is between an observed click event and a conclusion about who performed an action. This article proposes a process for preserving the original report, investigating automated inspection, and explaining results that remain uncertain. For campaign planning and reporting rates, see our guide to targeted email simulation and measurement.
A click event does not always prove that the recipient clicked
Microsoft documents that security tools can generate click telemetry in Attack simulation training without a manual click. Forwarding and reporting workflows can also matter. Its separate UrlClickEvents log does not necessarily contain every interaction, so absence there is not proof of no click. These are Microsoft-specific explanations; check the behaviour of your own simulation and email products. Source: Microsoft deployment considerations and FAQ
The starting point is not to remove suspicious records. Establish whether a record can appear when nobody interacts with the message, and whether automated and human activity are mixed in the same recipient's history. The following process is Yagura's proposed operational approach. It is neither a description of a particular product's automatic classification feature nor an official vendor procedure.
1. Preserve the original report and its conditions
Save the original report for the campaign under investigation. Alongside the displayed totals, record the export time, reporting cutoff, recipient scope, treatment of delivery failures, and handling of repeated events for one person. This helps distinguish differences in reporting time or update delays from classification errors.
Ten click events and ten recipients with click events are different measurements. Three inspection requests associated with the same recipient may represent several events but only one recipient. Decide whether the metric counts events or people before investigating, and keep that unit consistent.
Do not overwrite the original numbers when a conclusion changes. Keep the previous and revised result, the reason for the change, the reviewer and the review date. Even if the product cannot amend a simulation outcome, a separate investigation record linked to the official report can preserve the basis for your explanation.
2. Collect eight fields for event reconciliation
Gather the following information within the access already granted to the responsible team. Products do not all expose the same fields. Mark unavailable information as unavailable, and do not turn a blank field into a conclusion.
Campaign and recipient identifiers: IDs that prevent events from different campaigns or people being mixed.
Delivery and event times: the time zone, delivery time and recorded time of the click or other event.
Event definition: whether it represents a URL request, attachment action, completed submission or another interaction.
Connection details: available IP, browser and device information, used as supporting evidence rather than proof on their own.
Message route: forwarding, reporting, shared mailboxes and any subsequent automated processing.
Related processing logs: whether email inspection or reporting workflows match the event's time and target.
Statements from the recipient or operator: the approximate time and device used, and anything they cannot establish.
Classification and rationale: human activity, automated inspection or unresolved, with the reviewer and any follow-up date.
Microsoft's report documentation explains event meanings and supporting fields such as IP, browser, device and time. These are investigation inputs, not individual proof of a person's action. Read your product's event definitions before using them in a reconciliation record. Source: Microsoft simulation reports
3. Investigate timing, processing routes and user actions
Use an immediate cluster as a lead, not an exclusion rule
A cluster of events shortly after delivery is a reason to investigate automation. It does not establish a universal rule that any event within a particular number of seconds is automated. A person might act promptly, and timestamp precision may be limited. Check whether the same pattern appears for a test account whose message nobody interacts with.
Match the processing route instead of relying on an IP address
An IP address shared by many recipients may belong to a shared corporate connection or an intermediary service. An unfamiliar browser string is not sufficient evidence of automation either. Combine these observations with other evidence, such as a matching inspection-service record or a similar event following a recorded report submission.
Opening a live simulation link during an investigation may itself create another event. Start with logs or the product's investigation features. If reproduction is necessary, use a test recipient.
Evidence of a human action does not necessarily identify the original recipient as the actor. One hypothesis to investigate is whether someone who received a forwarded message opened the recipient-specific link. Without evidence that attributes the action to the original recipient, keep that person's involvement unresolved.
Separate recipient confirmation from missing evidence
A recipient's statement matters, but a denial alone does not establish automated inspection, and uncertain recall does not establish a click. Ask for whatever timing and actions they can recall, and reconcile that information with the records. If necessary logs are missing or the device cannot be identified, keep the result unresolved.
Classify individual events before aggregating recipients. If automated inspection is followed by evidence of a human click for the same person, do not exclude that person merely because automation occurred first. When classifying a result as automated inspection only, state the period and records examined.
4. Test three measurement paths before the main campaign
To reduce repeated investigations, use authorised test accounts and harmless simulation material to check the following paths. This tests measurement behaviour, not employee ability.
Leave a delivered message untouched. Observe which records automated processing creates without a human interaction.
Have an operator deliberately click and record the time. Check that the expected event is associated with the correct test recipient.
Report the message through the agreed channel without opening its body link. Check what the report, forwarding and subsequent investigation processes record.
Use separately identifiable test recipients or campaigns for each path so their histories do not become mixed. Record the actions and times, then compare the original report and related logs for each path. Include relevant differences in mail clients and reporting routes. Do not stop organisation-wide email inspection to improve simulation numbers. Consider changes under the product's documented behaviour and your organisation's change process. If settings must change, define their scope, duration and rollback method, and verify the resulting behaviour.
5. Report original numbers alongside the investigated breakdown
The following is a fictional calculation, not a Yagura result or an industry benchmark. Suppose delivery is confirmed for 100 recipients and the original report records clicks for 10 of them by the reporting cutoff. Investigation establishes actions by the original recipients for five, automated inspection only within the examined records for three, and leaves two unresolved. Report the results as follows:
Original report: 10 recipients with click records out of 100 delivered, or 10%.
Investigation result: five recipients whose own actions are confirmed out of 100 delivered, or 5%.
Remaining breakdown: three recipients with automated inspection only in the examined records, and two unresolved.
Do not label 5% the true click rate. The unresolved cases may include human activity, and some actions may not have been measured. Removing recipients from the denominator because an automated event occurred also changes the comparison conditions. In this example, retain the denominator of 100 delivered recipients and name each metric according to what has actually been established.
For comparisons over time, use the same reporting cutoff and classification rules. If the rules change, determine whether earlier campaigns can be recalculated in the same way. If they cannot, do not interpret a lower number as proof of training effectiveness. Assess the difficulty of the message separately using our guide to choosing phishing simulation difficulty.
Ask whether results can be explained, not just displayed
When reviewing an operating process or selecting a product, go beyond whether it displays a click rate. Check its event definitions, available raw data, retention periods, handling of automation, and ability to amend outcomes. If a vendor says it excludes automated inspection, ask about the classification's scope and limitations, treatment of uncertain data, and comparisons after changes to its logic.
Keep the reporting and advice channel available while disputed results are being investigated. Agree with stakeholders to separate measurement problems from learning needs, and to base follow-up on established actions rather than rushing to evaluate individuals using disputed figures.
See Yagura Awareness for our email training and education offering. To discuss your training goals, measurement questions and operating arrangements, contact us. The reconciliation process in this article is a general operational proposal; it does not state that Yagura's product collects or automatically classifies every field described here.
