Social engineering targets human psychology and organizational gaps rather than technical vulnerabilities to steal information or funds. Once dominated by primitive methods like "shoulder hacking," today's attacks primarily leverage phishing emails, fake phone calls, and fraudulent chats. Furthermore, generative AI can now synthesize email text, voice, and video in real time, leaving humans with almost no room to detect anomalies based on gut feeling. As previously high-cost capabilities like "human-like text generation" and "real-time voice synthesis" become instant and inexpensive, generative AI-driven social engineering attacks continue to rise. Based on the latest statistics and case studies, this article outlines the immediate steps defenders must take.
The Latest Landscape of Generative AI-Powered Social Engineering Attacks
Social engineering refers to manipulative techniques used to illicitly acquire sensitive information or steal money by exploiting human psychology and organizational procedural gaps rather than technical vulnerabilities. Previously, primitive methods like "shoulder surfing" (peeking at screens over someone's shoulder) or tailgating (sneaking into secure premises by pretending to be a visitor) were the norm. Today, these have been replaced by phishing emails, vishing (voice phishing), and fake chat messages.
The arrival of generative AI has decisively transformed this landscape. It is now possible to synthesize email text, voice, and video in real time, leaving humans with almost no clues or inconsistencies to spot. From a cost perspective, "human-like text generation" and "real-time voice synthesis"—which were once highly resource-intensive—can now be executed instantly and at minimal cost. Consequently, damage caused by generative AI-driven social engineering attacks continues to escalate rapidly.
Recent Statistics and Incidents
To illustrate this situation, we present three key statistics and incidents below.
Arup Deepfake Incident (February 2025)
A finance professional based in Hong Kong participated in a Zoom meeting with five participants claiming to be the CFO, Finance Director, and Auditors, and approved 15 international wire transfers in 24 hours (averaging 1.7 million USD per transfer). The CFO was traveling in Europe and did not refuse additional verifications due to the time difference. The video, voice, and background noise in the meeting were real-time deepfakes, which facilitated the deception. The transfers were sent to nine bank accounts in Hong Kong, Singapore, and the UAE; authorities were only able to freeze eight of these accounts (approximately 1.3 million USD).
BEC Losses Reach 3.73 Billion USD
The FBI IC3 reported that in 2024, it received 22,189 complaints regarding Business Email Compromise (BEC) with total losses reaching 3.73 billion USD. The IC3's Recovery Asset Team (RAT) processed 3,008 freezing requests and successfully recovered 66% (538 million USD) of the funds.
Surge in Generative AI Phishing
The IBM Security Threat Intelligence Index 2025 estimates the number of AI-generated phishing samples at approximately 2 million. This represents an 84% increase compared to the previous year (1.08 million). The financial and healthcare sectors alone accounted for 62% of these samples, and the rate of grammatical errors in the emails dropped from 6.9% to 0.8%.
Six New Threats Created by Generative AI
As these cases demonstrate, the evolution of generative AI has modernized traditional social engineering and diversified attack vectors. Here, we outline six new fraudulent methods enabled by generative AI.
Threat 1: AI-Generated Phishing
A typical scenario unfolds as follows: An attacker first gathers job titles, project details, and purchase codes using the LinkedIn API and purchasing lists from the dark web. Next, they prompt an LLM to generate highly polite, professional emails and automatically insert signature blocks that match internal corporate templates with 99% accuracy. Finally, they spoof SaaS billing domains to send the message. In May 2025, a legal department at a North American SIer mistakenly wired 90,000 USD due to this tactic. Because these emails bypass machine-learning-based typo detection and SPF/DKIM checks, traditional rule-based filters struggle to detect them.
Threat 2: Deepfake Voice and Video
Attackers can build a voice model by extracting just three minutes of audio from YouTube or webinars. Combining this synthesized voice with Zoom's virtual camera features, they can simulate realistic faces, backgrounds, and lighting to issue real-time transfer requests during meetings. In the Arup incident, this method was used to execute 15 international transfers averaging 1.7 million USD. Because environmental elements like air conditioner hums and keyboard clicks are also synthesized, the human senses can rarely detect any anomalies.
Threat 3: Impersonating Helpdesks and Stealing MFA Tokens
Attackers use AI-generated phone scripts to pressure IT support staff, claiming they lost their smartphone and urgently need their MFA reset. In April 2025, 140 accounts at a US-based SaaS vendor were compromised using this method. The compromise quickly escalated across privileges from Okta to Salesforce to expense management platforms, resulting in multi-layered damage.
Threat 4: AI Scraping and Spear-Phishing
Attackers scrape LinkedIn, GitHub, and investor relations materials to identify recently promoted managers or new project leads. They then weave personal interests such as cars or golf into email templates to quickly build trust. In June 2025, a newly appointed PM at a Japanese manufacturer mistakenly shared development specs to an external storage site. Because the detailed profile was built entirely on public information, it triggered no SOC alerts, making detection exceptionally difficult.
Threat 5: Multilingual Smishing
Because LLMs can seamlessly code-switch between languages, translated SMS campaigns are no longer flaggable by standard translation errors. In July 2025, 180 BYOD devices across three foreign-affiliated firms in Japan were infected with malware. These attacks bypassed country-code filters and built high credibility by including actual business travel itineraries and flight numbers in the message body.
Threat 6: AI-Generated Dropper Documents
LLMs write AES-encrypted JavaScript inside PDF or Office files, which are then distributed as attachments. Upon opening, they execute dynamic link libraries (DLLs) to establish connections with Command & Control (C2) servers. By the time the EDR responds, sensitive information has already been exfiltrated. In a May 2025 incident where research data was stolen from a European pharmaceutical firm, sandbox analysis took over 20 minutes, failing to provide real-time detection.
The Three Core Realities
Based on these facts, what must we learn about security in the age of generative AI? Yagura believes that addressing the following three gaps will be critical in future social engineering defense strategies.
1. The Speed Gap
There is a massive speed gap between humans and generative AI. While AI can draft highly convincing, personalized phishing emails in seconds, it takes a human dozens of seconds or even minutes to analyze and flag them. In many cases, humans cannot verify them even with time.
To counter these highly sophisticated, rapid attacks, defenders must also leverage generative AI. Fortunately, the development of generative AI also makes it highly cost-effective for defenders to analyze all incoming emails and flag anomalies. To close this speed gap, deploying AI in defensive security is essential.
2. The Cost Gap
Now that the barrier of entry and operational costs for attackers have dropped drastically, a monthly investment of just tens of dollars can yield millions in returns. This cost gap is the primary driver of attacker profitability, and attacks against organizations without generative AI defenses will only accelerate.
Japan, in particular, was historically shielded by the complexity of its language, which left defensive security awareness relatively underdeveloped. Consequently, Japanese companies lacking defenses against AI-powered attacks face severe risks of substantial financial and operational damage.
3. The Perception Gap
There is a growing gap in how people perceive the realism of AI-driven attacks. Emails, deepfake videos, and synthesized voices generated by AI have reached a level of quality where even family members can be deceived. While it was long assumed that advanced deepfakes in Japanese were difficult to construct, recent AI developments have completely eliminated this barrier.
However, many still operate under the cognitive bias of "this video looks too real to be fake," and this perception gap continues to widen the impact of these attacks.
Defensive Roadmap
How can organizations bridge these three gaps and prevent successful social engineering attacks?
STEP 1: Human Defense
The first line of defense is training personnel. While simulated exercises can significantly increase employee resilience, human error can never be completely eradicated. Therefore, human training must be combined with the technical defenses outlined in Step 2. Key initiatives include:
AI Simulation Training
Run simulations using realistic, AI-generated attacks. Measure click rates per employee and rotate department-specific scenarios (Finance, HR, R&D) to lower click rates. Share completion rates on a company-wide dashboard to drive behavioral change.
Deepfake Response Briefings
Conduct interactive sessions displaying two synthesized deepfake videos of a fake CFO, followed by short quizzes to evaluate and visualize employee understanding.
BEC Roleplaying
Use real-time voice modification tools to simulate emergency calls from a fake CEO requesting urgent wire transfers.
STEP 2: Technological Defense
Human training alone cannot entirely stop social engineering attacks. Therefore, organizations must build the following technical safeguards into their systems to solve the root issue.
Rather than relying on human vigilance, technical solutions should secure the entire cycle: detection, sanitization, containment, and forensic preservation.
Pre-transmission AI Inspections
Before emails are sent, AI scans the text and attachments. If the content deviates significantly from standard business communications, the transmission is quarantined and the sender is alerted, stopping data leaks and accidental sends before they occur.
Content Disarm and Reconstruction (CDR)
Incoming PDFs and Office documents are disassembled, malicious macros or code are stripped out, and secure versions are reconstructed instantly without impacting user workflow.
Biometric Verification for Meetings and Logins
Verify identities via 1–2 second biometric face and voice checks before allowing access to sensitive meetings or logins. This AI-driven step detects deepfake videos and synthesized voices. Meetings are automatically recorded and protected from tampering.
Behavioral Monitoring and Automated Isolation
Continuously log user and device activity. If malicious actions cross a specific threshold, the device is isolated from the network instantly, and alerts are dispatched to security teams within seconds.
Daily Vulnerability Assessments
Automate nightly scans of public-facing servers and ports. Close unnecessary services and vulnerable pages to minimize the external attack surface.
Automated Log and Memory Preservation
Upon detecting high-priority incidents, instantly copy relevant logs and volatile memory, apply tamper-proof digital signatures, and store them securely to support root-cause investigations and legal readiness.
Summary
The rapid evolution of generative AI has transformed social engineering from labor-intensive campaigns into a highly automated, low-cost, and precise business model. While technical solutions exist for all six threats outlined in this paper, attacks continue to succeed due to non-technical factors: human cognitive biases, slow organizational workflows, and delayed security investments.
In an era where attack ROI is incredibly high, defending against social engineering is no longer optional. Start protecting your organization from generative AI-driven threats today.
Related Services: Learn more about Yagura Awareness, our multi-channel security training and education platform designed for the generative AI era.
References (Selected)
World Economic Forum “Deepfake Heist Hits Engineering Giant” (2025‑02‑18)
FBI IC3 Report 2024, p.10
IBM Security “Threat Intelligence Index 2025”
Pindrop “Pulse: Real‑Time Deepfake Detection” (2025‑06‑12)
Additional industry reports from Tripwire, InceptionCyber, Unit 42, and F‑Secure
