AI-generated phishing and deepfakes: The social engineering threat that arrived in 2023

For years, the telltale signs of a phishing email were easy to spot: awkward grammar, broken English, a sense that something was slightly off. In 2023, that changed. Generative AI tools made it possible for attackers to write flawless, convincing phishing emails in any language, tailored to any target, in seconds. Voice cloning could mimic a manager's voice from a few seconds of audio. Deepfakes put a familiar face behind a fraudulent request. The human factor, already the weakest link in security, suddenly faced an adversary that could impersonate almost anyone. Here is what changed and what security awareness training now has to account for.

The end of the grammar tell

Historically, poor grammar and awkward phrasing were reliable red flags. Attackers operating in a second language left errors that sharp employees could catch. Generative AI eliminated that weakness entirely. Attackers could now produce polished, native-sounding messages that matched a target's industry, tone, and even writing style. The old advice to "look for spelling errors" became nearly useless. Employees needed new signals, because the linguistic tells they had relied on for a decade were gone.

Spear phishing at scale

Targeted attacks used to require manual effort and research, which limited how many attackers could launch them. AI changed the economics. Attackers could now generate personalized messages at scale, referencing real names, real projects, and real relationships scraped from public sources. What was once the work of a skilled operator became something a low-skill attacker could automate. The result was a surge in convincing, personalized phishing that was far harder to distinguish from legitimate communication.

Voice cloning and vishing

One of the most unsettling developments of 2023 was voice cloning. With just a few seconds of someone's voice, often taken from a public video or voicemail greeting, attackers could generate convincing audio of that person saying almost anything. This enabled a new wave of vishing, or voice phishing, where employees received calls that sounded exactly like their boss or a colleague asking them to transfer money, share credentials, or approve a payment. The familiar voice on the other end was the strongest possible social proof, and it was now for sale.

Deepfakes in business compromise

Deepfake video raised the stakes further. Attackers began using synthetic video in business email compromise schemes, impersonating executives on video calls or in recorded messages. In several documented cases, employees transferred significant funds after seeing what they believed was a live or recorded video of a trusted leader authorizing the transaction. The ability to fake both a face and a voice undermined the verification methods that organizations had long relied on.

What training must now cover

Traditional awareness training was not designed for this threat. Employees need to understand that: a well-written message is no longer proof of legitimacy; a familiar voice on the phone does not guarantee the speaker is who they sound like; urgent payment or credential requests must be verified through a separate, pre-agreed channel, regardless of how convincing the request appears; and AI can replicate tone, language, voice, and even appearance. The core lesson is verification by process, not by appearance.

Verification becomes the control

The most reliable defense against AI-powered social engineering is procedural verification. Organizations need clear, enforced rules: payments above a threshold require dual approval; requests to change bank details or credentials must be confirmed through a second, out-of-band channel; and no amount of urgency overrides the verification step. Training that ingrains these habits, combined with the understanding that any communication can now be convincingly faked, is the most practical protection organizations have.