The mundane (But just as dangerous) side of AI-Generated Fraud

Unless you’ve been asleep for the last few years, you already know that AI is everywhere. A friend is using it to plan a weekend in Vegas, your aunt is using it to name her cat, and maybe your coworker is using it to write his emails (and not fooling anyone). These are all mundane, legitimate tasks, but there’s also a darker side, because AI is writing ransom notes, leak announcements, and more for fraudsters and cybercriminals. 

Our analysts recently pulled the below screenshot of a data-release notice in progress, complete with tweaks and follow-up prompts. The finished message checks all the “AI-generated” boxes: clean structure, confident tone, compact paragraphs that clearly explain the situation. In the past, these types of notes would have been riddled with the kind of spelling and grammatical mistakes that any third grader could spot. Now they read like they came straight out of the marketing department.

The boring truth about AI cybercrime

Criminals are using AI for a lot more than writing emails, such as deepfakes for account takeovers, which often show up in headlines. But here’s the part that doesn’t make headlines but is just as problematic: They’re not using some super secret AI model straight out of a Bond villain’s playbook. They’re using the same tools we all are.

That’s because the tech doesn’t care who’s typing. The same capabilities that help a small business write a polished email newsletter also helps a fraudster write a clean, polished phishing email. The prompt “make this email sound professional and persuasive” will be just as successful, whether you’re writing about your company’s new conference room policy or creating the shakedown message that follows a ransomware attack. 

Why this matters for fraud teams

One of the things we’ve all been taught to look for in potential spam or fraud is terrible spelling and weird phrasing. But that’s aging out because when the polishing tools are free, that polish stops signaling safety or legitimacy. 

Because fluency no longer equals trust, the tells are moving from how the message is written to what it’s asking you to do. Things like urgency, unusual payment requests, and pressure to work outside established processes are what customers need to be looking for. 

The lighter lift to create an email is also a challenge because fraudsters can write and send more messages in less time. That means one thing you should be on the lookout for is more volume: more attempts, more variations, more tailoring, and a lot more headaches.

The good news is that the fundamentals are still fundamental. Things like MFA verification (with some caveats), slowing down when something feels urgent, and treating unexpected requests as suspicious no matter how clean they look. Those still work and awareness is still the best defenses against fraud. The more light we shine on what criminals are doing, the harder it is for them to succeed.

We’ll keep watching the dark corners of the web so you don’t have to. Your job is to teach customers and employees to stay skeptical, verify twice, and don’t automatically give a well-formatted email the trust it hasn’t necessarily earned.