Fraud Prevention

AI-Generated Fraud Is Surging. Here's What Platforms Need to Know.

Fake storefronts, phishing campaigns, and scam advertisements are scaling faster than traditional detection can handle. Platforms that process payments, serve ads, or host user-generated content are already facing this threat and seeing an increase in chargebacks, more frustrated users, and damage to brand reputation.

Fake storefronts, phishing campaigns, and scam advertisements are scaling faster than traditional detection can handle. Platforms that process payments, serve ads, or host user-generated content are already facing this threat and seeing an increase in chargebacks, more frustrated users, and damage to brand reputation. Generative AI has fundamentally changed the economics of fraud, causing the issue to accelerate.


The challenges AI brings

Generative AI lowers the barrier to entry for fraud in three ways:



  1. Produces convincing assets. Fake product images, professional-looking storefronts, and polished ad copy that used to require real effort can now be generated in minutes.
  2. Enables scale. What once required a team of scammers can now be automated. A single actor can spin up hundreds of fake domains with unique content.
  3. Increases credibility. AI-generated reviews, realistic product descriptions, and consistent branding make fraudulent operations harder for consumers to spot. The usual red flags (broken English, stock photos, sparse details) are disappearing.

What this looks like in practice

The shift from theoretical to operational is already here:



Fake storefronts at scale. Researchers have documented sharp spikes in scam e-commerce sites built using generative AI. One study found fraudulent social ads increased by roughly 179% year-over-year (Redpoint).



Mass domain impersonation. A recent campaign targeting TikTok Shop users deployed over 15,000 fake domains mimicking the platform. These sites distributed phishing pages and malware to steal credentials and cryptocurrency. Fraudsters favor top-level domains like ".shop", ".top", and ".icu" because they look legitimate at a glance.



Seasonal surges. Consumer protection agencies are warning of intensifying AI-enabled scams around major shopping events. The tactics span every channel: email, SMS, social media, fake ads, and newly registered websites. AI-cloned voices and texts add another layer of deception.


The impact on platforms

For platforms and service providers, this creates several problems:



  • Increased chargebacks. Fraud-driven disputes spike chargeback rates, triggering fees, penalties, and potential scheme violations.
  • Trust erosion. Consumers blame the platform where they encountered the fraud, regardless of where the actual transaction occurred.
  • Polluted ad inventory. Fraudulent ads degrade inventory quality and create brand safety exposure for legitimate advertisers.
  • Detection gaps. Rule-based systems can't keep pace with AI-generated content that rotates domains, refreshes creative, and evolves tactics faster than playbooks can update.

What platforms can't see today

Fraudsters rotate domains every few hours, rewrite content using AI, and alter visual layouts dynamically. A scam site flagged on Monday looks completely different by Wednesday. Manual review and traditional moderation systems can't keep pace with threats that evolve faster than analysts can update their playbooks.



Fixed rules break against adaptive threats. The only way to address fraud at AI scale is using AI systems to detect it.


How SafetyKit fights back

SafetyKit's merchant investigation tool combines content moderation, web scraping agents, and AI image identification to generate a full report on a shop's legitimacy. Simply put in URLs of suspected fraudulent merchants and get back a verdict with a full report from deep investigation. Payment platforms and ad networks can verify merchants before they cause chargebacks or trigger fines for allowing fraudulent shops on their platforms.



Our tools also provides continuous monitoring across your network, verifying MCC classification, checking for transaction laundering, and matching against known bad actor databases. It catches merchant drift (when a business starts processing for undisclosed products or drifts into prohibited categories) as it happens, not after the damage is done.



Together, these capabilities address the core challenges platforms face:



  • Domain and storefront analysis. Automated scanning flags suspicious domains based on registration data, naming patterns, and visual similarity to legitimate brands. A newly registered ".shop" domain impersonating a major retailer gets caught before it reaches consumers.
  • Content signals. AI evaluates ad creative, landing pages, and product listings for patterns associated with fraud: too-good-to-be-true claims, mismatched metadata, or visual assets that appear across multiple suspicious sites.
  • Behavioral anomalies. Unusual patterns often reveal fraud: sudden traffic spikes to new merchants, high click volume with low fulfillment rates, or clusters of negative reviews. These signals trigger investigation before the damage spreads.
  • Cross-platform intelligence. Fraudsters reuse assets, domains, and tactics across platforms. A bad actor flagged on one network can be identified when they appear elsewhere, even with slight variations.

How it works

Here's what a merchant investigation looks like in practice:



1. Set your prompt. In SafetyKit's UI, define what you want to investigate (fraud detection, media investigation, legal audit, etc.). SafetyKit will create a system for your team that allows you to plug in a URL and get immediate results.



Define your investigation parameters once. SafetyKit saves your configuration so your team can run investigations with a single click.

Edit Investigation Type form showing configuration options for fraud detection
Configure your investigation once: name it, define what SafetyKit should look for, and set your input fields. This template can be reused across your team.

2. Insert the URL(s) you want to investigate. SafetyKit handles the investigation automatically.



Enter any merchant URL to trigger an investigation. SafetyKit handles the rest.

Create Investigation modal with URL input field
Enter any merchant URL to start an investigation. Here we're investigating mabelanddaisy.com.

3. Get the full report. SafetyKit returns a thorough investigation based on all publicly available information: website content, third-party reviews, social media presence, AI-generated content detection, and verifiable business information.

Investigation Details panel showing verdict: Is the shop legitimate? No
SafetyKit returns a clear verdict: "Is the shop legitimate? No." The report includes 6 pieces of supporting evidence.

What SafetyKit surfaces

In a recent investigation of a suspicious storefront, SafetyKit identified:



  • Reputation signals: 517 Trustpilot reviews with a 1.0/5 rating, with customers reporting products that don't match photos, items shipping from China despite UK branding claims, and refused refunds
  • Risk database flags: ScamAdviser trust score of 1/100, Gridinsoft flagging as "very likely unsafe," ScamDoc score of 25% citing recent domain registration, hidden owner identity, and low traffic
  • Registration red flags: Domain registered July 2025 via Canadian privacy proxy, legal entity listed as a Hong Kong-based supply chain company
  • AI-generated content: BBC Bristol reporting AI-generated founder personas and misleading imagery, with identical product photos appearing across multiple scam networks
  • Pattern matching: Website structure matching templated scam networks with nearly identical layouts

Each of these signals comes from a different source: review platforms, risk databases, domain registrars, news outlets, and social media. SafetyKit's investigation pulls them together into a single report, so your team doesn't have to manually cross-reference across a dozen tabs.


Here's what that looks like in the platform:

The investigation starts with a verdict, then backs it up with evidence. Below, you can see how SafetyKit breaks down each fraud signal with source links and supporting details.

Signs of Fraudulent Activity panel showing 6 distinct fraud signals
The report surfaces 6 distinct fraud signals: negative Trustpilot reviews, risk database flags, security blog warnings, suspicious domain registration, AI-generated imagery, and social media scam reports.

Clicking into any signal reveals the underlying evidence. Here's what SafetyKit found when it pulled review data:

Expanded Trustpilot review evidence showing 517 reviews with 1.0/5 rating
SafetyKit pulls review data directly from Trustpilot: 517 reviews, 1.0/5 rating, with consistent complaints about product quality, China shipping despite UK branding, and refused refunds.

SafetyKit also cross-references business registration details against the merchant's claims. In this case, the mismatch was clear:

Contact information validity section showing failed verification and Instagram evidence
Contact verification failed: the site's contact page returns an error, and the legal terms reveal the business is actually DAYONE SUPPLY CHAIN LIMITED, registered in Hong Kong. SafetyKit surfaces social media evidence: an Instagram user warns others about fake addresses and deceptive return policies.
Physical address found section showing Terms and Conditions cross-reference
Business registration cross-check: the Terms and Conditions confirm the company is registered in Hong Kong, contradicting the site's "cosy UK boutique" branding.

This is the difference between waiting for chargebacks and catching fraud at onboarding.


The path forward

AI-enabled fraud isn't going away. Platforms need detection that catches fraud and identifies shifting tactics as fast as they appear.



The platforms that maintain consumer trust will be those that invest in proactive detection: AI-powered moderation that keeps pace with AI-generated threats, merchant investigation before harm occurs, and defenses that evolve as fast as adversaries do.



Ready to see what SafetyKit can surface about the merchants in your network? Get a demo.

See how SafetyKit can help with risk and fraud

Get a personalized walkthrough based on “AI-Generated Fraud Is Surging. Here's What Platforms Need to Know.”.

GET A DEMO
Stylised collage of people, shoes and city scenes reinforcing the message of platform safety and user trust
GET A DEMO
Collage of portraits and abstract shapes beneath the 'Protect your platform' call‑to‑action for trust and compliance
SafetyKit | AI-Generated Fraud Is Surging. Here's What Platforms Need to Know.