How AI chatbots are being manipulated โ and what you can do about it.
AI chatbots are being manipulated โ and they can't tell the difference between a real company and a fake one.
A 2025 experiment proved it: a researcher invented a completely fake company, posted a few made-up stories online, and within days, ChatGPT, Gemini and Copilot were confidently repeating the lies as verified facts. A BBC reporter did the same โ tricking AI into naming him the "hotdog eating champion of journalism" with nothing but a blog post.
This is not a glitch. It's a new reality of how AI gathers information.
| Data | What It Means |
|---|---|
| 89% of B2B buyers now use generative AI in their purchasing process (Forrester, 2024) | Your competitors are using AI. So are fraudsters. |
| Only 69% of AI-generated citations are real and correctly attributed | Nearly one in three are fabricated. AI confidently cites sources that don't exist. |
| Peer-reviewed study (Dec 2024) | ChatGPT makes a "significant number of errors" identifying business partners โ worse in smaller economies. The less information publicly available, the easier it is to poison. |
| Your Role | The Risk |
|---|---|
| Supply chain / procurement | You reach out to a "supplier" that doesn't actually exist โ or has no factory. |
| Joint venture / strategic partnership | A partner's lawsuits or regulatory issues are whitewashed from AI summaries. |
| Legal / compliance | Your AI-based screening misses red flags because the sources were poisoned. The red flags exist. AI just can't see them. |
Regulators are catching on. Under the EU CSDDD, US FCPA, UK Economic Crime Act, and Australia's 2024 AML amendments, relying on unverified AI data is not a legal defense.
If your due diligence relies on AI-generated information โ and that information was manipulated โ your company is exposed.
Don't let AI manipulation put your business at risk. Get government-verified due diligence from real experts.