The 30-second answer
A customer asks "do you ship to Brazil?" and the AI replies "yes, with 5-day delivery" — except you don't ship to Brazil at all. The AI didn't lie on purpose. It generated a reply that sounded like a reasonable shipping answer, because that's what a large language model does: it predicts plausible next text, not necessarily true next text.
In customer support, every hallucinated answer is a promise the company didn't make — and may have to honour, or apologise for, or get sued over.
Why it happens
A large language model isn't a database. It doesn't look up your shipping policy and report what it finds. It generates language one token at a time, predicting what plausibly comes next based on what came before. If your shipping policy isn't in the prompt, the model fills in something that sounds shipping-policy-shaped.
The same property that makes LLMs fluent makes them prone to hallucination: they always have an answer, even when they shouldn't.
Why RAG reduces but doesn't eliminate it
RAG (retrieval-augmented generation) is the standard fix. Before answering, the system searches your knowledge base for the relevant article, and the AI writes the reply using that article. Grounding the answer in real content cuts hallucination dramatically — when it works.
It still fails three ways:
- The right article doesn't exist. The bot finds nothing and either says "I don't know" (honest) or fills in from training data (hallucination). Most vendors let you tune which.
- Retrieval grabs the wrong article. The bot answers from a related-but-incorrect source. The reply is fluent and confident; it's also wrong.
- The model paraphrases too freely. The retrieved article says "14-day return window"; the bot replies "about two weeks." Close enough most days, wrong on the edges that matter for refunds.
The cost
A single hallucinated promise — "yes, we offer free returns" when you don't — can be a legal liability, a customer-trust event, or a refund the company has to honour to keep the customer. At consumer scale (thousands of conversations per day) the financial exposure compounds fast.
There's also reputation cost that doesn't show in any dashboard: a customer who finds out the AI lied to them tells other people. The next time they need to contact you, they go straight to a human and ignore your AI — which raises your escalation rate for reasons your dashboard can't explain.
Common examples in customer support
- Quoting a refund period, warranty term, or shipping window that doesn't match the actual policy.
- Confirming a feature exists in your product that doesn't ("yes, you can export to CSV" — except you can't).
- Telling the customer the issue "will be escalated to engineering" when there's no engineering ticket created.
- Inventing a discount code that doesn't exist.
- Making up a name of a colleague who'll "follow up by Friday."
How to detect and reduce it
- Spot-check with the right comparison. Take 100 AI responses, compare each answer to the source article it cited. The share where the answer isn't faithful to the source is your hallucination rate.
- Constrain action permissions. An AI that can't issue refunds can't promise refunds it has no authority for. Guardrails are the system that enforces this.
- Tune the "I don't know" threshold. Most vendors let the bot say "I'm not sure, let me get someone" instead of guessing. Default settings often favour confidence over caution; consider the trade-off explicitly.
- Use models that cite their sources inline. If the bot's reply includes "per our returns policy article" with a link, hallucination is harder to hide.
- Watch customer complaints. Customers report hallucinations through the front door. "Your bot told me X but you don't actually do X" tickets are the canary.