Why AI features give wrong answers, and how we stop it

An AI feature that crashes is bad. An AI feature that gives a wrong answer and sounds sure about it is worse.

When a feature crashes, everyone knows something is wrong. When it gives a confident wrong answer, nobody notices until it causes a problem.

A simple example

Say a customer asks your support chatbot about refunds. The bot says "Full refunds within 90 days, no questions asked."

Your real policy is 14 days.

The answer sounded friendly and sure, so the customer believed it. You only find out when they come back on day 60 and quote it back to you.

This is not a rare case. It's one of the most common problems with AI features that were added quickly.

Why it happens

Most of the time, it comes down to two things.

1. The AI answers from memory, not from your data.

Language models are trained on a huge amount of text from the internet. When you ask a question, they give the most likely answer based on that. If your business does something differently, the AI doesn't know unless you give it your information every time.

2. Nothing checks the answer before it goes out.

The AI writes an answer and it goes straight to the customer. There's no step in between that asks "is this actually true for this business?"

The rules we follow

When we add AI to a product, we stick to a few simple rules.

It only answers from your data

We give the AI your real information for each question, like your policies, your product details or your help docs. And we tell it to answer only from that.

If the answer isn't in your data, it says "I don't know" and points the person to your team. That's much better than a guess.

The answer comes back in a fixed format

Instead of letting the AI write a free paragraph, we ask it to return the answer in a set structure. For example: the answer, and which part of your data it came from.

This sounds small, but it means the app can check the answer before showing it.

If the answer doesn't match the source, it gets blocked

Before anything reaches a user, the app checks it against the source. If the AI mentions something that isn't there, the answer is thrown away and a safe reply is used instead.

A person reviews anything that matters

For things like refunds, prices or anything legal, a person on your team gets the final say. AI can draft it. A human sends it.

How we use this in our own tool

We use the same idea in Auditely, our website audit tool.

Auditely checks a website and then uses AI to write a short report in plain English. The AI only writes the words. The findings come from a list of problems and fixes we wrote ourselves.

And if the AI writes a number that isn't in the audit data, we throw that draft away and make a new one. A report with a made-up number is worse than no report at all.

Is it more work?

A little, yes. It's not the exciting part of building an AI feature.

But it's the difference between an AI feature people keep using and one they stop trusting after a week. Once someone catches your bot making something up, they stop believing it, even when it's right.

A short checklist

If you already have an AI feature, or you're planning one, ask:

• Does it answer from our own data, or from general knowledge?
• What happens when it doesn't know? Does it say so?
• Is anything checking the answer before a customer sees it?
• Who reviews answers about money, policies or legal things?

If some of these are "not sure", that's the place to start.

If you're adding AI to your product, or your agency's clients are asking for it, we're happy to help. Book a short call or send us a brief.