Why Your Chatbot Misses Answers That Are Right There in Its Knowledge Base

The complaint every chatbot operator recognizes

A visitor asks a question. The answer sits in your knowledge base, word for word. The bot replies "I don't have that information", or worse, invents something. You test it yourself, see the failure, and draw the obvious conclusion: the AI model is too weak. So you switch to a bigger model. The answers get better, the bot gets slower and more expensive, and the actual bug is still there.

We see this pattern in support tickets regularly. One customer put it precisely: "All the cheaper models are insufficient, they always miss information that is included in the knowledge base. All the more expensive models are too slow." That sentence describes a retrieval problem, not a model problem.

Your knowledge base is multilingual, even if you never decided that

A typical setup: the company website is in English because that is what international customers expect. The FAQ documents, price lists and internal notes the owner uploads are in Greek, German or Dutch, because that is the language the team works in. Nobody calls this a "multilingual knowledge base". It just happens.

Now a visitor asks a question in Greek. A chatbot does not read your entire knowledge base for every question. It runs a similarity search: the question and every text snippet are converted to vectors (embeddings), and the top-scoring snippets, typically around 20, are handed to the AI model as context. Whatever does not make that cut is invisible to the model. Not "less important". Invisible.

And here is the catch: embedding models systematically rank same-language matches far above cross-language ones. Research on multilingual retrieval measures a drop of 30 to 50 points in Hits@20 when query and document are in different languages. A Greek question scores high against Greek website chunks, even loosely related ones, and low against the one English document that actually contains the answer.

A real example, with ranks

From a production case (shared with permission, details anonymized): a student-housing provider runs a knowledge base with a Greek website crawl and English policy documents. The English document stating the exact security deposit exists. A visitor asks in Greek: "How much deposit is required?"

• The correct English deposit document ranked at position 46 in the similarity search. Only the top 20 snippets reach the model. The answer never arrived.
• The same question asked in English put that document at position 1 with a similarity score of 0.786.
• With cross-language retrieval enabled, the Greek question now surfaces it at position 4. Same knowledge base, same model, correct answer.

This also explains the "expensive models work better" illusion. A partially relevant English FAQ snippet had crept into the top 20 at position 16. A strong model can sometimes reconstruct the right answer from such scraps. A budget model takes the top-ranked Greek chunks at face value and answers about apartment availability instead of deposits. The strong model was not smarter about your business. It was better at guessing around a broken search result.

Why "just buy the bigger model" is the wrong fix

Upgrading the model treats the symptom and bills you for it twice: bigger models cost more per message and add seconds of latency. Worse, it is unreliable. The model can only work with the snippets it receives. If the answer sits at position 46, no amount of intelligence recovers it. You are paying premium rates for educated guessing.

What actually fixes it: search in every language your knowledge has

The method is established in research and in enterprise systems: translate the incoming question into the other main languages of the knowledge base, run the searches in parallel, and merge the results by best score. Each part of your knowledge gets searched in its own language, where embeddings are strong. The visitor notices nothing; the bot still answers in their language.

The naive version of this is slow: an extra translation step plus a second search before every answer. The engineering that makes it production-ready is mostly about latency and cost discipline:

• Detect the knowledge base languages once, not per question. We sample the stored content, let a small model identify the main languages, and persist the result. It only re-runs when your knowledge actually changes.
• Translate with a fast, small model, not the big chat model. The translation is a search query, not literature. In our stack this takes roughly 300 to 600 milliseconds and costs about a hundredth of a cent.
• Run the variant search in parallel with the main search, so it hides inside time the bot is already spending. In our production measurements the added wait is usually 0 milliseconds, capped at 1.5 seconds in the worst case.
• Cache translations. Visitor questions repeat heavily. A repeated question skips the translation entirely.

After shipping this, we re-ran the failing Greek deposit question against the cheapest model tier. It answered correctly, including the fine print about how the deposit is returned. The customer who reported the problem can now downgrade to a faster, cheaper model instead of upgrading to a slower, pricier one.

How this runs in WebChatAgent

Cross-language retrieval is live for every chatbot on our platform, on every plan, with nothing to configure. The bot detects the languages of your knowledge base automatically, keeps that knowledge current when your content changes, and searches all of them for every question. It combines with the rest of the retrieval pipeline: hybrid keyword plus vector search, neighbour-window stitching, and a per-document priority you can set yourself.

If you run a knowledge base that mixes languages, and statistically you almost certainly do, this is the difference between "the bot ignores my documents" and correct answers on the affordable models.

Test your own bot in two minutes

1. Pick a fact that exists in only one language of your knowledge base, for example an English-only policy document.
2. Ask for it in the other language your visitors use.
3. If the bot claims it has no information: that is this exact problem, not a model problem. Do not pay for a bigger model to work around it.

On WebChatAgent you can run this as a structured knowledge test from the dashboard, which grades the answers and shows you which snippets the bot retrieved.