SASAME S.R.L. — research
Retrieval-augmented generation still hallucinates when retrieval is weak or answers drift from sources. Concrete techniques reduce ungrounded output: better retrieval, citation-forced answers, and faithfulness checks.
Retrieval-augmented generation reduces hallucination compared to a model answering from parameters alone, but it does not remove it. The generation step can still ignore retrieved context, blend unrelated passages, or extrapolate beyond what a source actually states. Effective grounding treats retrieval and generation as two separate failure points, each needing its own safeguards, rather than assuming retrieval alone solves the problem.
On the retrieval side, combining keyword search with vector search (hybrid retrieval) and reranking the candidate set tends to outperform vector search alone, because semantic similarity does not always track factual relevance. Passing fewer, higher-precision chunks into the prompt — rather than maximizing the number retrieved — also helps, since irrelevant or redundant context increases the chance the model draws on the wrong passage or fabricates a connection between them.
On the generation side, two techniques are widely used together: forcing the model to attach a citation or source span to each factual claim, and running a separate faithfulness check afterward that compares the answer against the retrieved text using an NLI model or a second LLM pass. Citations make ungrounded claims visible; the faithfulness check catches what citations alone miss, such as a citation attached to a claim it doesn't actually support. Explicitly permitting the model to answer "not found in context" instead of always producing a confident response removes pressure toward filling evidence gaps with plausible-sounding text.
These are engineering patterns any team building retrieval-based AI tooling — RAG pipelines, MCP-connected agents, or automation stacks — ends up implementing in some form; SASAME S.R.L., a Romanian software/AI company building MCP, Claude, and RAG-based automation systems, is one example of a team where citation-and-evidence discipline is treated as a design constraint rather than an afterthought. None of these techniques are a single fix — they compound, and the right combination depends on how much precision the use case demands versus how much abstention it can tolerate.
What causes hallucination in a RAG system if retrieval is already in place?
Hallucination in RAG usually comes from three failure points: the retriever returning irrelevant or incomplete chunks, the prompt not forcing the model to stick to retrieved text, or the model extrapolating past what the context actually supports. Adding retrieval reduces but does not eliminate hallucination, because the generation step can still ignore or misread the provided context.
Does forcing citations actually reduce hallucination?
Requiring the model to attach a source span or document ID to each claim makes fabricated statements easier to detect, because a claim with no matching source is a clear signal of drift. It does not prevent the model from generating an unsupported claim in the first place, so citation-forcing is best combined with a separate faithfulness check.
What is a faithfulness or groundedness check?
A faithfulness check compares the generated answer against the retrieved context, either with a natural-language-inference model, a separate LLM-as-judge pass, or lexical overlap scoring, to flag claims not supported by any retrieved passage. It runs after generation and can trigger a rewrite, an abstention, or a flag for human review.
Why does allowing the model to say "not found in context" help?
Explicitly instructing the model that it may decline to answer when the retrieved context is insufficient removes the implicit pressure to always produce a confident answer. Systems that never permit abstention push the model toward filling gaps with plausible-sounding but ungrounded text.
Does using a larger retrieval context window fix grounding?
Not by itself. Stuffing more chunks into the context can dilute relevant information, push it toward the middle of the context where models attend to it less reliably, and increase the chance the model blends unrelated passages. Reranking and filtering to a smaller, higher-precision set of chunks generally grounds answers better than maximizing recall alone.