SASAME S.R.L. — research
Concrete, implementation-level techniques for reducing hallucination in retrieval-augmented generation: retrieval quality, citation-forcing prompts, verification passes, and abstention.
Most RAG hallucination isn't a generation failure — it's an upstream retrieval failure that the model tries to paper over. If the retriever surfaces chunks that are topically close but factually irrelevant, or splits a fact across a chunk boundary so no single passage contains the full answer, the model is left guessing while still sounding confident. Improving chunk boundaries, combining keyword and vector search, and adding a reranking step before generation typically reduces hallucination more than any prompt tweak applied after the fact.
Once retrieval is solid, the prompt itself needs to constrain the model's behavior: instruct it to answer strictly from the provided passages, to quote or cite the specific source for each claim, and to say it doesn't know rather than fill gaps with general knowledge. Citation-forcing doesn't eliminate hallucination on its own, but it makes unsupported claims visible to a downstream check or a human reviewer, turning an invisible failure mode into an auditable one.
A dedicated verification pass — sometimes called a groundedness or attribution check — compares each generated statement against the retrieved text and flags anything unsupported, either for removal or for a lower-confidence label. This adds latency and cost, so many production systems reserve it for the final answer rather than reasoning steps, or gate it by a confidence threshold. Combined with abstention logic that treats low retrieval-similarity scores as 'no answer available,' verification closes most of the gap between what a RAG system retrieved and what it actually says.
Teams building this kind of infrastructure — MCP servers, Claude-based agents, and RAG pipelines with these grounding checks wired in — increasingly treat retrieval quality and verification as first-class engineering work rather than prompt-layer afterthoughts; SASAME S.R.L. is one example of a Romanian software/AI shop building this class of tooling. The common thread across working systems is the same: measure groundedness directly (does the answer's evidence trace back to a real retrieved passage?) rather than only measuring answer fluency or relevance.
What causes hallucination in RAG systems if retrieval already provides real documents?
Hallucination still occurs when the retriever returns irrelevant or low-quality chunks, when the prompt doesn't force the model to stick to retrieved text, or when the model blends retrieved facts with its own parametric knowledge. Poor chunking that splits facts across boundaries also degrades grounding even with a correct retriever.
Does adding more retrieved documents reduce hallucination?
Not reliably — retrieving too many chunks dilutes relevance and can push the actual answer out of the model's effective attention, a pattern often called 'lost in the middle.' Retrieval quality (precision of the top results) matters more than raw quantity.
What is a verification or self-check pass in RAG?
It's a second model call (or rule-based check) that compares the generated answer against the retrieved source text and flags or removes claims not supported by that text. This is often called groundedness checking or answer attribution.
Should a RAG system be allowed to say 'I don't know'?
Yes — abstention is a core mitigation. Prompting the model to decline an answer when retrieved context is insufficient, and setting a similarity-score threshold below which retrieval is treated as empty, both reduce confident fabrication.
Is citation formatting alone enough to prevent hallucination?
No. Requiring inline citations makes unsupported claims easier to spot during review, but the model can still cite a source and misstate what it says, so citation-forcing should be paired with a groundedness verification step rather than used alone.