Grounding, citations, and letting a RAG system say I don't know

Three controls, three different jobs:

Grounding, more precisely. Two things are meant by the word, and both matter:

Grounding as an architecture — the fact that the model's answer is conditioned on retrieved documents at all. This is what RAG is. Compared with a model answering from weights alone (01-02), the answer is now tied to a corpus you control, can update, and can audit.

Grounding as an instruction — the explicit directive in the prompt: use only the provided sources; do not use prior knowledge; if the sources do not contain the answer, say so. Retrieval without this instruction is grounding-in-architecture without grounding-in-behaviour, and the model will freely blend context with parametric knowledge.

Both are needed. Retrieval alone gives the model access to the right facts; the instruction gives it the rule about which facts to use.

Citation, more precisely. A citation is only useful if it is verifiable, and verifiability has three requirements: the id must be stable (the same chunk keeps the same id across requests), it must be resolvable (a reader can get from the id to the source text), and it must be rendered into the context the model reads (07-08) — a model cannot cite an id it was never shown. A citation that names "the documentation" or "source 2" with no mapping is decoration.

Declining, more precisely. This is the control teams most often omit, because it looks like a capability regression. It is not. A system that answers every question has, by construction, no way to represent "I could not find this", so every retrieval failure becomes a fabrication. Designing the decline path means choosing a threshold (07-07), assembling a no-source context (07-08), instructing the behaviour, and — critically — scoring a correct decline as a success in evaluation (09-07). A decline counted as a failure trains your team to remove the control.

L1 — The intuition you can carry into an exam

Tell the model to use only what you gave it. Make it cite what it used. Let it say it does not know. Then check the citations.

Three facts to carry:

• Grounding reduces hallucination; citation makes it detectable; declining prevents a retrieval miss from becoming an invention.
• RAG does not eliminate hallucination. It is a mitigation with provenance, not a cure.
• A model can only cite ids that were rendered into its context.

L2 — The prompt structure that implements all three

The three controls are implemented in one prompt shape. Every element below does a specific job:

[SYSTEM]
Answer the question using ONLY the numbered sources below.
- Attach the source id in brackets after every factual claim.
- If the sources do not contain the answer, reply exactly:
  "The available sources do not answer this."
- If two sources conflict, say so and cite both, preferring the
  most recent effective date.
- Do not use knowledge outside the provided sources.

[SOURCE 1 · doc:ops-118#c14 · effective 2026-05-02 · status: current · official]
after replacing a CA, clients must be restarted to reload the trust store …

[SOURCE 2 · doc:err-009#c01 · effective 2026-01-20 · status: current · official]
ERR_CERT_AUTHORITY_INVALID: the presented chain terminates in an untrusted root …

[QUESTION]
customer rotated their CA and now gets ERR_CERT_AUTHORITY_INVALID — expected?

The exact-decline-string detail is worth dwelling on. A model that declines in free prose — "I'm not sure I can answer that based on what I have here" — is behaving correctly and is unmeasurable. A fixed string is detectable by a regex, which means you can count declines, alert on a decline-rate spike (a strong signal of a retrieval regression), and score declines properly in evaluation. Make the decline machine-readable and it becomes a monitoring signal instead of a mystery (12-14).

L3 — Citation validation, and the limits of instruction

Instructions are probabilistic. Three failure modes survive a well-written grounding prompt, and each needs a mechanism rather than more instruction:

Unsupported claims. The model asserts something no source states. Often it is true — pulled from pretraining — which makes it harder to catch and no less dangerous, because the next one will be false.

Citation drift. The claim is supported by source 2 but cited to source 1. The reader who checks source 1 finds nothing and loses trust; the reader who does not check is misled about provenance.

Fabricated citations. The model cites doc:ops-118#c99, an id that does not exist. This is the most alarming case and the easiest to catch.

The mechanisms, in increasing order of strength:

The quote requirement is the highest value-per-effort of these, and it is under-used. If the model must output the supporting sentence verbatim, you can verify by exact substring match against the cited chunk — no judge model, no ambiguity, no cost. A claim whose quote does not appear in its chunk is provably unsupported. It makes answers longer and slightly more stilted, which is a real trade, and for high-stakes domains it is obviously worth it.

The honest ceiling on all of this. Grounding is an instruction to a probabilistic system. Citation validation catches specific classes of error, not all of them. A model can produce a claim that is unsupported but entailed by the sources — an inference rather than a quote — and whether that counts as a hallucination depends on your requirements. There is no configuration that reduces fabrication to zero, and a system design that assumes there is will be wrong at the worst possible moment. What you can achieve is: fewer errors, detectable errors, and an honest non-answer instead of a confident wrong one. That is a large improvement and it is not a cure.

Where this sits in NVIDIA's framing. Grounding via RAG and citation of sources are named hallucination controls, alongside constrained decoding, guardrails, and human review [NVIDIA-DOC]. NeMo Guardrails keeps LLM applications accurate, appropriate, on-topic, and secure through topical, safety, and security rails [NVIDIA-DOC] — it is a complement to grounding, operating on the input and output rather than on the evidence. Citation also serves the Transparency pillar of NVIDIA's four trustworthy-AI principles: explaining in non-technical language how a system reached its output [NVIDIA-DOC]. A cited answer is self-explaining in exactly that sense. 13-01 covers the pillars and 13-02 covers Guardrails.

Four readings the exam probes.

Grounding and citation are not alternatives; they are complements with different jobs. Grounding lowers the error rate. Citation makes the residual errors findable. Shipping one without the other leaves a real gap: grounding alone gives you an undetectable residual, citation alone gives you a detectable flood.

Fine-tuning cannot substitute for either. A fine-tuned model has no sources to cite, cannot be updated by changing documents, and cannot be filtered per user (13-05 on why weights cannot forget). The confusable pair RAG-versus-fine-tuning is on the study guide's explicit list; 11-08 owns the decision rule. The one-line version for this lesson: if you need provenance, you need retrieval, because weights do not have footnotes.

Guardrails are a different layer. NeMo Guardrails keeps an application on-topic, appropriate, and secure [NVIDIA-DOC]. It is not a grounding mechanism and it does not verify that a claim came from a source. It is genuinely valuable and it sits outside the evidence path, so a guardrail on an ungrounded system is a policy filter over invented content.

Constrained decoding is about form, not truth. Forcing valid JSON (05-05) guarantees a parseable answer, not a supported one. A schema-valid hallucination is still a hallucination.

This is a constructed illustrative example. The generated answers are written to exhibit specific failure modes; they are not outputs sampled from a particular model.

The assembled context, produced by the pipeline from 07-09:

[SOURCE 1 · doc:ops-118#c14 · effective 2026-05-02 · current · official]
After replacing a certificate authority, clients must be restarted so they reload
the trust store. Until restart, clients continue to validate against the previous
trust anchors.

[SOURCE 2 · doc:err-009#c01 · effective 2026-01-20 · current · official]
ERR_CERT_AUTHORITY_INVALID indicates the presented certificate chain terminates in
a root that the client does not trust.

[QUESTION]
A customer on 4.2 rotated their CA and is now seeing ERR_CERT_AUTHORITY_INVALID.
Is this expected, and what is the fix?

Note what the context does not contain: any statement about how long a restart takes, any mention of a --reload-trust flag, and any version-specific behaviour for 4.2.

Generation A — no grounding instruction, no citations

Yes, this is a common issue. After rotating a CA you need to restart the client so
it picks up the new trust store. In 4.2 you can also use the --reload-trust flag to
avoid a full restart, which takes effect within about 30 seconds.

Audit it claim by claim:

Two of four claims are inventions, one of which names a command-line flag that may not exist. And nothing in the output signals which claims are which. A support engineer reads a fluent, specific, actionable answer and passes the flag to a customer.

This is what an ungrounded RAG generation looks like: correct in outline, fabricated in the specifics, and indistinguishable from a good answer.

Generation B — grounding instruction, no citations

Yes, this is expected. Replacing a certificate authority does not take effect on a
client until it is restarted, because the client continues validating against the
previous trust anchors until then. The error itself means the presented chain
terminates in a root the client does not trust. The fix is to restart the client.

Every claim is now supported. The grounding instruction worked. But you cannot tell that by reading it. The output is structurally identical to Generation A — fluent prose with no provenance — so the reader has no way to distinguish a grounded answer from an ungrounded one. Verifying it requires manually diffing the answer against the context, which nobody does at volume.

Grounding without citation is unverifiable compliance. It is better than A in expectation and identical to A in auditability.

Generation C — grounding plus citations

Yes, this is expected. Replacing a certificate authority does not take effect on a
client until the client is restarted, because until then it continues validating
against the previous trust anchors [SOURCE 1 · doc:ops-118#c14]. The error itself
indicates the presented certificate chain terminates in a root the client does not
trust [SOURCE 2 · doc:err-009#c01], which is what happens while the old trust store
is still loaded. The fix is to restart the client [SOURCE 1 · doc:ops-118#c14].

Now run the validation mechanisms:

Id existence check:   doc:ops-118#c14 ✓ in context   doc:err-009#c01 ✓ in context
Term-overlap check:   "restarted" + "trust store"  → present in ops-118#c14   ✓
                      "chain" + "root" + "trust"   → present in err-009#c01   ✓
Unsupported claims:   none — every sentence carries a citation
Fabricated flags:     none

The answer is now checkable by a machine and by a reader. The support engineer can click through to the source. The validator can assert that every cited id existed in the context. A monitoring job can alert if the unsupported-claim rate rises.

Compare the three:

Generations B and C have identical content and completely different operational value. That difference is what citation buys, and it is why NVIDIA's pipeline description ends with "cites sources" rather than "answers" [NVIDIA-DOC].

Generation D — the decline case

Now a different question against the same context:

"What is the maximum certificate chain depth the client supports?"

Retrieval returns the same two chunks — nothing in the corpus addresses chain depth. Reranker scores are 0.24 and 0.19, both below the 0.5 floor (07-07).

Without a decline path (k=3 assembled regardless of score):

The client supports a maximum certificate chain depth of 10 by default, which can
be adjusted in the TLS configuration.

Entirely fabricated, entirely plausible, entirely unsourced. A retrieval miss became a confident specification.

With a decline path (floor applied, no-source context assembled, exact decline string instructed):

The available sources do not answer this.

That output is correct, honest, machine-detectable, and — this is the part that matters operationally — it is a signal. A rising decline rate tells you either that users are asking about things the corpus does not cover (a curation gap, 08-01) or that retrieval has regressed (07-10). An invented answer tells you nothing.

Scoring these two in evaluation. If your eval set counts "produced an answer" as success, the fabrication scores 1 and the decline scores 0, and your metrics actively reward the dangerous behaviour. The correct labelling is: for an unanswerable question, the decline is the correct answer (09-07). This is the point where a metric decision becomes a safety decision.

What the four generations prove

All three controls are needed, and each covers a hole the others leave. Grounding fixes A→B. Citation fixes B→C. The decline path fixes D. Any two of the three leaves a live failure mode.

Two rules that generalise across the table:

The decline path is required wherever a wrong answer costs more than a non-answer. That is most enterprise systems. The exceptions are genuinely low-stakes or creative uses.

Rigour scales with what happens after the answer. An answer a human reads and evaluates needs less than an answer that triggers a tool call, a configuration change, or a customer communication. The agentic row is the strictest for exactly that reason: a fabricated parameter is not a wrong sentence, it is a wrong action.

And a caution about over-tightening. Grounding rigour costs coverage. A strict instruction plus a high decline threshold will decline questions the corpus does answer, because the passage was mid-context or the reranker scored it at 0.48. That is a real cost paid in usefulness, and the balance point belongs on your evaluation set (09-07), not in a default. A system that declines 40% of answerable questions has traded one failure mode for another.

This lesson serves both the RAG-building objectives and the Trustworthy AI domain, which makes it high-yield:

• 1.3 / 4.2 — Build LLM use cases such as RAG, chatbots, and summarizers [OFFICIAL]. NVIDIA's own pipeline description ends with the LLM synthesising and citing sources [NVIDIA-DOC] — citation is part of the stated architecture.
• 1.9 — Use prompt engineering principles to create prompts to achieve desired results [OFFICIAL]. The grounding instruction, the citation requirement, and the decline instruction are all prompt engineering, and prompt engineering is a Tier-1 reported topic [FIELD].
• 5.1 — Describe the ethical principles of trustworthy AI [OFFICIAL]. Citation directly serves the Transparency pillar: explaining in non-technical language how a system reached its output [NVIDIA-DOC].
• 5.3 — Describe how to use NVIDIA and other technologies to improve AI trustworthiness [OFFICIAL]. Grounding via RAG and citation of sources are named hallucination controls, alongside NeMo Guardrails, constrained decoding, and human review [NVIDIA-DOC].
• Derived Experimentation scope — hallucination causes and the mitigation ladder are explicitly in scope; 09-12 covers types and causes and 09-07 covers faithfulness metrics.

The hallucination mitigation ladder is a named drill emphasis, and it is worth memorising in order because it is the shape of several likely questions: grounding via RAG → citation → constrained decoding → guardrails → human review. Note the ordering logic: earlier rungs reduce the error rate, later rungs catch what got through.

Exam depth is general-level [FIELD]. Know what grounding and citation do and do not achieve, know that RAG reduces rather than eliminates hallucination, and know the mitigation ladder. Do not expect prompt-template minutiae.

Question phrasings you should recognise:

Distractor families:

• "RAG eliminates hallucination." The single most common over-claim, and a keyed-wrong option in many banks.
• Citation credited with preventing hallucination. It detects; it does not prevent.
• "Fine-tune the model to stop hallucinating." Does not add provenance and does not reliably fix fabrication (11-02).
• Guardrails offered as a grounding mechanism. NeMo Guardrails governs topic, appropriateness, and security [NVIDIA-DOC]; it does not verify that a claim came from a source (13-02).
• Constrained decoding offered as a truth control. It constrains form (05-05), not support.
• "Lower the temperature to stop hallucination." Reduces variation, not fabrication. A greedy decode (04-05) produces the model's most likely answer, which can be confidently wrong.
• "Increase k so the answer is definitely in there." Worsens assembly and raises contradiction risk (07-08).
• A decline treated as a system failure. It is the correct behaviour for an unanswerable question.

Mistake 9 is the one that quietly destroys the whole control. A team adds a decline path, the answer-rate metric drops, someone reports that quality "got worse", and the decline path is removed — restoring a system that fabricates instead of declining and scores better for it. The metric taught the team to remove the safety feature. Fixing this is a labelling decision in the eval set: mark the unanswerable items, and score declines on them as correct.

Mistake 13 is specific to chatbots and easy to miss. As a conversation grows, the original system instruction moves further from the model's most-attended region while retrieved chunks and history accumulate (07-08). Restating the grounding rule on each turn costs a few dozen tokens and prevents drift.

No. It reduces it, adds provenance, and makes the residual detectable. Stating the boundary precisely matters, because the over-claim is common enough to be an exam distractor and dangerous enough to be a design flaw.

What RAG genuinely fixes. A model answering from weights alone has no way to know whether a fact is current, whether it ever existed, or where it came from (01-02). Retrieval replaces "what the model absorbed during pretraining" with "what is in this corpus right now", which addresses staleness, corpus-specific knowledge, and the total absence of provenance. Those are the three biggest sources of factual error in an ungrounded LLM, and RAG addresses all three.

What RAG does not fix:

That last row is genuinely hard and worth flagging rather than glossing. A model told that clients must restart to reload the trust store and that the error means an untrusted root may conclude "so restarting fixes the error" — which is not stated in either source and is probably correct. Whether that is a helpful synthesis or an unsupported claim depends entirely on your requirements. Systems in regulated domains generally want quotes, not inferences, and that is a design decision to make explicitly.

The mitigation ladder, and why it is a ladder [NVIDIA-DOC]:

Each rung catches a different residual, and the residual never reaches zero. The honest position, and the one worth carrying into both the exam and a design review: RAG converts an unbounded fabrication risk into a bounded, attributable, auditable one. That is a large win. It is not a guarantee, and any architecture that treats it as one has a single point of failure at the most consequential moment.

Four components, all required, none of which is a prompt on its own.

1. A relevance floor. Reranker scores are more thresholdable than cosine similarities (07-07, 07-02), so set a minimum score a chunk must clear to enter the context. Calibrate it on your labelled eval set — pick the value that maximises correct answers plus correct declines. A value copied from elsewhere is a guess.

2. A no-source context. When nothing clears the floor, do not assemble the top-k anyway. Assemble an explicit context stating that no sufficient source was found (07-08). This is what makes declining the easy path for the model rather than a fight against three weakly relevant chunks.

[SOURCES] No source in the knowledge base met the relevance threshold for this query.

[QUESTION] What is the maximum certificate chain depth the client supports?

3. An exact decline instruction. Specify the string. "Reply exactly: The available sources do not answer this." This makes declines countable, alertable, and unambiguous in evaluation.

4. Evaluation that scores declines correctly. Label unanswerable items in the eval set and score a decline on them as correct (09-07). Without this, every metric you have will penalise the control you just built (mistake 9 above).

Two refinements that are worth the effort:

Partial declines. For multi-part questions, the honest answer is often partial: "Sources state the restart requirement [SOURCE 1] but do not address maximum chain depth." That is more useful than a blanket decline and it requires instructing the behaviour explicitly, because the default is to answer what it can and stay silent about the gap.

Decline-rate monitoring. Track it as a first-class metric (12-14). A rising decline rate has exactly two causes, and both are worth knowing about: users asking about things the corpus does not cover — a curation signal (08-01) — or a retrieval regression (07-10). Both are actionable; a stable invented-answer rate is not.

And the counterweight, because over-declining is also a failure: a system that declines answerable questions is broken in a different direction. Users stop trusting it and route around it. Measure both the decline rate and the correct-decline rate, and treat a gap between them as a bug.

Three properties, and a citation missing any one of them is decoration.

Stability. The same chunk must carry the same id across requests and across re-indexing where possible. If ids are positional — "Source 2" — they mean nothing outside the single response that produced them, and cannot be logged, audited, or clicked. Use a durable identifier: document id plus chunk position.

Resolvability. A reader must be able to get from the id to the source text. That means the id maps to something — a document, a page, an anchor, a chunk record — and that the mapping is exposed in the UI or the API. A citation nobody can follow is a claim about provenance rather than provenance.

Presence in the context. The model can only cite what it was shown. Ids must be rendered into the assembled prompt (07-08), which is the connection between the two lessons: assembly decides what can be cited, generation decides what is cited, and validation checks the two against each other.

Given those three, validation becomes mechanical:

for each citation in answer:
    assert citation.id in assembled_context_ids        # existence — catches fabrication
    assert overlap(claim_terms, chunk_text(citation))  # relevance — catches drift
    if quote_required:
        assert quote in chunk_text(citation)           # support — catches invention

Three assertions, all cheap, all runnable on every response in a test suite and sampled in production. The first alone eliminates fabricated citations completely, which is the most alarming class.

What good citation rendering looks like in the context, restating from 07-08 because it is where the two lessons meet:

[SOURCE 3 · doc:err-009#c01 · effective 2026-01-20 · status: current · tier: official]

Five fields, five jobs: a citable id, an effective date for precedence (07-03), a status so superseded content is visible, a trust tier so authority can be weighted, and an ordinal the model can refer to naturally. Every one of them is metadata that would be invisible to the model if it were only stored and not rendered.

A note on user-facing presentation. A verifiable citation still has to be usable. Inline bracketed ids are checkable and ugly; a footnote list with links is friendlier and equally checkable. The requirement is that the mapping from claim to source survives whatever presentation you choose. Stripping citations for readability and keeping them only in logs is a defensible engineering choice for some products and a serious loss for any product where a user needs to check an answer before acting on it.

• Three controls with three different jobs. Grounding reduces fabrication, citation makes the residual detectable, declining stops a retrieval miss from becoming an invention. Any two leaves a live failure mode.
• RAG reduces hallucination and adds provenance; it does not eliminate hallucination. The over-claim is a common exam distractor and a real design flaw.
• The worked example's headline result: Generations B and C had identical content and completely different operational value — the only difference was citations, which turned unverifiable compliance into an answer a human and a machine can both check.
• Same example: without a decline path, an unanswerable question produced a fabricated specification with a chain-depth number that appeared in no source.
• Citation is checkable programmatically. Assert every cited id was in the context (catches fabrication completely), assert term overlap (catches drift), require a verbatim quote (catches invention).
• A model can only cite ids that were rendered into its context. Assembly decides what is citable (07-08).
• Instruct an exact decline string. A prose decline is correct behaviour and unmeasurable; a fixed string is a monitoring signal.
• Score a correct decline as correct. An eval set that penalises declines teaches the team to remove the control.
• The mitigation ladder [NVIDIA-DOC]: grounding via RAG → citation → constrained decoding → guardrails → human review. Earlier rungs reduce errors; later rungs catch what got through.
• Citation serves the Transparency pillar of NVIDIA's trustworthy-AI principles: a cited answer explains how it was reached [NVIDIA-DOC].

Every lesson in this module has shown RAG working, or shown how to make it work. That is a dangerous education on its own, because a reader who has only seen a technique succeed will apply it where it cannot succeed — and there is a widely reported exam heuristic that a RAG option is usually the keyed answer [FIELD], which makes the failure mode worse rather than better. Next: 07-12 closes the module with the counter-cases: no corpus to retrieve from, a requirement to change style or format rather than facts, a hard latency floor, questions that need computation rather than retrieval, and the small-corpus case where the whole apparatus is unnecessary. Knowing when not to build RAG is the last thing that separates applying a pattern from engineering a system.