ETL and Data Quality for the Knowledge a RAG Agent Retrieves

The claim this lesson is built around, stated as plainly as the source material states it: retrieval quality is capped by the quality of the underlying data [GROUND TRUTH] (Sources/ncp-aai/domain-6-knowledge-integration.md). "Capped" is a precise word choice, not a rhetorical flourish, and it is worth sitting with exactly what it means. It does not mean bad data makes retrieval somewhat worse on average. It means bad data sets a ceiling that no downstream improvement can rise above — an agent's retrieval pipeline can have a state-of-the-art embedding model, a perfectly tuned vector index, a well-built knowledge graph, and an agentic reasoning loop that retries thoughtfully on thin results, and none of that raises the ceiling one bit if the document the agent needed to retrieve was duplicated four times with slightly different, contradictory edits, or if the one authoritative version of a policy was silently superseded eight months ago and nobody removed the outdated copy from the knowledge base.

This is why this lesson exists as its own numbered objective rather than as a footnote inside M6-01. Every retrieval mechanism covered earlier in this module — plain vector similarity, graph traversal, hybrid combination, agentic planning and retry — operates on top of a body of ingested knowledge, and every one of them inherits whatever defects that body of knowledge already had. A retry loop in agentic RAG can reformulate a query as many times as it likes; if every version of the relevant document in the corpus says something subtly wrong, retrying the query does not somehow retrieve a correct answer that was never written down anywhere. The defect is not in retrieval. It is upstream, and the fix has to be upstream too.

L1 — The intuition you can carry into an exam

ETL stands for extract, transform, load, and it names the pipeline that integrates enterprise or client data sources into a form an agent can retrieve over [GROUND TRUTH] (Sources/ncp-aai/domain-6-knowledge-integration.md). Extract pulls raw data out of wherever it currently lives. Transform reshapes, cleans, and standardizes that raw data into a consistent form. Load writes the transformed result into the store the retrieval pipeline actually reads from. Three verbs, one direction: from scattered, heterogeneous source systems toward one coherent, retrievable knowledge base.

L2 — What each stage does, and where its specific failure modes live

Extract is the stage where raw source material — PDFs, wiki pages, ticketing-system exports, database tables, spreadsheets, scanned documents — gets pulled out of the systems where it natively lives and into the pipeline. The characteristic failure mode here is incomplete or corrupted extraction: a PDF parser that flattens a table into an unordered jumble of words, a scanned document with no machine-readable text layer that yields nothing at all, or a database export that silently drops rows because of a permissions issue nobody noticed. These failures share a dangerous property worth naming precisely: they tend to be silent. A parser that mangles a table does not throw an error — it returns text, just wrong text, and everything downstream treats that wrong text as if it were a faithful representation of the source.

Transform is the stage where extracted raw material gets standardized: normalizing formats, resolving inconsistent terminology, stripping boilerplate and navigation elements that add no informational value, deduplicating near-identical content, and structuring data into whatever shape the retrieval pipeline expects — chunked passages for a vector index, extracted entities and relationships for a knowledge graph. This is where most of the deliberate, named data-quality work in the next section actually happens, and its characteristic failure mode is a transform step that runs successfully but makes a bad judgment call at scale — a deduplication threshold set too loosely so genuinely distinct documents get merged, or too tightly so obvious duplicates survive and crowd every retrieval result.

Load is the stage where the transformed, cleaned result actually lands in the destination store — the vector database from M6-02, the knowledge graph from M6-03, or both. Its characteristic failure mode is a mismatch between what transform produced and what load actually wrote: a load process that only partially completes and leaves the store in an inconsistent state, or a load step that overwrites an existing, still-current entry with a stale one because of an ordering bug in how updates get applied.

L3 — The exam-relevant edge case: why extract-stage failures are the most dangerous

Of the three stages, extract-stage failures deserve the most weight for a specific structural reason: they happen earliest, and their errors propagate through every single stage that follows without any of those later stages having a way to detect or correct them. A transform stage cannot deduplicate two documents correctly if the extraction already corrupted one of them into a different, no-longer-recognizable form. A load stage cannot preserve the freshness of content it never received intact. And every retrieval technique from the earlier lessons in this module — vector similarity, graph traversal, hybrid combination — operates on whatever finally lands in the store, with zero visibility into whether the extract stage silently mangled the source it was working from. This is precisely why data-quality checks belong as close to the extract stage as they can practically be placed: the earlier a defect is caught, the fewer downstream stages have already built on top of a corrupted foundation.

⭐ THE EARNED INSIGHT: Every lesson before this one in the module implicitly measured retrieval sophistication as the thing worth optimizing — a better index, a better graph, a better retry loop. This lesson's honest conclusion is that sophistication has a ceiling it cannot see past, and the ceiling is set somewhere the retrieval mechanism has no visibility into at all. A retrieval system cannot tell you its answer is capped by bad data, because from inside the retrieval mechanism, a stale document and a current one look identical, and a mis-extracted graph edge looks exactly as traversable as a correct one. The discipline this implies is uncomfortable but simple: when a RAG agent gives a wrong answer, check the source data before you touch the retrieval mechanism, because the retrieval mechanism is the last place the defect is likely to actually be.

Beyond the three ETL verbs themselves, Domain 6 names data-quality checks, augmentation, and preprocessing as their own explicit concern — the ongoing discipline of keeping a knowledge base clean, deduplicated, and well-formatted [GROUND TRUTH] (Sources/ncp-aai/domain-6-knowledge-integration.md), rather than a property that ETL delivers once and is done with. This distinction matters because ETL, as described above, is naturally framed as a pipeline with a beginning and an end for any given batch of source data; data quality is better understood as a standing property a knowledge base either has or loses over time, as new documents get added, old ones go stale, and the corpus grows large enough that problems invisible at a small scale become material at a large one.

Deduplication catches near-identical or exactly duplicated content that made it through extraction and transform — the same policy document uploaded under two different filenames, a boilerplate legal footer repeated across thousands of pages that, left unremoved, becomes the nearest semantic neighbor of an enormous share of queries simply because it appears so often. Staleness checks identify content that was correct once but has since been superseded — a product-version-specific troubleshooting guide that nobody flagged as outdated when the next version shipped, sitting in the corpus indistinguishable, to a retrieval system, from the current guidance. Format consistency catches content that survived extraction and transform in an unusable shape — a table rendered as run-on prose, a list that lost its structure, a heading hierarchy that collapsed into flat, undifferentiated text.

Augmentation and preprocessing, named alongside these checks, cover the work of actively improving raw extracted content before it becomes retrievable — attaching metadata that later retrieval stages will filter or rank on, normalizing terminology so that a query using one term reliably matches source content using a synonym, and preparing content in whatever specific structure a given retrieval mechanism needs, whether that is passage-sized chunks for a vector index or entity-relationship triples for a knowledge graph.

This is the section that ties this lesson's abstract claim — retrieval quality is capped by data quality — to something concrete for each specific technique this module has already covered, because the mechanism of poisoning is different for each one.

Plain vector RAG (M6-01, M6-02). A duplicated chunk does not just waste storage — it actively distorts retrieval, because a chunk repeated many times across the corpus effectively gets many "votes" in a similarity ranking that a unique chunk saying the same thing would only get once, and a boilerplate fragment repeated on every page can end up dominating retrieval results for queries that have nothing to do with its content, simply through sheer repetition. A stale document does not announce itself as stale to an embedding model — it embeds exactly as confidently as a current document would, and nothing in the vector similarity score reflects how out of date the content actually is.

GraphRAG and HybridRAG (M6-03). A knowledge graph's correctness depends entirely on the entities and relationships extracted into it being faithful to the source, and this is arguably more fragile than the vector case rather than less: a single mis-extracted relationship — a "sole-supplies" edge created between the wrong two entities because an extraction step misread ambiguous source text — produces a traversal result that looks exactly as confident and exact as a correctly extracted one, because a graph traversal has no built-in way to distinguish a well-founded edge from a poorly extracted one. The verifiability advantage M6-03 credited to GraphRAG is a genuine strength when the underlying extraction was done well, and it becomes actively dangerous when it wasn't, because a wrong answer delivered as an exact, inspectable path reads as more trustworthy than a wrong answer delivered as an uncertain-looking passage, even though both are equally wrong.

Agentic RAG (M6-03). The retry-and-reformulate loop that M6-03 covered as agentic RAG's defining strength is specifically a defense against thin retrieval results — too few matches, low similarity scores. It has no defense at all against retrieval that returns a confident, well-populated, wrong result, because thin and wrong are entirely different failure signatures, and an agent's retry logic can only act on the signal it's built to detect. If every version of a document in the corpus is confidently, consistently wrong, an agent that reformulates and retries its sub-question five different ways will retrieve five confident, consistent, wrong answers, and nothing about retrying tells the agent it should be suspicious of what it kept finding.

The row worth carrying forward past this lesson: metadata attachment during transform is the decision with the most asymmetric long-term cost. Skipping it is nearly free at transform time and nearly impossible to fix retroactively — reconstructing "when was this actually current" or "what source did this really come from" after the fact, once that information was never captured, is far harder than attaching it in the first place would have been.

This is a constructed scenario with invented numbers, built to make the mechanism of data-quality poisoning concrete rather than abstract.

An HR knowledge base for an internal agent contains a remote-work policy document. Trace what happens when that document is superseded but the old version is never removed.

TIMELINE

  Month 0:  "Remote Work Policy v1" ingested — states employees may
            work remote up to 2 days/week.
  Month 6:  Policy is revised. "Remote Work Policy v2" is published
            on the company wiki — states employees may work remote
            up to 4 days/week. The wiki page for v1 is archived but
            NOT deleted, and no automated re-sync process exists to
            detect that it should be removed from the agent's
            knowledge base.
  Month 9:  An employee asks the HR agent: "How many days a week
            can I work remotely?"

RETRIEVAL, MONTH 9

  Both v1 and v2 are still present in the vector store as separate,
  unrelated chunks — extraction never flagged v1 as superseded, and
  no staleness metadata was attached to either at transform time.

  Query embedded: "How many days a week can I work remotely?"

  Top 2 retrieved chunks, by similarity (v1 and v2 are near-identical
  in phrasing, differing mainly in the number):
    0.89  "Remote Work Policy v1: up to 2 days per week"
    0.87  "Remote Work Policy v2: up to 4 days per week"

  Both chunks are handed to generation. The model has no signal in
  either chunk's TEXT indicating which one is current — nothing
  about "v1" or "v2" as filenames necessarily makes it into the
  chunk content itself, and even if it does, the model has no
  external fact telling it "v2 supersedes v1."

GENERATED ANSWER (illustrative, one plausible failure):
  "Employees may work remote up to 2 days per week, though some
  more recent guidance suggests this may have been updated to 4
  days — please confirm with HR."

Notice exactly where the defect lives in this trace: not in the embedding model, not in the vector index, not in the retrieval ranking — all of those performed correctly, retrieving both genuinely similar chunks with reasonable confidence scores. The defect is that the knowledge base contained two versions of a policy with no staleness metadata distinguishing which one was authoritative, a data-quality gap from the transform stage that no downstream retrieval sophistication could have caught. Now contrast what a staleness-aware transform step would have produced: metadata tagging v1 as superseded at month 6, either removed from the retrievable store entirely or down-weighted and excluded from generation, leaving only v2 retrievable and eliminating the ambiguity before it ever reached the agent's answer.

ETL and data quality make up objectives 6.3 and 6.4 of Domain 6, and the domain's framing makes this material's weight explicit: because retrieval quality is capped by data quality, this work is not optional plumbing — it directly determines answer accuracy [GROUND TRUTH] (Sources/ncp-aai/domain-6-knowledge-integration.md). This is one of the more direct statements in the entire domain about why a topic matters, not just what it is, and it signals that a scenario question is likely to test the causal chain rather than mere definitions: given a description of poor source data, what happens to retrieval accuracy downstream, regardless of how good the retrieval mechanism itself is.

Expect a question shape that presents a scenario where retrieval mechanics are described as functioning correctly — a vector index returning genuinely similar chunks, a graph traversal completing successfully — but the answer is still wrong, and asks where the problem most likely originates. The domain's own self-check material states the correct reasoning directly: retrieval quality is capped by the quality of the underlying data, so poor data poisons retrieval regardless of model quality [GROUND TRUTH] (Sources/ncp-aai/domain-6-knowledge-integration.md) — the keyed answer in this question shape is consistently upstream (the data), not downstream (the retrieval or generation mechanism), even when a distractor option plausibly points at a retrieval-mechanism fix.

ETL failure and data-quality problems overlap but are not identical, and the distinction is about when and how the defect was introduced versus discovered. An ETL failure is typically a defect in one of the three pipeline stages themselves — extraction that corrupted content, a transform step that mishandled formatting, or a load step that wrote data inconsistently. A data-quality problem is broader: it includes ETL failures, but it also includes issues that a technically flawless ETL run would still leave behind, such as a document that was accurately extracted, transformed, and loaded exactly as written, but was simply outdated or duplicated to begin with at the source. Data-quality checks, augmentation, and preprocessing exist specifically to catch this second category — problems that survive a correctly functioning ETL pipeline because the pipeline faithfully carried forward a defect that was already present before extraction ever started.

No, and this is the single most important negative result in this lesson: retrieval quality is capped by data quality, which means no retrieval-side improvement raises that cap [GROUND TRUTH] (Sources/ncp-aai/domain-6-knowledge-integration.md). A better embedding model retrieves the same wrong or stale document just as confidently as a worse one would. A knowledge graph traversal returns a mis-extracted relationship just as exactly and just as inspectably as it would return a correctly extracted one — the traversal mechanism itself cannot tell the difference. An agentic retry loop can only detect and correct thin results, not confidently wrong ones, because retrying a query against the same flawed source material simply retrieves the same flaw again. Every meaningful fix for a data-quality problem happens upstream, in extraction, transform, deduplication, and staleness management — never downstream, in the retrieval mechanism itself.

• Retrieval quality is capped by the quality of the underlying data — no retrieval-side improvement, however sophisticated, raises that cap.
• ETL is three separable stages — extract, transform, load — each with its own distinct failure modes, and extract-stage failures are the most dangerous because they are typically silent and propagate through everything downstream.
• Data-quality checks, augmentation, and preprocessing are an ongoing discipline, not a one-time setup task performed once and forgotten.
• Every retrieval technique from earlier in this module — plain vector RAG, GraphRAG, HybridRAG, agentic RAG — inherits whatever data-quality defects the underlying knowledge base already had, and each is poisoned by those defects in a mechanically different way.
• A mis-extracted knowledge-graph relationship is arguably more dangerous than a bad vector-retrieved chunk, because it is delivered as an exact, inspectable, and therefore more trustworthy-looking path.
• Agentic RAG's retry-on-thin-results behavior has no defense against retrieval that returns a confident, well-populated, but wrong result — thin and wrong are different failure signatures.
• Metadata attached at transform time — source, date, version — is cheap to capture and very expensive to reconstruct after the fact; skipping it forecloses staleness and provenance checks later.

Every lesson so far in this module has treated the agent's knowledge as unstructured text — documents, chunks, graph entities extracted from prose. Real production systems rarely have the luxury of that simplicity. Next: M6-05 closes the module by covering what happens when an agent needs to reason in real time over structured sources — databases, knowledge graphs, tables — alongside unstructured ones like documents, chat, and images, bringing together everything this module has covered into the multimodal reality of a production RAG system.