Real-Time Reasoning Over Structured and Unstructured Knowledge for Agents

Structured knowledge lives in a form with an explicit, predefined shape: rows and columns in a database table, entities and typed relationships in a knowledge graph, cells and headers in a table. Its defining property is that a query against it can be exact — asking a database for "current stock count for SKU 4471" or asking a knowledge graph for "which vendors supply component X" returns a precise, verifiable answer, not an approximation, because the shape of the data was fixed ahead of time specifically to support that kind of query. M6-03's knowledge graph is structured knowledge; so is any relational database an agent might query as a tool call; so is a well-formed table.

Unstructured knowledge lives in a form with no such predefined shape: free-form prose in a document, a back-and-forth chat transcript, an image. Its defining property is the opposite of structured knowledge's — there is no fixed schema to query against directly, which is exactly why M6-01 through M6-03's entire retrieval apparatus (chunking, embedding, semantic search) exists in the first place: it is the machinery built specifically to make unstructured text approximately queryable, when no explicit schema was ever going to make it exactly queryable the way a database table is.

Production agents combine both kinds constantly, because real enterprise knowledge was never filed away as one format or the other exclusively [GROUND TRUTH] (Sources/ncp-aai/domain-6-knowledge-integration.md). An inventory question needs the database's exact current count, not a document that says stock levels "vary" — but the question of why stock levels are currently low might only be explained in an unstructured incident report that never made it into any database field. Treating either kind of knowledge as sufficient on its own leaves an agent blind to whichever kind it ignored.

L1 — The intuition you can carry into an exam

"Real-time" in this objective does not mean "fast" in some vague sense — it means the agent is reasoning over knowledge sources at the moment the question is asked, combining fresh reads from whichever sources the question actually needs, rather than working from a single static snapshot prepared in advance. An agent answering "how many units of SKU 4471 do we have left, and does that match what the incident report from last week said about a shortage" needs a live database read for the first half and a retrieved passage for the second half, and it needs both to be current at the moment of the question, not cached from whenever the agent last happened to check.

L2 — The mechanism: routing each part of a question to the right source, then combining the results

The mechanism this requires is not a new retrieval technique on top of everything M6-01 through M6-03 already covered — it is a routing and combination layer sitting above techniques this module has already built. Given an incoming question, an agent capable of real-time reasoning over mixed knowledge has to do three things in sequence. First, it has to recognize which parts of the question point at which kind of source — the "how many units" clause points at a structured database query; the "does that match the incident report" clause points at unstructured retrieval, likely the same vector or hybrid retrieval M6-01 through M6-03 already described. Second, it has to actually execute each of those lookups — a database call is a tool call, exactly the kind of tool call M2's material on building and connecting custom tools covers, no different in kind from a retrieval call; a knowledge-graph traversal is M6-03's GraphRAG mechanism; a semantic search is M6-01's and M6-02's pipeline. Third, it has to combine the results from every source it queried into one coherent answer, the same synthesis step M6-03 described for agentic RAG's multiple sub-question retrievals, now extended across source types rather than just across sub-questions within one type.

⭐ THE EARNED INSIGHT: The module opened by asking how an agent grounds itself in knowledge it doesn't have in its weights, and every lesson since has answered a narrower version of that question — one retrieval pipeline, one storage mechanism, one extension pattern, one data-quality discipline. This closing lesson's honest answer to the module's full question is that no single one of those pieces was ever the whole answer, and a production agent that only implements one — a great vector pipeline with no structured-query path, or a solid database integration with no way to retrieve an explanatory document — is not doing a smaller version of knowledge integration, it is doing an incomplete one, blind to exactly the kind of knowledge it didn't build a path to.

Nothing about this three-step shape requires new machinery beyond what the earlier lessons in this module already built. What it requires is treating a database query, a graph traversal, and a document retrieval as interchangeable in the agent's reasoning loop — each is simply a callable source of evidence, selected because the current sub-question calls for that particular kind of evidence, exactly the way M6-03's agentic RAG treated a vector search and a graph traversal as interchangeable, callable legs of one retrieval decision.

L3 — The exam-relevant edge case: why structured and unstructured sources cannot substitute for each other

The trap worth being precise about is assuming that a sufficiently good retrieval-over-text system can substitute for a structured query, or vice versa. A database's current inventory count is exact, current to the second, and directly queryable — no amount of semantic search over a document describing "typical" stock levels produces that same precision, because the document was never going to be updated as frequently as the actual count changes, and even if it were, prose describing a number is a strictly lossier representation of that number than the number itself sitting in a database field. Conversely, a knowledge graph or database has no native way to represent the nuance in a paragraph explaining why a shortage happened, what mitigation was attempted, and how a customer should be advised to respond — squeezing that into structured fields loses exactly the texture that made the unstructured source worth having. Each kind of source is exact where the other is not, and each is expressive where the other is not; an agent that only reasons over one is not doing a worse version of complete knowledge integration, it is doing a fundamentally incomplete version of it.

Beyond the structured-versus-unstructured split, the unstructured side of an agent's knowledge is itself no longer just text. Modern RAG spans text, tables, charts, and diagrams [GROUND TRUTH] (Sources/ncp-aai/domain-6-knowledge-integration.md) — a product manual might have its critical safety information in a diagram no text passage ever restates, a financial report's most important number might live inside a chart whose caption paraphrases it too vaguely to be useful, and a support ticket might include a screenshot of an error state that no one bothered to transcribe. An agent whose retrieval pipeline can only chunk and embed prose is blind to all of that content, in the same silent way M6-04 described extraction failures as silent — the pipeline runs, returns confident results, and simply never surfaces the fact that a huge share of the relevant information was never text in the first place.

NVIDIA's Enterprise RAG Blueprint is named directly in the domain's own material as a modular reference architecture connecting unstructured data to agentic systems [GROUND TRUTH] (Sources/ncp-aai/domain-6-knowledge-integration.md), and its relevance here is specifically that it is built around this multimodal reality rather than treating text as the only input worth architecting for. The exam-relevant fact to hold onto is not any particular implementation detail of that blueprint, but the framing it represents: an enterprise RAG system in production is expected to be a reference architecture spanning modalities, not a text-only pipeline with images and tables as an afterthought.

This section is this lesson's synthesis, and it is worth walking through explicitly rather than leaving as an implication, because it is the thing this closing lesson exists to make concrete: none of the previous four lessons describes a standalone system. Each describes one component of a single production agent's knowledge-integration layer.

M6-01's five-stage pipeline — ingest and chunk, embed, store, retrieve, augment and generate — is the backbone that handles the unstructured, text-shaped share of an agent's knowledge. Every document, every chat transcript, every piece of prose an agent needs to ground itself in flows through some version of that pipeline, whether the retrieval leg feeding it is plain vector search or one of M6-03's extensions.

M6-02's vector database and encoder-consistency discipline is the specific infrastructure that makes M6-01's stages 3 and 4 (store and retrieve) work correctly and at scale, and its one hard rule — query and document embeddings must share a model and vector space — applies without exception no matter which higher-level retrieval pattern from M6-03 is running on top of it.

M6-03's GraphRAG, HybridRAG, and agentic RAG are the answer to what happens when a question's shape outgrows plain semantic similarity — multi-hop relational questions that need M6-03's knowledge-graph traversal, compound or ambiguous questions that need agentic planning and retry. In a real production agent, this is also where the structured-knowledge half of this lesson's own subject connects back: a knowledge graph is itself a structured source, and M6-03's GraphRAG traversal is one concrete example of the "route to a structured source" behavior this lesson describes in general terms, not a separate concern from it.

M6-04's ETL and data-quality discipline underlies every one of the above, for both structured and unstructured sources alike. A stale document poisons M6-01's pipeline exactly as M6-04 described; a mis-extracted relationship poisons M6-03's knowledge graph exactly as M6-04 described; and a structured database this lesson has been discussing is just as vulnerable to its own version of the same problem — a table with an unmaintained schema, an inventory count that hasn't synced with a warehouse system in hours, a set of records with duplicated entries under slightly different names. Data quality is not a concern specific to unstructured text; it is a concern specific to any knowledge source an agent depends on, structured or not.

This lesson's own subject — routing a question's different clauses to whichever source (database, graph, document, image) actually answers them, and combining the results — is the layer that sits on top of all four, deciding, for any given question, which of the previous lessons' machinery to invoke, in what combination, and how to synthesize what comes back into one answer.

The row worth carrying forward is the failure-mode row: a structured source's failure under poor data quality looks more dangerous than an unstructured source's, not less, because it fails by returning something exact-looking and confident rather than something that at least has the texture of an approximation. This directly echoes M6-04's point about knowledge-graph traversal results looking more trustworthy than they may deserve to.

This is a constructed scenario with invented numbers, built to make the routing-and-combination mechanism concrete.

A logistics agent is asked: "Is the delayed shipment for order #88214 related to the supplier issue mentioned in this morning's ops chat, and do we have a photo of the damaged pallet on file?"

QUESTION DECOMPOSITION (real-time routing across source types)

  Clause 1: "Is the delayed shipment for order #88214..."
    → STRUCTURED source: query the order-tracking database directly
    → Result: order #88214, status "delayed", carrier note
      "awaiting replacement component from Supplier C"

  Clause 2: "...related to the supplier issue mentioned in this
             morning's ops chat..."
    → UNSTRUCTURED source: retrieve from the chat transcript log
      via M6-01's pipeline (embed the clause, search transcript chunks)
    → Result: chat excerpt, 9:14am: "Supplier C flagged a component
      shortage affecting three open orders, including some pallet
      shipments"

  Clause 3: "...and do we have a photo of the damaged pallet on file?"
    → UNSTRUCTURED, image modality: search an image index tagged
      with order metadata (the multimodal extension this lesson
      names, not covered in pipeline depth by earlier lessons)
    → Result: one image on file, tagged order #88214, uploaded by
      the receiving warehouse

COMBINATION AND SYNTHESIS

  All three results are combined into one answer:

  "Yes — order #88214's delay is linked to Supplier C's component
  shortage flagged in this morning's ops chat at 9:14am. There is
  one photo on file of the damaged pallet, uploaded by the
  receiving warehouse."

Notice what this trace demonstrates about combination specifically: no single source answered the whole question. The database confirmed the order was delayed and named a supplier, but said nothing about the chat conversation or the photo. The chat retrieval confirmed the supplier link the database gestured at but didn't fully explain. The image search answered a completely separate clause that neither of the other two sources could have touched. Real-time reasoning over mixed knowledge, in this example, was not a fourth retrieval technique — it was the routing decision that sent each clause to the one source actually equipped to answer it, and the synthesis step that stitched three separately-sourced facts into one coherent answer.

This is objective 6.5 of Domain 6 [GROUND TRUTH] (Sources/ncp-aai/domain-6-knowledge-integration.md), explicitly framed as the closing objective of the domain's own scope: enabling real-time access and reasoning over structured and unstructured knowledge, with production agents combining structured sources (databases, knowledge graphs, tables) with unstructured ones (documents, chat, images). Expect a question shape that names a specific kind of knowledge — a live inventory count, a chat log, an image — and asks whether it is structured or unstructured, or which kind of source an agent should query to answer a given question; the underlying test is whether you can classify a knowledge source correctly and reason about which retrieval mechanism from earlier in this domain applies to it.

A second recurring shape probes the multimodal framing directly: expect a question naming NVIDIA's Enterprise RAG Blueprint and asking what it represents [GROUND TRUTH] (Sources/ncp-aai/domain-6-knowledge-integration.md) — the correct answer is a modular reference architecture connecting unstructured data to agentic systems, not a specific product or a single-modality tool. Domain 6 as a whole carries a 10% exam weight and is framed as the retrieval backbone behind both agent memory (Domain 5) and grounding (Domain 9) — this closing objective is where that backbone framing becomes most literal, since real production grounding depends on exactly the structured-plus-unstructured combination this lesson describes, not on any one retrieval technique in isolation.

Querying a structured source is a direct, exact lookup against a predefined schema — asking a database for a specific field's current value, or asking a knowledge graph to traverse a named relationship — and it returns a precise result or nothing at all, with no notion of "approximately relevant." RAG retrieval, as covered across M6-01 and M6-02, is fundamentally approximate: it finds text whose meaning is close to a query, ranked by similarity, with no guarantee that the closest match is the single correct answer the way a database field lookup is guaranteed to be. An agent reasoning over mixed knowledge needs both because most real questions have some part that wants an exact current fact and some part that wants explanatory or contextual nuance, and neither querying mechanism can substitute for the other on the part of the question it wasn't built for.

Because a meaningful share of an enterprise's important knowledge was never written down as prose in the first place, and a text-only retrieval pipeline is silently blind to all of it. A safety diagram in a product manual, a trend visible only in a chart, or a photo documenting a damaged shipment carry information that a caption or nearby paragraph often restates only vaguely, if at all — treating the surrounding text as a faithful stand-in for the image or chart itself loses exactly the information that made the visual worth including. Extending retrieval to cover these modalities directly, rather than relying on whatever prose happens to be nearby, is what NVIDIA's Enterprise RAG Blueprint frames as a first-class architectural concern rather than an edge case [GROUND TRUTH] (Sources/ncp-aai/domain-6-knowledge-integration.md).

• Production agents combine structured sources (databases, knowledge graphs, tables) with unstructured sources (documents, chat, images), because real enterprise knowledge was never filed away as one format exclusively.
• Structured sources are exact but expressively narrow; unstructured sources are expressive but only approximately queryable — neither substitutes for the other.
• "Real-time" means reasoning over fresh, current reads from whichever sources a question needs at the moment it's asked, not working from a single static snapshot.
• Modern RAG is increasingly multimodal, spanning text, tables, charts, and diagrams, because a meaningful share of enterprise knowledge was never text in the first place.
• A structured source's failure under poor data quality is arguably more dangerous than an unstructured source's, because it fails by returning something exact-looking and confident.
• This module's four earlier lessons are not standalone systems — M6-01's pipeline, M6-02's vector mechanics, M6-03's extensions, and M6-04's data-quality discipline are components a real production agent combines, routing each part of a question to whichever piece actually answers it.
• NVIDIA's Enterprise RAG Blueprint is named as the reference architecture for exactly this combined, multimodal reality, not as a single-modality retrieval tool.

This module has answered the question it opened with: an agent grounds itself in knowledge outside its weights by retrieving it at query time, and a plain vector lookup stops being enough exactly when a question needs relational reasoning, agentic planning, or knowledge that was never text in the first place — with every bit of that retrieval only as trustworthy as the data-quality discipline underneath it. What this module has not covered is how any of this actually gets deployed, scaled, tuned for GPU throughput, or wrapped in the platform-specific tooling NVIDIA ships for exactly this purpose. Next, the course moves from the retrieval backbone this module built to the NVIDIA platform components that implement, serve, and scale it in production — the natural continuation once an agent knows what to retrieve and how to reason over it.