A trustworthy AI checklist for your own LLM service

A checklist here is not a policy document and not a questionnaire. It is a table with four columns:

The fourth column is what distinguishes a checklist from a wish list. "Have we considered fairness?" has no pass criterion and will always be answered yes. "Does a per-slice evaluation table exist covering the eight enumerated slices, with no slice more than five points below the best slice, re-run on the current model version?" has a pass criterion, and it can fail.

The seven properties, from 13-01: NVIDIA's four named pillars — Privacy, Safety and Security, Transparency, Nondiscrimination — plus the three the blueprint's scope statement adds: reliability, verifiability, energy-consciousness.

How to use this in practice. Take one service you are actually building or have access to. Work down the list. Mark each item present, absent, or not applicable — and where you mark not applicable, write the one-line reason. The output is not a certificate; it is a gap list, and the gap list is the useful artifact. A completed checklist with eleven honest gaps is worth far more than a checklist with thirty-two ticks and no evidence behind them.

L1 — The intuition: walk the pipeline once per pillar

Two orderings are possible and they find different gaps. Walking by pillar finds missing categories of control — "we have no Nondiscrimination evidence at all." Walking by pipeline stage finds missing stages — "nothing checks the corpus at ingestion." Build the checklist by pillar, then review it by stage, and the intersection is where the real gaps live.

L2 — The mechanism: the thirty-two items

Block A — Privacy (7 items). Evidences the Privacy pillar. Source lesson: 13-05.

The item that fails most often is A6, and it fails on the index — the source record is deleted and the vectors remain. The item with the largest architectural consequence is the corollary of A6 from 13-05: if the service fine-tunes on personal data, A6 cannot pass. Record that as a gap rather than a tick, because there is no cheap fix.

Block B — Safety and Security (8 items). Source lessons: 13-02, 13-03.

B8's second half is the item teams discover too late. If answers do not log which chunks they cited, incident scope is unknowable, and "we cannot determine who was affected" is the worst line in an incident report.

Block C — Transparency (6 items). Source lesson: 13-06.

C6's second clause repays attention. In the worked scenario in 13-06, the harm was not a wrong answer but an applicant treating an advisory answer as a determination. Disclosing authority often matters more than disclosing mechanism.

Block D — Nondiscrimination (5 items). Source lesson: 13-04.

That is six rows under a heading that said five, and the extra one is deliberate: D6 is the item most checklists omit, because a feedback loop is invisible at a single point in time. Count it.

Block E — Reliability (4 items). Source lessons: 01-08, 10-04, 12-14.

E2's pass criterion is the sharp one. A CI gate that has never failed is indistinguishable from no gate. Look for a historical failure.

Block F — Verifiability (4 items). Source lessons: 09-11, 12-12.

Block G — Energy-consciousness (3 items). Source lesson: 13-07.

Thirty-two items — thirty-three counting D6. Seven blocks, and the shape is the answer to "describe how to improve AI trustworthiness": you produce artifacts, per property, with pass criteria.

L3 — What makes a checklist item real rather than decorative

Four tests to apply to any item you write, including any you add to this list:

Test 1: Could it fail? An item that always passes measures nothing. "Have we thought about privacy?" fails this test; "is there a tested deletion path?" does not.

Test 2: Does it name an artifact? Not an activity, not an intention — a file, log, table, number, or test that exists and can be shown. This is 13-01's principle-versus-instrument distinction operationalised.

Test 3: Could someone else verify it? If checking the item requires the author's private knowledge, it is not evidence. Verifiability is one of the seven properties and it applies to the checklist itself.

Test 4: Does it have an owner? An item nobody owns is done by nobody. Assign every row.

And one thing a checklist is not: a substitute for judgement. Thirty-two ticks do not make a system safe; they make its known gaps known. New harms appear that no list anticipated, which is exactly why B7 (red-teaming) and E3 (scheduled monitoring) are on the list — they are the items whose job is to discover what the other items missed.

The relationship worth stating: the checklist produces what the audit consumes. A risk assessment tells you which items matter most for your service; the checklist tells you whether the corresponding artifacts exist; the audit checks whether the artifacts describe reality. If you build only one of the three, build the checklist, because the other two have nothing to work with otherwise.

A constructed scenario, invented for teaching.

The service. An internal assistant answering employee questions about HR policy — leave entitlement, expense rules, the grievance process. RAG over the employee handbook, HR guidance notes, and a wiki any HR staff member can edit. A hosted general-purpose model. One tool: open_hr_case(employee_id, category, summary). It has been live for four months. Nobody has run a checklist against it.

Here is the honest result of the first pass — the shape of a real first pass, which is mostly gaps.

Privacy. A1 fails: no inventory exists, and nobody can immediately say whether the wiki contains personal data. It does — several guidance notes quote real grievance cases as examples, with names. A5 fails: the index has no permission model, so any employee can retrieve any indexed document including those grievance examples. A6 fails: no deletion path has ever been tested; when the team tries one, the source wiki page is deleted and the vectors survive, so the content keeps answering questions. A7 fails: prompt logs, containing four months of employees asking about grievances and medical leave, sit in a general observability store with team-wide access and no retention limit.

Assessment: the privacy block is the worst in the service, and the single highest-severity finding is A5 combined with A1 — personal data in an index with no authorisation. Note that it is not a model failure. It is an ingestion and authorisation failure, exactly the class 07-05 names.

Safety and Security. B1 partially passes: a third-party toxicity filter runs on input, and it logs. B2 fails: no groundedness check, so the assistant confidently paraphrases policy it did not retrieve. B4 and B5 fail: the HR wiki is editable by dozens of people and is ingested with the same trust as the handbook, and nothing strips hidden text — the indirect-injection surface from 13-03, wide open. B6 fails in an interesting way: open_hr_case takes employee_id from the conversation rather than the session, so a payload in a wiki page could open a case against a different employee. B7 fails: never red-teamed. B8 fails: answers do not log cited chunks, so if a bad wiki edit is found, nobody can determine who saw it.

Assessment: B6 is the highest-severity item in the block because it is the only one with an action attached — and it is a one-line fix. Fix the tool authorisation first. That is the least-privilege priority from 13-03 showing up as a checklist ordering.

Transparency. C1 through C4 fail: there is no model card, no data card, and nobody outside the team can say what the assistant is for or what it is known to get wrong. C5 partially passes: a source list is appended to each answer, but not per claim and without version dates, so employees cannot tell whether they are reading current policy. C6 fails: the assistant has a friendly persona and no disclosure notice, and — the finding that matters most here — nothing tells employees that its answers are not authoritative HR determinations. Employees have been treating them as such.

Assessment: the cheapest block to fix in the entire service. Two documents and an interface notice. Also the block whose absence caused the most real confusion, which is typical.

Nondiscrimination. D1 through D6 all fail: no slices enumerated, no per-slice table, no gate. When the team does a first pass, they enumerate slices that make sense for this service — question language, whether the employee is in a country whose policies live in a less-maintained part of the corpus, seniority, and full-time versus contract — and find that employees in one region get materially worse answers because that region's policy annexe was never ingested. The bias here is a corpus-coverage bias, not a model bias, which is the most common finding in a RAG system and one an aggregate quality score cannot see.

Assessment: D1 and D2 are the unblocking items; nothing else in the block is possible without them.

Reliability. E1 partially passes: a 30-item eval set exists from the prototype, never updated. E2 fails: no CI gate. E3 fails: monitoring covers uptime and latency, not answer quality. E4 partially passes: latency alerts have an owner.

Assessment: the quality gate is absent, so no other quality claim in the checklist is defensible over time. Note the dependency — G2 (right-sizing) is impossible without E1, because you cannot tell whether a smaller model is good enough without an eval set. Reliability items unblock energy items.

Verifiability. F1 fails: the hosted model is called by an alias that has silently moved twice. F2 fails: the system prompt lives in an environment variable, has been edited by three people, and has no history. F3 fails: no past result can be reproduced. F4 fails: nobody recorded which embedding model built the index, which means a future migration is a discovery exercise.

Assessment: F2 and F4 are near-zero-cost and prevent expensive future confusion. Do them the same afternoon.

Energy-consciousness. G1 fails: nobody has measured anything; the only figure available is a monthly invoice with no per-request denominator. G2 fails: the model was chosen at prototype time and never revisited. G3 fails: batching off, ten chunks retrieved per query, no cache, no output cap.

Assessment: the invoice divided by the request count is a five-minute measurement that satisfies G1 for a stated boundary, and G3's items are configuration changes.

The prioritised remediation list. This is the output that makes the exercise worth doing — ordered by severity times cost, not by checklist order:

1. B6 — take employee_id from the session, not the conversation. One line. Highest severity, lowest cost.
2. A5 and A1 — inventory the corpus for personal data; remove the named grievance examples; add permission-aware retrieval. Highest-severity privacy exposure.
3. A7 — restrict access to prompt logs, add a retention period, redact at write.
4. C6 and C2 — interface disclosure and an intended-use/out-of-scope statement. Hours of work; resolves the most common real confusion.
5. B5 and B4 — tier the wiki below the handbook; sanitize at ingestion; flag assistant-directed imperatives for review.
6. E1 and E2 — extend the eval set and put a gate in CI. Unblocks D, G2, and every future quality claim.
7. D1 and D2 — enumerate slices, size them, run the table. Finds the missing regional annexe.
8. F2 and F4 — prompt into version control; record the index's embedding model.
9. B8 — log cited chunk IDs per answer. Cheap now, invaluable during an incident.
10. A6 — build and test the deletion path end to end, including the vectors.
11. G1 and G3 — measure per-request cost; turn on batching; rerank to fewer chunks; cap outputs.
12. B7 — red-team, including a planted-document test on a staging corpus.

What the pass revealed that no individual lesson would have. Three cross-cutting things, and they are the real product of running a checklist rather than reading about one:

• The dependencies run between blocks. No eval set means no reliability gate, no per-slice table, and no right-sizing decision. 01-08's eval set turns out to be load-bearing for three of the seven properties.
• The worst findings were architectural, not model-related. Personal data in an unauthorised index; a tool taking identity from conversation text; a corpus missing a region's policies. Not one of them is fixed by a better model.
• The cheapest fixes were in Transparency, and their absence caused the most confusion. Two documents and a notice. Transparency is the pillar most often skipped precisely because nothing about it is technically hard, so nobody's engineering instinct claims it.

The one-page version. If you memorize one table from this module for the exam, memorize the harm-to-pillar table in 13-01; if you memorize a second, memorize this one, because it answers "describe how to improve trustworthiness" for every pillar at once.

All four Domain 5 objectives converge here, which is why this is the module's terminal applied lesson:

• 5.1 Describe the ethical principles of trustworthy AI — the seven properties are the checklist's blocks.
• 5.2 Describe the balance between data privacy and the importance of data consent — Block A is that balance made operational: consent recorded per purpose, data minimized, deletion honourable, and the utility preserved by keeping the data in a layer you can delete from.
• 5.3 Describe how to use NVIDIA and other technologies to improve AI trustworthiness — the "named technologies" column of §5's table.
• 5.4 Describe how to minimize bias in AI systems — Block D, and note that it minimizes with a gate and discloses the residue rather than claiming elimination.

Question phrasings to expect. Checklist-shaped questions on this exam are scenario questions asking which control is missing or which control best addresses this:

1. "A team wants to demonstrate to an auditor that unsafe content is being blocked. What must they have?" — the guardrail layer's decision log, not an aligned model.
2. "A service must support customer data deletion. Which design decision most affects whether this is possible?" — whether the data is in a retrieval store or in fine-tuned weights.
3. "Which artifact documents intended use, limitations, and evaluation results?" — the model card.
4. "An organisation reports 92% accuracy and receives subgroup complaints. What was missing?" — per-slice evaluation with a release gate.
5. "Which practice makes a past evaluation result attributable to a specific configuration?" — version pinning and run logging.
6. "Which NVIDIA tool would you name for curating and validating unbiased datasets?" — TAO Toolkit.
7. "Which of the following is an energy-conscious design decision?" — choosing the smallest model that passes the evaluation set.

Distractor families across the whole domain, consolidated — this is the highest-value revision table in the module because it covers all four objectives at once:

A RAG service concentrates its risk in places a bare-model checklist does not look, so add these to the seven blocks:

Corpus items. Where every source came from, its owner, its trust tier, its ingestion date and expiry, and what it does not cover. Coverage gaps are simultaneously a transparency item (C4), a reliability item, and — as the worked example showed — a nondiscrimination item, because a missing regional annexe is a group harm.

Retrieval items. Permission enforcement in code at query time (A5). Retrieval scope per application. Per-slice retrieval metrics, because disparate retrieval quality precedes disparate answers. Ingestion sanitization and imperative-pattern flagging (B4).

Prompt-assembly items. How many chunks, in what order, with what delimiting, with retrieved content labelled as untrusted data. Minimization of personal data reaching the prompt (A4).

Citation items. Per-claim provenance with version and date (C5), and per-answer source logging for incident scope (B8) — the same mechanism serving transparency and security.

Index-lifecycle items. Deletion propagating to vectors (A6). Embedding-model version recorded with a migration plan (F4). Refresh cadence and expiry of superseded documents.

That is the practical answer to why this module's checklist is longer than a generic AI-ethics list: in a retrieval system, most of the trust-relevant behaviour lives outside the model, and a checklist that only examines the model will pass a service whose real problems are all in the corpus and the authorisation layer.

Bring the gap list, not the ticks. A candidate who says "we ran a trustworthy AI review on our assistant and found eleven gaps; here are the three we fixed first and why" is describing engineering judgement. A candidate who says "we completed our checklist" is describing a form.

The presentable version is one page: the service in two sentences, the seven properties as rows, present/absent/N/A per block, the three highest-severity findings with their fixes, and the remaining gaps with an honest reason each. Add the two artifacts that carry the most weight in a conversation — the per-slice evaluation table and the model card, because they are concrete, unusual for a candidate to have, and immediately legible to a technical interviewer.

Two things to be ready to say out loud, since they are the most likely follow-up questions and both have good answers in this module:

• "What did you decide not to fix, and why?" — a real prioritisation answer, ideally citing severity times cost.
• "What can your system not do?" — the model card's limitations section, delivered without defensiveness. "We cannot remove an individual's data from the fine-tuned checkpoint, which is why we moved the sensitive corpus to retrieval" is a sentence that demonstrates you understand 13-05 better than most people who have read about it.

If you can only build four, build these, in this order — chosen because each unblocks the most other items per unit of effort:

1. The frozen evaluation set. It unblocks reliability (the gate), nondiscrimination (the per-slice table is a slicing of it), and energy-consciousness (right-sizing is undecidable without it). One artifact, three properties. This is why 01-08 sits in week one of the course.

2. Tool authorisation in code. The highest severity-to-cost ratio in the entire list. It bounds the damage from prompt injection, from hallucination, and from ordinary bugs, all at once, and it is usually a small change.

3. Guardrail logs. Not just rails — rails with logs. The logs are what convert your safety claim into evidence, and they are also the dataset you use to tune the rails and to scope incidents.

4. The model card. Cheap, legible to outsiders, and it forces the team to write down intended use, out-of-scope use, and limitations — which is the exercise that surfaces disagreement about what the system is actually for. Teams routinely discover, while writing section two, that they disagree.

Everything else on the list is worth doing, and those four make the rest possible or make it cheap. Notice that only one of the four is a document, and the other three are engineering — which is the shape of this whole domain: four objectives that start with "Describe," delivered by a service whose trustworthiness is built rather than asserted.

• Trustworthy AI checklist — a table of items, pillars, artifacts, and pass criteria for one specific service.
• Pass criterion — the verifiable condition that decides an item, phrased so someone else could check it.
• Gap list — the checklist's real output: what is absent, with severity and an owner.
• Data inventory — a written list of every personal-data field the service touches, with purpose and lawful basis.
• Tested deletion path — a deletion runbook that has actually been exercised across source, index, cache, logs, and eval sets.
• Source tier — a trust level assigned to a corpus source, used to restrict which questions it may answer.
• Tool authorisation in code — validating every proposed tool call's arguments against the authenticated session, outside the model.
• Per-answer source logging — recording which chunks and versions each answer cited, so an incident can be scoped.
• Release gate — an automated check that fails the build, as distinct from a report that is read.
• Version manifest — the recorded set of model, embedding-model, and prompt versions for a run.
• System card — documentation of the deployed application as a whole, not just the model.
• Severity times cost prioritisation — ordering remediation by harm prevented against effort required, rather than by checklist order.

1. Seven blocks: Privacy · Safety and Security · Transparency · Nondiscrimination · Reliability · Verifiability · Energy-consciousness — NVIDIA's four pillars plus the blueprint's three added properties.
2. Every item names an artifact and a pass criterion. An item that cannot fail measures nothing.
3. The output is a gap list, not a certificate. Eleven honest gaps beat thirty-two unevidenced ticks.
4. Build the evaluation set first. It unblocks the reliability gate, the per-slice table, and the right-sizing decision — three properties from one artifact.
5. Fix tool authorisation second. Highest severity-to-cost ratio in the list; it bounds injection, hallucination, and plain bugs together.
6. Rails need logs. The log is the evidence; the rail without it is a claim.
7. Transparency is the cheapest block and the most skipped. Two documents and an interface notice.
8. In a RAG service the risk is in the corpus and the authorisation layer, so add corpus, retrieval, prompt-assembly, citation, and index-lifecycle items.
9. A CI gate that has never failed is not a gate. Look for a failure in its history.
10. Prioritise by severity times cost, never by checklist order.
11. Mark N/A with a reason, or blocks disappear silently.
12. Bring the gap list to an employer. The three highest-severity findings and what you did about them is the answer that demonstrates judgement.

The checklist is the last thing this course builds for the exam. Everything in it — the pillars, the rails, the slice table, the model card, the levers — is current as of the sources this course was built from, and the field will move. The last capability you need is not another artifact but a habit: reading a research paper efficiently enough to tell whether a new technique changes anything you do, and tracking the field without drowning in it.

Next: 13-09 covers how to read an AI research paper and track new trends — the read order that gets you the claim in ten minutes, how to check whether a result is real, and how to build an information diet that survives the pace of this field. It serves objective 1.7, it is the course's exit into the field, and it is genuinely better after the exam.