Data privacy, consent, and why model weights cannot forget

Six principles. Learn them as a list you can recite, because a recall question can ask for any one of them and a scenario question will describe a violation of one.

Two clarifications the exam rewards.

Consent is not a checkbox; it is a record. "Informed" means the person could understand what they were agreeing to. "Specific" means consent to one purpose is not consent to all purposes. "Freely given" means declining was a real option. And — the engineering part — consent must be recorded in a form that later answers the question "what did this person agree to, and when?". A consent record is the Privacy pillar's artifact, in exactly the sense 13-01 uses the word.

Consent is one lawful basis, not the only one. Depending on jurisdiction, processing personal data may also be lawful on other grounds — performing a contract, complying with a legal obligation, and others. This course deliberately teaches the principles rather than any single jurisdiction's rules, because privacy law differs by country and region and changes over time. For an internationally administered associate exam, that is also the right depth: principles are examinable, statute numbers and penalty amounts are not, and an option that names one is usually a distractor.

L1 — The intuition: personal data has five places to leak

Draw the pipeline and mark the exits.

COLLECTION ──► CORPUS / INDEX ──► RETRIEVAL ──► PROMPT ──► MODEL ──► OUTPUT ──► LOGS
     │              │                 │            │          │          │         │
  consent?      minimized?      authorised?    in context   TRAINED    leaked in  retained
  purpose?      de-identified?   permission-   with whose   ON IT?     an answer? how long?
                                  aware?        data?     (no undo)

Five distinct exposures, and they need five distinct controls:

1. Collection — data gathered without a lawful basis, or beyond what the purpose needs.
2. Storage — personal data sitting in a corpus or index in identifiable form when it did not need to.
3. Retrieval — the index returning a document to a user not entitled to it. This is not a model problem; it is an authorisation problem, and it is exactly the failure 07-05 exists to prevent.
4. Training — personal data absorbed into weights, where it can be memorized and later regurgitated, and from where it cannot be removed.
5. Output and logs — personal data emitted in an answer, or accumulated in prompt and completion logs that were never scoped as a personal-data store. This last one is the most commonly forgotten: your observability stack quietly became a database of everything users typed, and it inherits every retention and deletion obligation the rest of the system has.

L2 — The mechanism: memorization, and why training is a one-way door

A language model is trained to predict the next token. To do that well it compresses regularities in the data. Most of what it learns is genuinely general — syntax, facts repeated across thousands of documents, patterns of reasoning. But when a specific sequence appears in the training data, especially if it appears more than once or is unusual enough to resist compression, the model can end up assigning it high probability. That is memorization, and it means a model can complete a rare string it saw during training: a name and an address that appeared together, a phone number, a fragment of a leaked credential file, a distinctive medical narrative.

Three properties of memorization matter for design:

• It is a spectrum, not a switch. Training data influences a model on a continuum from a faint statistical nudge to near-verbatim recall. There is no clean line between "learned a pattern" and "stored a record."
• Duplication increases it. A sequence appearing many times across the corpus is more likely to be reproducible. This makes deduplication — the corpus hygiene of 06-04 — a privacy control as well as a quality one, which is a connection most study material misses.
• You cannot enumerate what was memorized. There is no query that lists everything a model can regurgitate. You can probe for specific strings, and you can red-team, but you cannot produce a complete inventory.

Now put those together with a deletion request. Someone asks you to delete their data. If the data is a row in a table or a document in an index, you delete it and the deletion is verifiable — query for it, get nothing. If the data was in the training set for a model you fine-tuned, what exactly do you delete? There is no parameter that "holds" their record. Their contribution is diffused across the whole tensor, entangled with everyone else's. The gradient updates they caused cannot be individually reversed.

Your options, honestly ranked:

That last row is the design conclusion, and it is why this lesson exists where it does in the course.

L3 — The retrieval-versus-fine-tuning privacy argument, in full

Compare two architectures for the same application: an assistant that answers questions using your organisation's personal-data-bearing records.

Architecture A — fine-tune on the records. The model's weights encode the records. Every property you want for privacy is unavailable:

• Deletion: impossible per record, as above.
• Access control: the model knows what it knows for every user equally. There is no per-user view of a weight. You cannot make the model "not know" a record for one requester and know it for another.
• Purpose limitation: once the knowledge is in the weights, any prompt can reach it. The model does not know which purpose it was trained for.
• Auditability: you cannot say which record produced a given answer. There is no provenance.
• Correction: if a record was wrong, the model's belief is wrong, and fixing it requires retraining.
• Consent withdrawal: structurally unhonourable.

Architecture B — retrieve from the records. The records stay in a store; the model sees only what a query returns:

• Deletion: delete the document, re-index, done — and verifiable by querying for it.
• Access control: enforce the permission model at retrieval time, per user, per request. Two users asking the same question legitimately get different context. This is the mechanism 07-05 builds.
• Purpose limitation: the retrieval scope is configuration. You can restrict which collections a given application may query.
• Auditability: every answer can name the documents it used. That is the citation practice from 07-11, doubling as a privacy audit trail.
• Correction: fix the document; the next answer is correct immediately.
• Consent withdrawal: remove the person's documents from the index and their data stops reaching any answer.

Six properties, all of them present in one architecture and absent in the other. This is not a small preference. For sensitive personal data, retrieval is not merely the cheaper adaptation strategy — it is the one that makes privacy obligations technically satisfiable. The customization ladder in 11-08 chooses between prompting, RAG, and fine-tuning on grounds of cost, data volume, freshness, and whether you need to change style versus knowledge. Add this row to that decision: does the data carry privacy obligations that include deletion? If yes, the ladder stops at retrieval.

Two honest caveats, because overclaiming here is how candidates get caught out:

Retrieval is not automatically private. An index without permission-aware retrieval will hand any user any document, which is worse than not having built it. And retrieved personal data still enters the prompt, still reaches the model provider if the model is hosted externally, and still lands in your logs. Retrieval gives you the ability to control, delete, and audit; it does not exercise it for you.

Fine-tuning is not always wrong. It is the right tool for style, format, and behaviour — things that are not facts about people. Fine-tune on de-identified or synthetic examples to teach the model how to answer, and retrieve the personal data it answers about. That split is the mature architecture, and it is a good sentence to have ready.

Read the "consent withdrawal honourable?" row on its own. If you promised users they could withdraw, and you trained on their data, you made a promise your architecture cannot keep. That is the sentence to carry into the exam and into real design reviews.

A constructed scenario, invented for teaching. No real institution, no real figures.

The requirement. A hospital network wants an assistant that answers clinicians' questions about a patient in front of them — "what did the cardiology consult conclude?", "any documented penicillin reaction?" — over years of unstructured clinical notes. Patients consented, at intake, to their records being used for their own care and for "quality improvement." A separate research consent, which about a third of patients signed, additionally permits use for research.

Proposal on the table. Fine-tune an open-weights model on ten years of clinical notes so it "knows the patient population," then serve it to clinicians.

Why that proposal fails, principle by principle. This is the analysis to be able to produce.

Informed consent. Patients consented to their records being used for care and quality improvement. Did they consent to their notes being absorbed into a model's parameters, from which they could be reproduced for a different patient's clinician? Almost certainly not — and the "informed" test is whether they would have understood that was what they were agreeing to. They would not have, because the intake form did not describe it.

Purpose limitation. Fine-tuning creates an artifact that serves any purpose any future prompt asks for. The purpose boundary that consent established dissolves the moment the knowledge is in the weights. Note that the research-consenting third does not rescue this: a model trained on the whole corpus cannot be restricted to the consenting subset after the fact.

Data minimization. The proposal ingests every note. The purpose — answering questions about the patient in front of the clinician — needs that patient's notes at query time, not all patients' notes in a parameter tensor.

PII handling. Clinical notes are dense with identifiers and, worse, with narrative details that re-identify a person even after names are stripped. A rare diagnosis plus an approximate date plus a specialty is often enough. This is why de-identification of free text is genuinely hard and why "we removed the names" is not de-identification.

Retention. Weights have no retention schedule. There is no expiry on a parameter.

Right to withdraw. A patient revokes consent. Their data is in the weights. The hospital cannot honour it, cannot verify honouring it, and cannot even enumerate what the model absorbed about them.

Six for six. The proposal is not a tuning problem; it is the wrong architecture.

The design that works.

Retrieval, with authorisation at query time. Notes stay in the clinical record system. The assistant retrieves only from the record of the patient in the current clinical context, with the clinician's own access rights enforced in code — not in a prompt — at retrieval time. A clinician who cannot open a chart cannot retrieve from it either. Every answer cites the note it came from, with date and author, so the clinician can open the source. That citation is simultaneously a hallucination control (07-11), a transparency artifact (13-06), and a privacy audit trail.

Fine-tune only on the how, never the who. If the base model writes poor clinical summaries, tune it for that — on de-identified or synthetic examples, or on the hospital's own style guide and published protocols. Behaviour from tuning, facts from retrieval.

Minimize what reaches the prompt. Retrieve the passages needed, not whole charts. Fewer tokens is cheaper (12-09), and fewer personal-data tokens in context is less exposure at every downstream point — including the model provider and the logs.

Treat logs as a personal-data store, because they are. Prompts contain patient data. Completions contain patient data. Therefore the log store needs classification, access control, a retention period, and inclusion in deletion workflows. Redact identifiers at write time where the operational purpose does not need them. This is the control teams forget until an audit finds a three-year-old unrestricted log bucket full of clinical narrative.

Make deletion a real, tested workflow. A withdrawal request must propagate to: the source record system, the vector index (delete the vectors, not just the source — a stale index is a live copy), any cache, any log store, any evaluation set built from production data, and any derived artifact. Write it down as a checklist and test it, by issuing a request and then querying every store for the person's identifiers. Untested deletion pipelines routinely miss the index and the eval set.

Say what you cannot do. If the hospital did fine-tune on notes at some earlier point, the correct disclosure is that removal from the resulting checkpoint is not possible and the remedy is to retire the checkpoint. That is unpleasant to write. It is also the only honest version, and the Transparency pillar requires it.

Where the balance actually sits. Objective 5.2's word is balance, and this scenario has a real one. The most privacy-protective design is no assistant at all, and that has a cost measured in clinician hours and missed details in long charts. The most useful design ingests everything and is unbuildable within consent. The balance is: retrieval scoped to the patient in front of the clinician, authorised per user, minimized per request, cited for verifiability, logged with a retention limit, and deletable on request. That is a defensible sentence, and being able to produce something like it is exactly what "describe the balance between data privacy and the importance of data consent" is asking for.

The single row worth memorizing for the tool-mapping question: Confidential Computing protects data in processing. Encryption at rest and encryption in transit are the two states everyone knows; the exam's interest is the third state, data in use, and that phrase is the tell.

Objective 5.2 is worded distinctively — "Describe the balance between data privacy and the importance of data consent" — and the wording tells you the expected answer shape. It is not "list privacy rules." It is articulate a trade-off: personal data makes systems useful, consent constrains what you may do with it, and a defensible design maximizes utility inside the consent boundary rather than treating either side as absolute. An answer that says "never use personal data" is as wrong as one that says "collect everything."

Privacy is also the first of NVIDIA's four pillars (13-01), defined as complying with privacy law and societal norms for personal data — note that the pillar's own wording goes beyond law to norms, which is what makes "it was technically permitted" an insufficient defence.

Question phrasings to expect:

1. "Why is it difficult to remove an individual's data from a trained model?" — because the data's influence is distributed across parameters and cannot be individually reversed; retraining is the only clean remedy.
2. "An organisation must be able to delete customer data on request. Which architecture better supports this: fine-tuning on the data or retrieval from a governed store?" — retrieval.
3. "Data was collected for one purpose and is now used for another. Which principle is violated?" — purpose limitation.
4. "Which practice reduces privacy risk by collecting only what the purpose requires?" — data minimization.
5. "Which technology protects data while it is being processed?" — Confidential Computing.
6. "What makes consent 'informed'?" — the person understood what they were agreeing to, specifically, and freely.
7. "An LLM reproduces a rare string from its training data. What is this called?" — memorization.

Distractor families:

Not in any straightforward or verifiable way. This is the question the lesson is named for, and the answer needs to be precise rather than merely gloomy.

Personal data that entered training influenced the model through gradient updates that touched large numbers of parameters, mixed with the influence of every other example. There is no lookup from person to parameter. There is no delete operation. What exists:

• Retraining without that person's data genuinely removes their contribution, and is the only remedy you can stand behind. It costs a full training run, so it is at best a periodic batch process — which is why organisations that must support deletion at scale keep personal data out of training entirely.
• Machine unlearning is an active research area aiming to approximate removal without full retraining. Treat it as promising research rather than as a control you can promise an auditor; effectiveness depends on the method and setting, and verifying that removal succeeded is itself an open problem.
• Output filtering for a person's known identifiers suppresses one surface. The information remains in the weights and may surface through a paraphrase, a different language, or an indirect prompt. Mitigation, not deletion.
• Retiring the checkpoint works, completely, and costs you all the adaptation in it. Sometimes it is the right call, and being willing to say so is a mark of a serious answer.

Two contrasts sharpen it. Retrieval: delete the document, re-index, query for it, get nothing. Verifiable in minutes. Weights: no operation, no verification, no inventory of what was memorized.

And note the structural echo from 13-03: an indirect prompt injection planted in a corpus is reversible by deleting the document, while data poisoning of a training set is not. Same asymmetry, different harm. Corpora are editable; weights are not. That one idea earns its place as the most useful thing in this module, because it decides architecture for privacy and for security with a single argument.

Objective 5.2's phrasing puts privacy and consent in tension, and the tension is real in three directions.

Utility versus protection. Personal data is what makes a system useful — an assistant that cannot see your chart cannot answer about your chart. Absolute protection means no system. The resolution is not to pick a side but to shrink the exposure until the remaining exposure is justified by the benefit: minimize fields, scope retrieval to the case at hand, retain briefly, de-identify where the purpose allows, and put the personal data in the layer you can delete from.

Consent versus feasibility. Consent is meaningful only if refusal is a real option and if the promises made are keepable. Blanket consent that nobody reads is compliance theatre. Granular per-purpose consent is meaningful but forces you to build systems that can act on it — separate stores, per-purpose retrieval scopes, working withdrawal. Consent you cannot honour is worse than no promise, because it converts a design gap into a broken commitment.

Privacy versus other pillars. Two genuine conflicts, and naming them is more valuable than pretending they do not exist:

• Privacy versus Nondiscrimination. Detecting bias per slice requires the attribute you would minimize away (13-04). Standard resolutions: collect it only for a consented evaluation sample, keep it in a separately governed store used solely for fairness auditing, or use a purpose-built benchmark set.
• Privacy versus Transparency and auditability. Explaining a decision and retaining logs to evidence controls both mean keeping data. Resolution: retain the minimum needed to audit, with access controls and an expiry, and redact what the audit does not need.

So the balanced statement, which is close to what a well-formed exam answer looks like: personal data may be processed where there is a lawful basis and informed, specific consent for that purpose; only the minimum needed is collected and retained; it is de-identified where the purpose permits and protected in transit, at rest, and in processing; access is authorised per user at query time; the person can withdraw and the withdrawal actually propagates; and the architecture keeps the data in a layer it can be deleted from, which means retrieval rather than training on it.

Yes, on the properties that privacy obligations actually consist of — with the caveat that RAG gives you the capability, not the outcome.

What retrieval genuinely buys you: per-record deletion that is verifiable; per-user access control enforced at query time; purpose limitation as retrieval configuration; provenance for every answer; immediate correction of a wrong record; and a withdrawal mechanism that works. Six capabilities, none of which exists for a fine-tuned model.

What it does not buy you. An index with no permission enforcement is a leak with a search interface. Retrieved personal data still enters the prompt, still travels to the model provider if the model is hosted, and still lands in logs. Vectors are a copy of the data and must be deleted along with the source — a stale index is a live copy. And retrieval does nothing about whatever the base model already memorized from its own pretraining, which is a fact about the vendor's corpus and not about yours.

The mature architecture, in one line: retrieve the facts, tune the behaviour, authorise per user, minimize per request, cite per answer, expire the logs, and test the deletion path.

That is also the practical answer to objective 5.3's "how do you use technologies to improve trustworthiness" as it applies to privacy: permission-aware retrieval, de-identification at ingestion, Confidential Computing for data in processing, output rails against PII leakage, and dataset curation to remove personal data and duplicates before anything is trained.

• Informed consent — agreement given knowingly, specifically, and freely, recorded in an auditable form.
• Purpose limitation — data collected for a stated purpose is not repurposed without a fresh lawful basis.
• Data minimization — collecting and retaining only what the purpose requires.
• PII (personally identifiable information) — data that identifies a person directly or in combination with other data.
• De-identification — removing or obscuring identifiers; incomplete for free text, where narrative detail can re-identify.
• Re-identification risk — the chance that de-identified data can be linked back to a person, often via combinations of fields.
• Retention limit — the defined period after which data is deleted.
• Right to withdraw — a person's ability to revoke consent and have their data removed, with the removal actually propagating.
• Memorization — a model's capacity to reproduce specific sequences from its training data; increased by duplication, unenumerable in scope.
• Machine unlearning — research techniques attempting to remove a data point's influence from trained weights without full retraining.
• Confidential Computing — protecting data while it is being processed, complementing encryption at rest and in transit.
• Permission-aware retrieval — enforcing the user's access rights at query time, in code, before documents reach the prompt.
• Consent record — the persisted artifact stating what a person agreed to, for which purpose, when, and against which version of a notice.
• Deletion propagation — extending a deletion request to every copy: source, index, cache, logs, evaluation sets, derived artifacts.

1. Six principles: informed consent · purpose limitation · data minimization · PII handling and de-identification · retention limits · right to withdraw.
2. A trained model's weights cannot selectively forget one person's data. No per-record delete exists; retraining or retiring the checkpoint are the only real remedies.
3. Therefore: do not train on data you may have to delete. Retrieval makes deletion, access control, purpose limitation, provenance, correction, and withdrawal all technically satisfiable.
4. Behaviour from tuning, facts from retrieval. Fine-tune on de-identified or synthetic examples for style; retrieve the personal data.
5. Objective 5.2 asks for a balance. Neither "never use personal data" nor "consent is a formality" is the answer. Shrink exposure until the remainder is justified.
6. Consent is per purpose, withdrawable, and must be recorded. A blanket acceptance is not consent to training.
7. Logs are a personal-data store. Redact at write, restrict access, set a retention period, include them in deletion.
8. Delete the vectors too. A stale index is a live copy of the data you thought you removed.
9. Confidential Computing covers the third state — data in processing. That phrase is the exam's tell.
10. De-identification of free text is hard. Removing names is not de-identification; narrative detail re-identifies.
11. Deduplication is a privacy control, because duplication increases memorization.
12. Authorisation belongs in code, not in a prompt. Instructing a model not to reveal data is not an access control.
13. Name the pillar conflicts — privacy versus fairness auditing, privacy versus transparency retention — and state the standard resolutions rather than pretending they do not exist.

Privacy gives you obligations you can now evidence: a consent record, a retention schedule, a tested deletion path, an architecture that keeps deletable data deletable. What is still missing is the document that tells anyone outside your team what the system is for, what it was trained and evaluated on, where it fails, and who should not rely on it — written so that a person affected by an output can actually read it.

Next: 13-06 takes up transparency, explainability, and model cards — what belongs on a model card and a data card, why NVIDIA's Transparency pillar insists on non-technical language, what NVIDIA's Model Card Generator is for, and how disclosure and auditability turn a claim into evidence.