Choosing a Model Adaptation Strategy Under Real Constraints

Adaptation strategy selection is the decision about which rung of the customisation ladder — prompt, RAG, prompt learning, PEFT/LoRA, full fine-tune, or alignment — you will actually use, given constraints that were set before the technical question was asked. It has two halves that are frequently confused.

The merit half asks which option best solves the problem. That is 11-08.

The feasibility half asks which options are available at all. This is where most real decisions are settled, and it runs through six constraint classes in a fixed order of authority:

Run them in that order and each pass is cheap. Run them backwards — costing a full fine-tune before checking whether the compliance requirement forbids weight changes — and you have spent the analysis budget on a non-option.

The output is not "the best approach." It is a short list, a recommendation, an explicit statement of what the recommendation gives up, and the measurement that would change your mind. That last item is what makes it a defensible engineering document rather than an opinion.

L1 — The intuition: what is fixed, what is a fact, what is money

Sort every constraint into three buckets before you think about technique.

• Fixed: compliance requirements, contractual SLAs, data residency. These are not engineering trade-offs. They eliminate, full stop.
• Facts: what data you hold, what the base model can do, what your GPU has in it today. Facts can be changed, but changing them is a separate project with its own timeline.
• Money: compute budget, annotation budget, engineering hours. These are negotiable, and the negotiation is worth having only for an option that survived the first two buckets.

The common failure is treating a fixed constraint as money ("surely legal will make an exception") or treating money as fixed ("we could never buy annotation"). Both misclassifications produce recommendations that get rejected for reasons that had nothing to do with the technical content.

L2 — The six passes, with the elimination logic for each

Pass 1 — Governance and compliance. These clauses eliminate options unconditionally:

Note the shape: most of these push you toward retrieval, and that is not a coincidence. Retrieval keeps data in a store you can inspect, filter, and delete, which is what governance requirements are about.

Pass 2 — Data shape. You can only train what you have data for. One question resolves this:

Documents with no target responses?          → RAG, or continued pretraining
                                                if the corpus is large enough
(prompt, ideal response) pairs?              → SFT / LoRA
(prompt, better, worse) comparisons?         → DPO, or full RLHF
None of the above?                           → prompting, and a data project

This is the same test as 11-01, and it is the single most common place a plan collapses. A team that has decided to fine-tune and holds only PDFs has decided to do a data-generation project without noticing.

Pass 3 — Hardware. Apply the memory arithmetic from 11-04. For a model with P parameters:

inference only, bf16        ≈ P × 2 bytes
LoRA, bf16 frozen base      ≈ P × 2 bytes + small adapter terms
LoRA, 4-bit frozen base     ≈ P × 0.5 bytes + small adapter terms
full fine-tune, bf16+Adam   ≈ P × 12 bytes  (to 16 with a master copy)
RLHF stage 3                ≈ 3–4 models resident simultaneously

Then compare against what you have, leaving headroom for activations and framework overhead. For a 7B model this yields, at one significant figure:

7B inference                 ≈ 14 GB   → one 24 GB card
7B LoRA (bf16 base)          ≈ 14 GB   → one 24 GB card
7B LoRA (4-bit base)         ≈  4 GB   → one 16 GB card comfortably
7B full fine-tune            ≈ 84 GB   → multi-GPU
7B RLHF with PPO             ≈ 56 GB+  → multi-GPU, and far slower per step

Pass 4 — Latency. RAG adds a retrieval hop plus a longer prefill, and both are measurable before you build anything. If the budget is a hard SLA figure, cost it:

budget          = p95 end-to-end target
retrieval hop   = embedding + ANN search + rerank
prefill         = grows with retrieved context length
generation      = output tokens × inter-token latency

If retrieval plus the inflated prefill does not fit, RAG is eliminated for that path — which is one of the genuine counter-cases to the RAG heuristic (07-12). Fine-tuning and prompting add no retrieval latency, and a fine-tune can reduce prefill by shortening the prompt. 12-10 is the measurement discipline.

Pass 5 — Time. Map each option to its critical path, and note that the training run is almost never the long pole:

Pass 6 — Budget. Only now do you cost the survivors, and cost both halves: one-off and per-request, over a stated horizon, as 11-08 section 4 demonstrates. Costing before the earlier passes is wasted work.

L3 — What to do when the passes leave you nothing

Sometimes every option is eliminated. That is a real and important result, and it has a small number of legitimate responses. Naming them is more useful than pretending the elimination cannot happen.

Option 1 — relax the requirement. Often the requirement is stricter than the need. "Real-time" frequently means "within an hour." "Cite the source" sometimes means "link to the document," not "quote the paragraph." Take the actual constraint back to whoever set it with the cost of each interpretation attached.

Option 2 — change the model, not the method. A smaller model makes every hardware constraint easier. A stronger model may clear the bar with prompting alone and remove the need for adaptation entirely. Model selection is the most underused lever in this whole module, and 10-02 makes zero-shot capability testing the cheap first experiment.

Option 3 — buy the constraint away. Rent GPUs instead of owning them. Buy annotation rather than staffing it. This converts a hardware or data constraint into a budget constraint, which is the most negotiable class.

Option 4 — reduce scope. Solve the requirement for one high-value slice of traffic rather than all of it. A narrower behaviour needs less data, less compute, and less time, and it produces a shippable result that funds the rest.

Option 5 — say the project is not feasible as specified. This is a legitimate engineering output and the one people are most reluctant to produce. A written statement of which constraint blocks which option, with the arithmetic, is far more valuable than a plan that will fail in month four.

The meta-rule that outranks all of them: every path requires an evaluation set, and the eval set is the cheapest item on every list. If constraints have eliminated everything, build the eval set anyway. It costs days, it is a prerequisite for all six options, and it converts the next round of this conversation from opinion into measurement.

The matrix. Read down a constraint column to see what it kills; read across a row to see what a given option is vulnerable to.

Two rows are worth reading twice.

"Base model capability is short" blocks everything. No adaptation method raises the ceiling set at pretraining (11-02). If the model cannot do the task at the best prompt you can write, the answer is a different model, and every hour spent on adaptation before checking this is wasted.

"No evaluation set" blocks everything. Not as a matter of good practice but as a matter of logic: without a baseline you cannot detect improvement, cannot detect regression, cannot select a checkpoint, and cannot know when to stop climbing the ladder. It is also the cheapest item on the list, which makes it the obvious first action in every scenario.

And the column with the most "OK" entries is prompting, which is the practical restatement of why the ladder starts there.

Each scenario runs the six passes. All figures are constructed illustrations, not measurements.

Scenario A — Regulated bank, internal policy assistant.

Requirement: answer staff questions about internal policy, with the
             policy section cited. Policies revised monthly. Auditors
             must be able to trace any answer to its source.
Hardware:    2 × 80 GB GPUs available
Data:        14,000 policy documents; no Q&A pairs
Time:        12 weeks    Budget: generous

Pass 1 Governance: audit trail required → eliminates weights-only.
                   Traceability is architectural. → RAG mandatory.
Pass 2 Data shape: documents only, no pairs → no SFT possible anyway.
Pass 3 Hardware:   ample; not the binding constraint.
Pass 4 Latency:    internal tool, no hard SLA → RAG fine.
Pass 5 Time:       12 weeks is comfortable for a RAG build.
Pass 6 Budget:     not binding.

Recommendation: RAG with citation rendering, permission filtering, and monthly re-indexing. No fine-tune in phase 1. Pass 1 decided it before any other consideration, and Pass 2 confirmed there was no fine-tuning option to weigh. Note that the generous hardware is irrelevant — a common trap, where available capacity invites a technique the requirement forbids.

Scenario B — Startup, one GPU, needs a house voice.

Requirement: marketing copy in a distinctive brand voice, consistently.
             No factual grounding needed — inputs contain the facts.
Hardware:    one 24 GB GPU
Data:        2,200 human-approved examples of on-brand copy
Time:        6 weeks    Budget: minimal

Pass 1 Governance: nothing binding; the copy is the company's own.
Pass 2 Data shape: (brief, approved copy) pairs → SFT is available.
Pass 3 Hardware:   7B full fine-tune ≈ 84 GB → blocked.
                   7B LoRA bf16      ≈ 14 GB → fits with headroom.
Pass 4 Latency:    no retrieval needed; not binding.
Pass 5 Time:       curation is largely done; 6 weeks is feasible.
Pass 6 Budget:     LoRA training cost is trivial; ~12 GPU-hours.

Recommendation: prompt first for two weeks with versioned variants; if it plateaus, a LoRA adapter on the 2,200 pairs. Voice is exactly what imitation learning transfers, and Pass 3 makes LoRA the only weight-changing option that fits. State the give-up explicitly: the adapter will not make the model factually better, and any factual requirement that appears later needs retrieval.

Scenario C — Real-time assistant with a hard latency SLA.

Requirement: 150 ms p95 end-to-end, in-product suggestion.
             Answers drawn from a stable, rarely-changing set of rules.
Hardware:    ample inference capacity
Data:        900 (input, ideal suggestion) pairs
Time:        8 weeks    Budget: moderate

Pass 1 Governance: nothing binding.
Pass 2 Data shape: pairs available → SFT possible.
Pass 3 Hardware:   fine for LoRA.
Pass 4 Latency:    150 ms p95. Estimated retrieval hop plus the
                   inflated prefill would consume most of the budget
                   → RAG eliminated for the hot path.
Pass 5 Time:       8 weeks feasible.
Pass 6 Budget:     moderate; LoRA is cheap.

Recommendation: a LoRA adapter encoding the rules and the output format, no retrieval on the hot path. This is the case where the RAG heuristic gives the wrong answer, and it is worth recognising: rules are stable, so freezing them in weights costs nothing in freshness, and the latency budget forbids the hop. Give-up: rule changes now require a retraining run, so document the cadence and check it is acceptable. If rules turn out to change monthly, revisit — possibly with a small cached rule lookup rather than full retrieval.

Scenario D — Team that wants RLHF.

Requirement: "make the assistant more helpful," no sharper definition.
Hardware:    one 40 GB GPU
Data:        no preference data; no SFT pairs; a document corpus
Time:        10 weeks    Budget: no annotation budget

Pass 1 Governance: nothing binding yet.
Pass 2 Data shape: no preference data → RLHF and DPO blocked.
                   No pairs → SFT blocked.
                   Documents → RAG available.
Pass 3 Hardware:   RLHF needs 3–4 resident models (≈56 GB+ for 7B)
                   → blocked on one 40 GB card regardless.
Pass 4 Latency:    unstated; assume flexible.
Pass 5 Time:       a preference programme alone exceeds 10 weeks.
Pass 6 Budget:     no annotation budget → all labeled options blocked.

Recommendation: define "helpful" as a measurable eval slice first, then prompt and add RAG. RLHF is blocked four ways. This scenario appears constantly in real life and the answer is almost always the same: the requirement is not yet specified well enough to optimise, and the highest-value work is turning "more helpful" into something you can score. Also note the free alternative — an already-aligned instruct model inherits somebody else's alignment budget at no cost (11-06).

Scenario E — High-volume API with a long system prompt.

Requirement: same behaviour as today, cheaper.
Volume:      45,000,000 requests / month
Current:     3,200-token system prompt carrying format and few-shot examples
Data:        the few-shot examples plus 5,000 logged good responses
Hardware:    ample    Time: 10 weeks    Budget: moderate

Pass 1 Governance: nothing binding.
Pass 2 Data shape: pairs available → SFT possible.
Pass 3 Hardware:   fine.
Pass 4 Latency:    a shorter prompt *reduces* prefill → improvement.
Pass 5 Time:       feasible.
Pass 6 Budget:     compute the payback.

Tokens removable per request              3,200 − 200 = 3,000
Input price                        $0.15 / 1M tokens
Monthly saving = 3,000 × 45e6 × 0.15/1e6           = $20,250
One-off fine-tune (curation + train + eval)        = $22,000
Payback                = 22,000 / 20,250 ≈ 1.09 months
Annual saving thereafter                           ≈ $243,000

Recommendation: fine-tune (LoRA) to fold the system prompt into the weights. Here the cost argument genuinely works — a payback of about five weeks — because the removable prompt is large and the volume is high. Contrast with 11-08's example where a 350-token prompt at 2 million requests had a 15-year payback. Same technique, opposite verdict, and the arithmetic is what distinguishes them.

What the five scenarios have in common. In every case the decision was made by a constraint, not by which technique is best in the abstract, and in four of five the deciding constraint was found in Pass 1 or Pass 2 — before any cost model was built. That is the argument for the ordering.

The deliverable a senior reviewer can sign. Six sections, one page, and the arithmetic shown.

ADAPTATION STRATEGY RECOMMENDATION

1. REQUIREMENT (one sentence, testable)
   e.g. "Answer staff policy questions with the source section cited,
        against policies revised monthly."

2. CONSTRAINTS, CLASSIFIED
   Fixed:  audit trail required; data stays in-region
   Facts:  14,000 documents, no Q&A pairs; 2 × 80 GB GPUs
   Money:  12 weeks; budget approved to $X

3. ELIMINATION TRACE  (which pass killed which option)
   Full fine-tune  — eliminated Pass 1 (no provenance) and Pass 2 (no pairs)
   Alignment       — eliminated Pass 2 (no preference data)
   LoRA            — eliminated Pass 2 (no pairs)
   Prompt-only     — insufficient: no grounding in the corpus
   Surviving:      RAG (+ prompt engineering)

4. RECOMMENDATION
   RAG with citation rendering, permission filtering, monthly re-index.
   Prompt engineering for task framing. No weight changes in phase 1.

5. WHAT THIS GIVES UP  (stated, not hidden)
   - No control over house voice beyond what prompting achieves
   - Per-request token cost rises ~3× from retrieved context
   - Added retrieval latency: est. N ms at p95

6. WHAT WOULD CHANGE MY MIND  (the measurement)
   - If the eval set shows format inconsistency above X% after prompt
     work, add a LoRA adapter for format in phase 2.
   - If p95 latency exceeds the SLA, revisit retrieval depth or
     move stable rules into weights.

Sections 3, 5, and 6 are what make it defensible. Section 3 shows you considered the alternatives and why they were unavailable — which pre-empts the reviewer's first question. Section 5 is honest about the cost, which is what distinguishes a recommendation from a pitch. Section 6 names the measurement that would reverse the decision, which is what makes it an engineering claim rather than a preference.

Decision table: which constraint to check first, by symptom.

This is the blueprint's own framing of the associate role, almost verbatim. The official job-role description has the associate contributing to development, programming, and QA of generative-AI LLM systems on a team of senior professionals: developing datasets, selecting models, training models, implementing testing and debugging, and understanding deployment. The objectives are repeatedly phrased as assisting under the supervision of a senior team member. That phrasing tells you exactly what is being tested: recognising correct practice and correct tool choice, not designing novel research.

Concretely, this lesson serves the objectives on identifying the system data, hardware, or software components required to meet user needs, and on assisting with deployment and evaluation of scalability, performance, and reliability. Both are constraint-recognition objectives, and both are examined as scenario questions.

The exam's calibration reinforces this. Reported experience describes questions as general-level rather than deep-technical, with the winning posture being to know at a high level what each thing is and when to use it. Constraint elimination is precisely that skill, expressed as a procedure.

Question phrasings to expect:

• "A team has a single 16 GB GPU and needs to customise a 7B model. What is the most appropriate approach?" — parameter-efficient fine-tuning, likely with a quantised base.
• "A requirement states that a customer's data must be removable on request. Which approach satisfies this?" — RAG, because index records can be deleted.
• "A team has 50,000 documents and wants a model that answers questions about them. What should they do first?" — RAG; they have no labeled pairs for fine-tuning.
• "Which factor most limits a team's ability to full-fine-tune a large model on-premises?" — GPU memory, driven by gradients and optimizer state.
• "A product has a hard 150 ms latency budget. Which approach is least suitable?" — RAG, due to the retrieval hop and longer prefill.
• "What should a team do before any adaptation work?" — build an evaluation set and test the base model's capability.
• "Which is the long pole in a fine-tuning project?" — data curation, not the training run.
• "A team wants RLHF but has no preference data or annotation budget. What is the appropriate recommendation?" — define the target measurably and use cheaper rungs; RLHF is not feasible.

Distractor families:

Whether you can measure the outcome. Everything else is downstream of that, and it is not a formality — it is the constraint that blocks every option in the matrix.

Without an evaluation set that predates the change you cannot tell whether prompting plateaued, cannot detect catastrophic forgetting (11-03), cannot select a checkpoint during alignment (11-07), cannot know when to stop climbing the ladder, and cannot defend any recommendation to a reviewer. It is also the cheapest artifact in the project — a small set of hand-written items with expected outputs, built in days.

The correct opening sequence, in order:

1. Write down the requirement as something testable. "More helpful" is not a requirement; "answers under 120 words, citing a source section, refusing legal questions" is.
2. Build a small eval set covering it, plus general-capability and safety slices. 01-08 for construction, 09-01 for scaling it later.
3. Score the base model on it, zero-shot and few-shot. This is the capability check and may end the project happily.
4. Then run the six constraint passes.
5. Then compare the survivors on merit and cost.

Teams that reverse steps 2 and 4 spend weeks arguing about technique without a shared definition of success. Teams that skip step 3 adapt models that were never capable of the task.

Usually you do not decide: hardware decides for you. From the arithmetic in 11-04, a 7B full fine-tune under Adam needs on the order of 84 GB of static memory, which is a multi-GPU job, while the same adaptation with LoRA lands near 14 GB, or near 4 GB with a 4-bit frozen base. If you have one GPU, LoRA is the only weight-changing option available, and the question is answered before it is asked.

Where you genuinely have the hardware to choose, the considerations are:

Note the last row of the left column and the second-to-last of the right: escalate only on evidence. A rank and target-module sweep with an eval set is far cheaper than a full fine-tune, so exhaust it first. 11-05 is the procedure.

There is a third option that people forget: a different base model. If LoRA on a 7B model falls short, a LoRA on a stronger base is usually a better bet than a full fine-tune of the weak one, because it raises the capability ceiling instead of pushing harder against it.

Yes, and governance is the most common reason. If a requirement says a user can demand deletion of their data, or that consent is revocable, then training that user's data into weights creates an obligation you cannot discharge — there is no supported unlearning operation, only retraining from a prior checkpoint without the data. Having 4,000 perfect training pairs and eight idle GPUs does not change that; the constraint is legal, not technical (13-05).

Three other cases where capability and capacity are present and fine-tuning is still wrong:

• The facts change. Weights freeze at training time. A fine-tune of a monthly-revised policy set is stale by design, and the freshness requirement is not something more data fixes.
• Answers must be traceable. Trained weights carry no provenance. If an auditor needs the source of an assertion, the architecture must supply it, and only retrieval does.
• The requirement is still moving. Early in a product, the ability to change your mind in minutes is worth more than the marginal consistency a fine-tune buys. Prompting is reversible; a fine-tune is a commitment. Fine-tuning a behaviour you have not validated by prompting bakes in a decision you had not finished making.

The general principle to leave with: feasibility is necessary but not sufficient. "We can" and "we should" are different questions, and the constraint passes exist to keep the second from being answered by the first.

• Adaptation selection is elimination before optimisation. Constraints remove options; merit chooses among the survivors.
• Run the six passes in order of authority: governance, data shape, hardware, latency, time, budget. In four of the five worked scenarios the decision was settled in the first two passes, before any cost model.
• Governance clauses eliminate weight changes outright. Deletion rights, revocable consent, per-tenant isolation, and audit trails are architectural, not negotiable, and they all point toward retrieval.
• The data-shape test resolves most plans: documents are not fine-tuning data. Pairs are. Comparisons are preference data. Having "lots of data" says nothing until you name its shape.
• Hardware usually decides LoRA versus a full fine-tune for you — 14 GB against 84 GB for a 7B model, or about 4 GB with a 4-bit frozen base.
• A hard latency SLA can eliminate RAG on the hot path, which is a genuine counter-case to the RAG heuristic and worth recognising in a scenario.
• Data curation is the long pole, not the training run. Schedules that budget for the GPU hours and not the labelling slip in the data phase.
• Two constraints block every option: an insufficient base model and a missing evaluation set. Check both first; the eval set is also the cheapest item on every list.
• When everything is eliminated, the legitimate responses are relax the requirement, change the model, buy the constraint away, reduce scope, or state that the project is infeasible as specified. Build the eval set regardless.
• Deliver a one-page recommendation with an elimination trace, an explicit give-up statement, and a reversal condition. That combination is what a senior reviewer can actually sign.
• Feasibility is not sufficient justification. "We can" and "we should" are different questions, which is the entire reason the passes come before the arithmetic.

Every memory figure in this module rested on an assumption stated in passing: two bytes per value for bf16, four for fp32, half a byte for a 4-bit base. Those numbers are not interchangeable conveniences — the formats differ in exponent range and mantissa precision, and picking the wrong one produces training runs that diverge, or inference that quietly loses accuracy in a way no arithmetic warned you about. Next: 12-01 opens the deployment and optimisation module by taking the precision formats apart, so that every byte count you have used in this module has a justification underneath it.