When to Fine-Tune: New Skills, Style, and Smaller Specialized Models

[GROUND TRUTH] (Sources/ncp-genl/domain-5-fine-tuning.md) states the positive case for fine-tuning in one sentence: "fine-tune when you need a new skill, style, or behavior baked in, or a smaller specialized model." That sentence names two structurally distinct reasons, and it is worth separating them, because they fail differently when the wrong tool is substituted.

Case one: a new skill, style, or behavior baked in permanently. This covers changes to how a model produces output rather than what facts it has access to — a consistent tone, a specific output format followed reliably across every request, a refusal pattern calibrated to a specific policy, a domain-specific reasoning style. The word "permanently" is doing real work: fine-tuning changes the weights, so the behavior travels with the model to every future request without needing to be re-specified, unlike a prompt instruction that must be repeated (and paid for, in tokens) on every single call.

Case two: a smaller specialized model matching a larger general model's performance on one narrow task. A large general-purpose model can often perform a narrow task well through prompting alone, but at a cost and latency the general model's size imposes on every request. Fine-tuning a smaller model specifically for that one narrow task can recover most of the large model's task-specific performance while running cheaper and faster, because the smaller model does not need to carry the large model's broad general capability to do the one thing it was fine-tuned for.

Both cases share the same underlying property: the requirement is about the model's behavior or its resource footprint, not about the facts it needs access to. That property is the diagnostic this lesson builds around, and it is the same distinction M5-02's alignment methods, M5-01's PEFT methods, and M5-03's contrastive training all serve — they are all ways of changing behavior, at different granularities and for different behavioral targets, and none of them is a mechanism for injecting facts.

L1 — Intuition: hiring a specialist versus handing someone a reference manual

Imagine a new hire who is generally competent but does not yet know your company's specific tone, format standards, or escalation policy. Two very different interventions are available depending on what is actually missing. If the new hire simply was never told the standard — nobody explained the format, nobody stated the policy — the fix is telling them, once, clearly: that is prompting's role, an instruction that fixes a gap in what they were told to do. If the new hire has been told the standard repeatedly and still cannot produce it reliably across a hundred different requests — the habit has not become automatic despite clear instructions — no amount of re-explaining fixes that; what changes it is practice specifically targeted at making the behavior automatic, which is what fine-tuning does to a model's weights.

Now imagine a different problem entirely: the new hire needs to look up a customer's account history, which changes every day and which nobody could have taught them in advance because it did not exist yet at training time. No amount of instruction or practice makes someone permanently "know" information that changes daily — the correct fix is giving them a lookup tool: a system they query at the moment they need the answer. That is RAG's role, and it is a categorically different kind of gap from either of the first two.

L2 — Mechanism: reading the requirement for its actual shape

[GROUND TRUTH] (Sources/ncp-genl/domain-5-fine-tuning.md) states the countervailing case directly: "prefer prompting/RAG ... when knowledge is dynamic, must be cited, or data/compute is limited." Combined with the positive case from section 1, the diagnostic reduces to three questions asked of the actual stated requirement, in order:

• Does the requirement concern facts that change over time, or that must be traceable to a specific source? If yes, RAG is structurally required regardless of anything else about the situation — fine-tuning bakes knowledge into frozen weights with no update path short of retraining, and no way to point at where a specific claim came from.
• If the requirement is not about facts, is it about behavior, style, or task-specialization that needs to hold reliably across every future request without being re-specified each time? If yes, and prompting alone has already been tried and has plateaued, fine-tuning is the structurally correct tool — this is case one or case two from section 1.
• Even having concluded fine-tuning is the structurally right tool, is there enough data and compute available to run it? If not, [GROUND TRUTH] (Sources/ncp-genl/domain-5-fine-tuning.md)'s own framing says prefer prompting or RAG anyway, as a practical constraint on top of the structural diagnosis — a correct diagnosis you cannot afford to act on is not actionable, and the professional answer under a genuine resource constraint is to fall back to what the resources on hand can actually support.

L3 — The exam-relevant edge case: a behavior-shaped symptom does not automatically mean fine-tuning is affordable or correct

The professional-level trap this domain sets is narrower than "know the three options." It is: a scenario can describe a symptom that looks behavior-shaped — inconsistent tone, a format the model does not reliably follow — while also stating a constraint that rules fine-tuning out anyway, such as "the team has 40 labeled examples" or "compute budget is one shared GPU for prototyping." The correct read is not "this is a behavior problem, therefore fine-tune" — it is "this is a behavior problem, and also the data/compute constraint stated in the scenario makes fine-tuning currently infeasible, so the answer is to exhaust prompting first, or accept a smaller improvement than fine-tuning could theoretically deliver." Conversely, a scenario can describe a knowledge-shaped symptom — the model does not know about last week's policy change — while offering "fine-tune on the updated policy documents" as a tempting-sounding option; the correct read there is that fine-tuning on a to-be-updated-again-next-month policy document produces a model that is wrong again the moment the policy changes next, which is exactly the freshness failure RAG exists to avoid.

All figures and scenarios below are constructed, not measurements of any real deployment. Three different teams report the same surface complaint — "the model's answers are inconsistent" — about three different systems.

Team A. A customer-support model is expected to always end responses with a specific compliance disclaimer, in a fixed wording, verified against 500 past transcripts where the disclaimer is present roughly 60% of the time despite the system prompt explicitly instructing it every time.

Diagnosis: behavior requirement, already prompted extensively, prompting
has plateaued at 60% compliance despite explicit instruction on every call
Correct tool: fine-tuning (SFT on transcripts demonstrating the disclaimer
consistently applied) — this is case one from section 1, and prompting has
already been ruled out by the stated 60% failure rate under explicit
instruction

Team B. A support model answers questions about current shipping policies, and "inconsistent" turns out to mean: correct as of three months ago, wrong today, because the company changed its shipping policy twice since the model's last training run.

Diagnosis: knowledge requirement — the facts themselves changed, not the
model's behavior around them
Correct tool: RAG, retrieving the current policy document at request time
Fine-tuning would "fix" this only until the next policy change, at which
point the same complaint recurs — a freshness failure fine-tuning cannot
structurally solve

Team C. A support model's answers vary in quality because the team has never actually specified, in the system prompt, what tone or level of detail is expected — different requests get wildly different framing because nothing has told the model what "good" looks like here.

Diagnosis: an underspecified instruction, not yet a behavior-plateau or a
knowledge gap — the model was never told what was wanted in the first place
Correct tool: prompting — write the missing instruction and measure whether
that alone resolves the inconsistency before assuming a deeper fix is needed

Step 4 — the point of running all three side by side. Identical surface language — "inconsistent answers" — produced three different correct tools, because the diagnostic question is never "what does the symptom look like," it is "what is actually missing: an instruction that was never given, a fact that changed, or a behavior that instruction alone has already failed to fix." Team C's mistake, if made, would be jumping straight to fine-tuning without first trying the cheap, reversible fix; Team B's mistake, if made, would be fine-tuning on facts that will be stale again next quarter; Team A is the only one of the three where fine-tuning is actually the correct, evidenced answer, and it is correct specifically because prompting was tried first and specifically failed.

⭐ THE EARNED INSIGHT

The exam's likely framing treats this as "pick prompting, RAG, or fine-tuning for the stated need," which invites memorizing a lookup table. The professional-depth trap underneath that framing is that the same symptom — inconsistent behavior, a wrong answer, a format failure — can be produced by three different underlying gaps, and only one of the three gaps is what fine-tuning actually fixes. Read past the symptom to the gap: is something missing that was never said (prompt it), missing because it changed (retrieve it), or missing because instruction alone has already failed to make it stick (fine-tune it). Fine-tuning is correct for exactly the third case, and offering it as the answer to the first two is this domain's standing distractor.

All figures below are a constructed scenario, not a measurement. A team wants to fine-tune a model to specialize in generating structured incident reports from raw log excerpts, a genuine case-two-shaped requirement: a smaller model matched to one narrow, well-defined task, exactly the pattern from section 1. The structural diagnosis, on the behavior alone, correctly points at fine-tuning.

Step 1 — the requirement, stated in full, including the resource facts a scenario question would give you.

Task: generate structured incident reports from raw logs
Team's available labeled examples: 45 (log excerpt, correct report) pairs
Team's available training compute: a single shared GPU, used part-time by
  three other teams, no dedicated training window

Step 2 — what 45 examples and shared, part-time compute actually support. M5-01's parameter-efficient methods reduce, but do not eliminate, a fine-tuning run's need for a training set large enough to teach a genuinely new specialization reliably, and M5-04's early-stopping discipline needs a held-out validation set of its own — carved out of an already-small pool of 45 examples, that leaves very little for either training or validation to work with. A run this data-constrained risks a model that has memorized its 30-or-so training examples rather than learned a generalizable report-writing pattern, which is exactly the overfitting failure M5-04 names, compounded here by having too little data to even measure it reliably on a meaningful held-out set.

Step 3 — the practical fallback this scenario's stated constraints point toward. [GROUND TRUTH] (Sources/ncp-genl/domain-5-fine-tuning.md) names limited data and limited compute as reasons to prefer prompting or RAG even when the underlying requirement is structurally fine-tuning-shaped. Here, a well-designed prompt template with a handful of the team's 45 examples used as few-shot demonstrations, rather than as a training set, can approximate much of the structured-output benefit immediately, using the same small pool of examples in a way that does not require any dedicated training compute at all.

Step 4 — what changes the answer, and what does not. If the team later accumulates several thousand labeled examples and secures a dedicated training window, the structural diagnosis from section 1 does not change — this remains a case-two, smaller-specialized-model requirement, and fine-tuning becomes the right call once the resource gate clears. What this worked example demonstrates is that the diagnosis and the actionability of that diagnosis are two separate checks, and a scenario question that states the resource facts explicitly is testing whether you weigh both, rather than stopping at "this looks behavior-shaped, so fine-tune."

Fine-Tuning is tied for third-largest domain on the NCP-GENL blueprint at 13%, and this decision closes the domain because it is the gate every other lesson in this module assumes has already been passed: M5-01 through M5-04 all teach how to fine-tune well, and none of them can be correctly applied to a problem that should never have reached a fine-tuning stage at all. [GROUND TRUTH] (Sources/ncp-genl/domain-5-fine-tuning.md) states both directions of the rule in one place — fine-tune for "a new skill, style, or behavior baked in, or a smaller specialized model," prefer prompting/RAG "when knowledge is dynamic, must be cited, or data/compute is limited" — which is exactly the shape a professional-level scenario question tends to take: a short situation naming one or more of these signals, asking which of the three approaches is the correct fit.

Expect this material in a few recurring shapes: a scenario naming a knowledge-freshness or citation requirement and testing whether fine-tuning is wrongly selected despite that explicit signal; a scenario naming a resource constraint (small dataset, limited compute) alongside a behavior-shaped symptom, testing whether the constraint is weighed at all; and a direct comparison question asking what fine-tuning changes that prompting and RAG structurally cannot — the keyed answer being the model's own weights, and therefore its behavior on every future request without per-call re-specification.

What the distractors typically look like

This domain's house style favors a real, well-described technique attached to the wrong requirement: offering "fine-tune on the latest documents" for a freshness problem that will recur at the next update (a real technique, wrong axis — this is RAG's job); offering "fine-tune" as the answer when the stated scenario explicitly says data or compute is scarce (correct diagnosis, ignored practical constraint); and offering "just prompt it more clearly" for a scenario that explicitly states prompting was already tried extensively and plateaued (the option that was already ruled out by the scenario's own stated facts).

Yes, and in practice they frequently are, because they address different halves of the same system rather than competing for the same job. A model can be fine-tuned for the permanent behavior a deployment needs — a specific tone, a specific output format, task-specialized reasoning for a narrow domain — while simultaneously being served with RAG for the facts that requirement also depends on. [GROUND TRUTH] (Sources/ncp-genl/domain-5-fine-tuning.md)'s framing of the two as answers to different questions — behavior versus knowledge — is exactly why combining them is not a contradiction: a fine-tuned model's weights determine how it behaves and reasons; RAG's retrieved context determines what facts it has in front of it at the moment of that reasoning.

Fine-tuning changes the model's weights; prompting and RAG both change only what is sent to the model at request time, leaving the weights untouched — the same weights-versus-input distinction [GROUND TRUTH] (Sources/ncp-genl/domain-5-fine-tuning.md) draws when it separates "fine-tuning changes weights" from prompting and RAG, which do not. That one structural fact is what makes fine-tuning permanent (the behavior travels with the model to every future request, with no per-call re-specification) and simultaneously what makes it unable to track facts that change after training (there is no update path short of retraining). Per that same [GROUND TRUTH] (Sources/ncp-genl/domain-5-fine-tuning.md) distinction, prompting and RAG's shared property — no weight change — is the mirror image: both stay instantly reversible and always current with whatever is sent at request time, but neither is automatically consistent across requests the way a trained-in behavior is, and neither can install a skill or specialization the model did not already have some capacity for.

• Fine-tune for two structural reasons only: a permanent behavior/style/skill change, or a smaller specialized model matching a larger general model on one task. [GROUND TRUTH] (Sources/ncp-genl/domain-5-fine-tuning.md) states both cases directly.
• Prefer prompting or RAG when knowledge is dynamic, must be cited, or data/compute is limited — three separate conditions, any one of which is enough to redirect the decision away from fine-tuning.
• Read past the symptom to the actual gap. The same surface complaint — inconsistent output — can be an unstated instruction, a stale fact, or a genuinely unfixed behavior, and only the third of those is fine-tuning's job.
• A correct structural diagnosis is not automatically actionable. A stated data or compute constraint can rule out fine-tuning even when the requirement is genuinely behavior-shaped.
• Fine-tuning and RAG are not mutually exclusive. A deployment can be fine-tuned for permanent behavior while served with RAG for the facts that behavior operates on.
• The weights are the whole distinction. Fine-tuning changes them, which is what makes a behavior permanent and simultaneously what makes fine-tuned knowledge go stale; prompting and RAG never touch them, which is what makes both instantly current and never automatically consistent on their own.
• This decision is the gate every other method in this module assumes has already been passed — PEFT, alignment, contrastive training, and early stopping are all tools for fine-tuning well, not arguments for fine-tuning at all.

Fine-Tuning has now covered its full territory: which parameter-efficient method to reach for, which alignment method needs which supporting piece, how retrieval's own embedding models get trained, how to know when a training run has gone far enough, and finally, whether to be running any of these methods at all. What comes next moves from changing a model to measuring one — the discipline of knowing whether any model, fine-tuned or not, is actually good at what it claims to do. Next: the Evaluation domain takes up exactly that question, starting from the metrics that decide whether a model's output is actually worth trusting.