NCA-GENL Concept Glossary

Every concept taught across the 106-lesson NVIDIA Certified Associate: Generative AI LLMs prep course, in one place: 100 terms, each with a plain definition and a link to the lesson that introduces it.

Next-token predictionC01

The training objective of an LLM — predicting the most likely next token given everything before it — which underlies all text generation.

Introduced in Next-token prediction: what a language model is trained to do

LLM parametersC02

A model's learned weights, where everything it "knows" is actually stored, as opposed to anything written in a prompt.

Introduced in LLM parameters: what they are and where knowledge is stored

Tensor shapesC03

The batch, sequence, and hidden-size dimensions that describe how data flows through a transformer — the single most depended-on concept in the course.

Introduced in Linear algebra for LLMs: dot product, cosine similarity, and NumPy shapes

Dot product and cosine similarityC04

The vector-algebra operations used to measure how similar two embeddings are.

Introduced in Linear algebra for LLMs: dot product, cosine similarity, and NumPy shapes

Cross-entropy lossC05

The loss function that scores how far a model's predicted next-token distribution is from the true token, minimized during training.

Introduced in Loss functions and cross-entropy explained

Gradient descent and backpropagationC06

The optimization procedure — computing gradients via backpropagation and stepping weights against them — by which a model's parameters are updated.

Introduced in Gradient descent and backpropagation

Train/validation/test splitsC07

Dividing data into disjoint sets so training, tuning, and final evaluation never leak into each other.

Introduced in Train, validation, and test splits

Train vs. validation loss gapC08

The gap between training and validation performance that signals overfitting or underfitting, read directly off a loss curve.

Introduced in Train, validation, and test splits

Text-to-numbers conversionC09

The basic fact that a model can only operate on numbers, so text must first be converted into a numeric representation.

Introduced in Why text must be converted to numbers

Tokens and subword tokenizationC10

The subword pieces produced by a tokenizer, which a model actually operates on instead of whole words.

Introduced in Tokens, vocabulary, and subword tokenization

Token countingC11

Counting how many tokens a piece of text becomes, since tokens — not words or characters — determine cost, context usage, and chunk sizing.

Introduced in Counting tokens: why tokens are not words

Tokenizer algorithmsC12

The specific subword algorithm a model family uses — BPE for GPT, WordPiece for BERT, SentencePiece for T5 — a fact to memorize, not derive.

Introduced in BPE vs WordPiece vs SentencePiece: which model uses which

Stemming, lemmatization, and stop wordsC13

Classic text-normalization techniques that reduce words to a root form or drop low-information words before further processing.

Introduced in Stemming vs lemmatization, and stop-word removal

Bag-of-words, TF-IDF, and n-gramsC14

Pre-embedding text representations that count word or n-gram occurrences, weighted by how distinctive a term is across documents.

Introduced in Bag-of-words, TF-IDF, and n-grams

Text embeddingsC15

Learned dense vectors that represent the meaning of a piece of text as a point in a continuous space.

Introduced in What text embeddings are: learned dense vectors

Token vs. sentence/document embeddingsC16

The distinction between an embedding for a single token and one that represents a whole sentence or document — conflating the two is a common applied-retrieval mistake.

Introduced in Token embeddings vs sentence and document embeddings

Choosing an embedding modelC17

The practical criteria — max sequence length, dimension vs. index memory, symmetric vs. asymmetric objective, domain vocabulary — for picking an embedding model.

Introduced in How to choose an embedding model

Vector arithmetic and word analogiesC18

The word2vec-era idea that vector offsets encode analogies (king − man + woman ≈ queen), taught mainly to correct the bad intuitions it creates.

Introduced in Vector arithmetic and word analogies (word2vec)

Self-attentionC19

The mechanism by which each token compares itself against every other token in the sequence, at a cost that grows quadratically with context length.

Introduced in Self-attention and why context length costs quadratically

Positional encodingC20

The signal added to token representations so a transformer, which otherwise has no notion of order, can tell which position each token occupies.

Introduced in Self-attention and why context length costs quadratically

Encoder/decoder architecturesC21

The three transformer shapes — encoder-only (BERT), decoder-only (GPT), encoder-decoder (T5) — and why you embed with one family and generate with another.

Introduced in Encoder-only vs decoder-only vs encoder-decoder models (BERT, GPT, T5)

Autoregressive generationC22

The generation loop in which a model produces one token at a time, feeding each output back in to produce the next.

Introduced in Autoregressive generation: how an LLM produces text

Sampling: temperature, top-k, top-pC23

The decoding controls that determine how deterministic or varied a model’s next-token choice is.

Introduced in Temperature, top-k, top-p, and greedy decoding

Context windowC24

The fixed maximum number of tokens a model can attend to at once, budgeted across prompt, history, and retrieved content.

Introduced in The context window: what it is and how to budget it

Zero-shot and few-shot promptingC25

Getting a model to perform a task from instructions alone, or from a handful of in-context examples, without any weight updates.

Introduced in Zero-shot vs few-shot prompting and in-context learning

Prompt structureC26

Organizing a prompt into instruction, context, and output-format sections so the model’s job is unambiguous.

Introduced in How to structure a prompt: instruction, context, and format

Chain-of-thought promptingC27

Prompting a model to emit intermediate reasoning before its answer — which can help, but is not a faithful trace of its actual computation.

Introduced in Chain-of-thought prompting: when it helps and when it misleads

Prompt templates and versioningC28

Treating prompts as versioned, tested artifacts rather than one-off strings — the precondition for regression-testing an LLM system.

Introduced in Prompt templates, versioning, and testing

Structured JSON outputC29

Techniques for getting an LLM to reliably return well-formed, schema-conforming JSON rather than free text.

Introduced in Getting structured JSON output from an LLM

Prompt vs. RAG vs. fine-tuneC30

The decision rule for choosing among prompting, retrieval-augmented generation, and fine-tuning to solve a given problem.

Introduced in Prompt, RAG, or fine-tune? A first decision rule

Sparse retrieval (BM25)C31

Keyword-based retrieval that ranks documents by lexical overlap with a query, measured as the baseline before dense retrieval.

Introduced in Sparse retrieval and BM25 keyword search

Dense retrievalC32

Retrieval that ranks documents by embedding similarity to a query; mismatched query/passage encoders return confident nonsense.

Introduced in Dense retrieval with embeddings

Limits of embedding searchC33

The systematic blind spots of embedding-based retrieval — negation, recency, source authority — that motivate hybrid search and reranking as corrections.

Introduced in Limits of embedding search: negation, recency, and authority

Vector databases and ANN indexesC34

Approximate-nearest-neighbor index structures (HNSW, IVF) and the vector databases built on them; unnecessary for small corpora, and often over-engineered.

Introduced in Vector databases and ANN indexes (HNSW, IVF)

Hybrid searchC35

Combining sparse (keyword) and dense (vector) retrieval scores to correct for the blind spots either has alone.

Introduced in Hybrid search: combining keyword and vector retrieval

RerankingC36

Re-scoring an initial retrieval candidate list with a more expensive cross-encoder model to improve final ranking quality.

Introduced in Reranking with a cross-encoder

Document parsing for RAGC37

Extracting usable text from real documents — PDFs, tables, scanned pages — the ingestion stage whose failures are silent.

Introduced in Document parsing for RAG: PDFs, tables, and silent failures

Chunking strategiesC38

Splitting documents into retrievable pieces — fixed-size, recursive, or semantic — where chunk size is a granularity decision distinct from the context-window ceiling.

Introduced in Chunking strategies for RAG: fixed, recursive, and semantic

RAG metadataC39

The distinction between what content gets embedded for retrieval and what additional metadata gets stored and returned with a retrieved chunk.

Introduced in Metadata in RAG: what to embed versus what to return

The RAG pipelineC40

The complete retrieval-augmented-generation pipeline as one multi-stage object whose end-to-end quality equals its single worst stage.

Introduced in The complete RAG pipeline, stage by stage

RAG failure diagnosisC41

The first diagnostic question in RAG debugging — retrieval failure or generation failure — which alone resolves about half of all debugging time.

Introduced in Debugging RAG: retrieval failure versus generation failure

Lost-in-the-middleC42

The tendency of LLMs to under-attend to information placed in the middle of a long context, which makes chunk order matter.

Introduced in Assembling context: chunk order and the lost-in-the-middle problem

Grounding and citationsC43

Tying a model's generated answer back to its retrieved sources with citations, and letting it say "I don't know" rather than fabricate an answer.

Introduced in Grounding, citations, and letting a model say I don't know

Multi-turn query rewritingC44

Rewriting a follow-up question using conversation history so a retriever receives a self-contained query instead of an ambiguous fragment.

Introduced in Multi-turn chat history and query rewriting

Vector index freshnessC45

Keeping a vector index current as the corpus changes; switching embedding models requires re-embedding the entire corpus.

Introduced in Keeping a vector index fresh: re-embedding and migration

RAG evaluation metricsC46

Metrics — faithfulness, relevance, context recall — that split RAG quality across the retrieval/generation boundary so a change shows where it helped.

Introduced in RAG evaluation metrics: faithfulness, relevance, and context recall

When RAG is the wrong toolC47

Cases where retrieval-augmented generation is not the right architecture at all, taught last so RAG is not over-applied.

Introduced in When RAG is the wrong tool

Deduplication and corpus cleaningC48

Removing near-duplicate and boilerplate content from a corpus before indexing, since a repeated footer can become the nearest neighbour of every query.

Introduced in Deduplication and corpus cleaning for RAG

Access control in RAGC49

Enforcing per-user permissions on what a retriever is allowed to return, a concern that depends on how the vector index is built.

Introduced in Access control and permissions in RAG retrieval

Dataset curationC50

Deliberately assembling a dataset for an LLM task, including the unanswerable or out-of-scope cases most curation efforts skip.

Introduced in How to curate a dataset for an LLM task

Data quality problemsC51

The class of dataset defects — label noise, leakage, class imbalance, distribution drift — that silently degrade a model trained or evaluated on it.

Introduced in Data quality problems: label noise, leakage, imbalance, and drift

Exploratory data analysis on textC52

Profiling a text corpus before using it, including measuring length distributions in tokens rather than characters.

Introduced in Exploratory data analysis (EDA) on a text corpus

Chart selectionC53

Choosing the right chart type — histogram, box plot, scatter, heatmap — for the question a dataset visualization needs to answer.

Introduced in Choosing the right chart: histogram, box plot, scatter, bar, heatmap, line

Aggregated vs. group-level visualizationC54

The fact that an aggregated chart can structurally hide group-level harm, making disaggregated visualization a fairness instrument.

Introduced in Choosing the right chart: histogram, box plot, scatter, bar, heatmap, line

GPU-accelerated data scienceC55

NVIDIA RAPIDS, cuDF, and cuML for running data-science workloads on the GPU, taught at recognition depth.

Introduced in NVIDIA RAPIDS: cuDF, cuML, and cuGraph for GPU data science

Pretraining vs. instruction tuningC56

The distinction between pretraining, continued pretraining, and instruction tuning as different stages of building a usable LLM.

Introduced in Pretraining vs continued pretraining vs instruction tuning

Supervised fine-tuning (SFT)C57

Fine-tuning a model on labeled input-output pairs, and specifically what SFT can and cannot change about a model’s behavior.

Introduced in Supervised fine-tuning (SFT): what it can and cannot change

Catastrophic forgettingC58

The failure mode where fine-tuning on new data destroys a model's prior capabilities, which an eval run before the fine-tune is needed to catch.

Introduced in Catastrophic forgetting when fine-tuning

LoRA and PEFTC59

Low-Rank Adaptation and other parameter-efficient fine-tuning methods that train a small number of extra weights instead of the full model.

Introduced in LoRA and parameter-efficient fine-tuning (PEFT)

GPU memory for trainingC60

The arithmetic of how much GPU memory training an LLM actually requires — naive estimates run far higher than techniques like LoRA make necessary.

Introduced in GPU memory requirements for training an LLM

RLHFC61

Reinforcement learning from human feedback: the mechanism that moves human judgments about output quality into a model's weights.

Introduced in RLHF: reinforcement learning from human feedback explained

Reward models and reward hackingC62

Models trained to score outputs for RLHF, and the failure mode where a policy learns to exploit the reward proxy rather than the real objective.

Introduced in Reward models, reward hacking, and preference data

Choosing an adaptation strategyC63

Selecting among prompting, RAG, and fine-tuning approaches under real constraints of cost, latency, and data availability.

Introduced in Choosing a model adaptation strategy under real constraints

Building an evaluation setC64

Hand-building a small, concrete evaluation set as the earliest step in an LLM project, later scaled up to a larger set.

Introduced in How to build an evaluation set for an LLM project

PerplexityC65

A metric measuring how well a language model predicts a held-out text sample; blind to instruction-following quality.

Introduced in Perplexity: what it measures and what it misses

Human evaluation and inter-annotator agreementC66

Using human raters with a scoring rubric to judge model outputs, and measuring how much independent raters agree.

Introduced in Human evaluation: rubrics and inter-annotator agreement

BERTScoreC67

An embedding-based evaluation metric that scores generated text against a reference by semantic similarity rather than exact overlap.

Introduced in BERTScore and embedding-based evaluation metrics

Choosing an evaluation metricC68

The decision process for picking the right evaluation metric for a task, where loss functions and explained-variance metrics are actually delivered.

Introduced in How to choose an evaluation metric: loss functions, R², precision vs recall

BLEU, ROUGE, and exact matchC69

Classic overlap-based text-generation metrics, and which one fits which kind of task.

Introduced in BLEU vs ROUGE vs exact match: which metric for which task

LLM-as-a-judgeC70

Using an LLM to score another model’s outputs, and the biases — position, verbosity, self-preference, rubric drift — that make this fail silently.

Introduced in LLM-as-a-judge: how it works and where it fails

Reproducibility at temperature 0C71

The fact that even temperature-0 decoding is not fully deterministic across runs, complicating reproducible LLM evaluation.

Introduced in Reproducibility: why temperature 0 is not deterministic

Cross-validationC72

K-fold and stratified cross-validation for estimating a model's performance more robustly than a single train/test split, and when not to use it.

Introduced in Cross-validation: k-fold, stratified, and when not to use it

A/B testing an LLM featureC73

Running a controlled experiment on live production traffic to measure whether an LLM feature actually improves an outcome.

Introduced in A/B testing an LLM feature in production

Regression testing in CI/CDC74

Automatically re-running an evaluation set against every change to an LLM system inside a CI/CD pipeline, so a regression fails the build.

Introduced in Regression testing an LLM system in CI/CD

Sample size and statistical significanceC75

The arithmetic of standard error and multiple-comparison inflation needed to tell whether an observed evaluation difference is real or noise.

Introduced in Sample size and statistical significance in LLM evaluation

HallucinationC76

An LLM generating fluent but false or unsupported content, and the different types this can take.

Introduced in Why LLMs hallucinate, and the types of hallucination

Public benchmarks and contaminationC77

Standard benchmarks like GLUE and MMLU, and the risk that benchmark data leaked into training (contamination) inflates a reported score.

Introduced in Public benchmarks (GLUE, MMLU) and data contamination

Zero-shot/few-shot capability testingC78

Testing what a base model can already do without changes — the cheapest experiment available, and often enough to make a larger project unnecessary.

Introduced in Zero-shot and few-shot capability testing

Error analysisC79

Systematically reviewing model failures to turn a raw evaluation score into a concrete, prioritized fix list.

Introduced in Error analysis: turning failures into a fix list

Numeric precisionC80

The numeric formats — FP32, TF32, FP16, BF16, INT8 — a model’s weights and activations can be stored and computed in, trading memory and speed against precision.

Introduced in Numeric precision: FP32, TF32, FP16, BF16, and INT8

QuantizationC81

Reducing a model’s numeric precision after training (PTQ) or during training (QAT), a quality intervention that requires re-running the eval set.

Introduced in Quantization: PTQ vs QAT and recovering accuracy

Reading loss curvesC82

Diagnosing a training run's health — overfitting, underfitting, instability — directly from its training and validation loss curves.

Introduced in Reading loss curves to diagnose a training run

Distributed training (AllReduce/NCCL)C83

Splitting training across multiple GPUs via data parallelism, synchronizing gradients with AllReduce over NCCL.

Introduced in Distributed training: data parallelism, AllReduce, and NCCL

KV cacheC84

The cached key/value attention states from previous tokens that make autoregressive generation memory-bound rather than compute-bound.

Introduced in The KV cache and why LLM generation is memory-bound

Inference batchingC85

Grouping multiple requests together at serving time — static, dynamic, or continuous — to improve GPU utilization.

Introduced in Batching for inference: static, dynamic, and continuous

PagedAttention and vLLMC86

An OS-inspired paging scheme for the KV cache, implemented in vLLM, that lets inference serving use GPU memory far more efficiently.

Introduced in PagedAttention and vLLM: virtual memory for the KV cache

ONNX/TensorRT compilationC87

Compiling a trained model into an optimized runtime format — ONNX, TensorRT, TensorRT-LLM — for faster inference.

Introduced in ONNX, TensorRT, and TensorRT-LLM: compiling a model for inference

LLM cost accountingC88

Calculating what an LLM feature costs per million tokens, per request, and per month — the number that often decides architecture.

Introduced in LLM cost per million tokens, per request, and per month

Latency and throughputC89

Serving performance measured as time-to-first-token, tokens per second, and tail latency (p95); batching improves throughput at the cost of the tail.

Introduced in Latency and throughput: TTFT, tokens per second, and p95

Triton Inference Server and NIMC90

NVIDIA's Triton Inference Server and NIM microservices for deploying a model into production serving.

Introduced in Deploying with NVIDIA Triton Inference Server and NIM

Production monitoring and driftC91

Instrumenting a deployed LLM to detect performance decay over time, using the same evaluation instrument that validated the original build.

Introduced in Monitoring an LLM in production and detecting drift

Trustworthy AI principlesC92

NVIDIA's stated trustworthy-AI principles, and the concrete instrument required to actually evidence each one.

Introduced in NVIDIA's four pillars of trustworthy AI and how to implement each

NeMo GuardrailsC93

NVIDIA's guardrails framework for content moderation around an LLM, valued because it produces an auditable log.

Introduced in NVIDIA NeMo Guardrails and content moderation for LLM applications

Prompt injectionC94

Attacks that hijack a model’s behavior via malicious instructions embedded in a prompt, or indirectly in retrieved documents — making corpus trust a security property.

Introduced in Prompt injection and indirect injection through RAG

Bias in AIC95

Sources of bias in AI systems and how to measure and mitigate them, findable only through per-slice, not aggregate, evaluation.

Introduced in Bias in AI: sources, measurement, and mitigation

Data privacy and the right to forgetC96

The fact that a model's weights cannot selectively "forget" one person's data once trained, the strongest practical argument for retrieval over fine-tuning on sensitive data.

Introduced in Data privacy, consent, and why model weights cannot forget

Transparency and model cardsC97

Documenting a model's intended use, limitations, and training data via a model card, for explainability and transparency.

Introduced in Transparency, explainability, and model cards for LLM systems

Trustworthy AI checklistC98

A concrete, employer-shareable checklist that operationalizes the trustworthy-AI principles for a specific LLM service.

Introduced in A trustworthy AI checklist for your own LLM service

AI energy use and carbonC99

The energy and carbon footprint of training and running AI systems — figures that are contested and vendor-reported, not settled science.

Introduced in AI energy use, carbon, and efficient inference

Reading AI research papersC100

How to read an AI research paper and keep up with new developments after the course ends.

Introduced in How to read an AI research paper and track new LLM trends