NVIDIA · NCP-GENL

The model-efficiency thread

Making a trained model smaller and faster without wrecking it. Opens in M4 with quantization, distillation, pruning, and KV caching, runs through M7's parallelism families and Nsight profiling, and closes in M8 where dynamic batching, NIM, and Multi-Instance GPU turn those optimizations into a served deployment.

NCPG-T1 · 17 lessons across 3 modules

  1. M4M4-01Quantization: PTQ vs QAT vs GPTQ
  2. M4M4-02Knowledge distillation: the DistilBERT trio
  3. M4M4-03Pruning and structured 2:4 sparsity
  4. M4M4-04KV caching as the primary latency lever
  5. M4M4-05Streaming attention and TensorRT runtime optimization
  6. M4M4-06Encoder foundation models and masked language modeling
  7. M7M7-01The parallelism families: data, tensor, pipeline, sequence, context, and expert
  8. M7M7-02Tensor Parallelism vs Pipeline Parallelism: the #1 distractor pair
  9. M7M7-03Memory sharding: FSDP and ZeRO
  10. M7M7-04Mixed precision and Tensor Cores
  11. M7M7-05Gradient accumulation and effective batch size
  12. M7M7-06Profiling and troubleshooting with Nsight
  13. M8M8-01Dynamic batching vs. sequence batching in Dynamo-Triton
  14. M8M8-02NVIDIA NIM as an LLM-serving container: model formats and the TensorRT-LLM backend
  15. M8M8-03Concurrent model execution and instance groups
  16. M8M8-04Containerization, Kubernetes, and Multi-Instance GPU
  17. M8M8-05Model-type compute tradeoffs: encoder vs. decoder vs. encoder-decoder

Part of the throughlines running across the NCP-GENL prep course.