NVIDIA · NCA-GENM

The multimodal-measurement thread

Judging a multimodal model honestly. Opens in M1 with classic comparison metrics, runs through M2's chart selection and attention-map caveats and M3's per-task metrics and FID, and closes in M7 where disaggregated bias evaluation and a hallucination/grounding checklist turn measurement into an audit trail.

NCAM-T2 · 32 lessons across 4 modules

  1. M1M1-01Machine learning fundamentals: learning paradigms, feature engineering, and cross-validation
  2. M1M1-02Overfitting, underfitting, and the bias-variance tradeoff
  3. M1M1-03Model comparison metrics: accuracy, precision, recall, F1, ROC-AUC, MAE, MSE, R²
  4. M1M1-04Deep learning frameworks: TensorFlow, PyTorch, and Keras
  5. M1M1-05Neural network basics: neurons, activation functions, and the training loop
  6. M1M1-06Convolutions and the building blocks of vision models
  7. M1M1-07Nonsequential networks and residual connections
  8. M1M1-08Multimodal loss functions: cross-entropy, contrastive, reconstruction, adversarial, and composite
  9. M1M1-09Training stability in multimodal settings: normalization, LR warmup, loss weighting, gradient clipping
  10. M1M1-10Multimodal transfer learning: pretrained encoders, full fine-tuning vs. parameter-efficient adaptation
  11. M1M1-11Model fusion and orchestration: early, intermediate, and late fusion; modality vs. agent orchestration
  12. M1M1-12Prompt engineering fundamentals and emerging multimodal trends
  13. M2M2-01Data cleaning: missing values, outliers, scaling, and categorical encoding
  14. M2M2-02Exploratory data analysis: descriptive statistics and correlation
  15. M2M2-03Choosing the right chart and avoiding misleading visuals
  16. M2M2-04Attention maps as an explainability and debugging tool in multimodal settings
  17. M2M2-05Preparing multimodal data: augmentation and OCR for PDF extraction
  18. M2M2-06Identifying relationships, trends, and confounding factors in an analysis
  19. M3M3-01Experiment design: one variable at a time, A/B testing, fixed evaluation sets, reproducibility
  20. M3M3-02Managing and preprocessing multimodal data from multiple sources
  21. M3M3-03Diffusion models: forward noising and reverse denoising (DDPM)
  22. M3M3-04Controlling diffusion output with context embeddings
  23. M3M3-05GANs and the generator-discriminator min-max game
  24. M3M3-06Evaluating generative image quality: Fréchet Inception Distance (FID)
  25. M3M3-07Conversational AI on Riva: ASR, NLP/LLM, and TTS pipelines
  26. M3M3-08Evaluation metrics by task: classification, regression, text generation, image generation, and RAG
  27. M3M3-09Explainability and testing data/model quality and consistency
  28. M7M7-01Ethical principles of trustworthy AI: privacy, safety, transparency, nondiscrimination
  29. M7M7-02Minimizing bias: disaggregated evaluation and mitigation
  30. M7M7-03Data privacy vs. data consent
  31. M7M7-04Content authenticity for multimodal generative AI: provenance, watermarking, disclosure, detection
  32. M7M7-05Hallucination, grounding, guardrails, and a trustworthy-AI checklist

Part of the throughlines running across the NCA-GENM prep course.