- CLIPContrastive Language-Image Pretraining20%
- A model that jointly trains an image encoder and a text encoder with a contrastive loss, so matching image-text pairs land close together in a shared embedding space. It enables zero-shot image classification and provides the context embeddings that condition text-to-image diffusion models — no task-specific fine-tuning required.
- Residual connection20%
- An identity add of a layer's input to its output (F(x) + x), introduced in ResNet. It gives gradients a direct path backward, which mitigates vanishing gradients and is what makes very deep networks — including transformers and the U-Net — trainable at all.
- Contrastive loss20%
- A loss that pulls matched pairs together and pushes mismatched pairs apart in an embedding space, most commonly seen as InfoNCE in CLIP's image-text training. It is the mechanism behind CLIP's shared embedding space.
- Model fusion20%
- How and when a multimodal model combines its inputs. Early fusion merges raw or low-level features and tends toward lower inference latency; late fusion merges each modality's separate output or decision and is robust to a missing modality, often at the highest accuracy; intermediate fusion merges in hidden layers as a practical middle ground.
- Attention map10%
- A heatmap rendering of a model's attention weights over image regions or tokens, showing where the model focused when producing a prediction or caption. Useful for explainability and debugging, but it shows where weight went, not a guaranteed causal account of the model's reasoning.
- Data augmentation10%
- Artificially expanding a training set by creating modified copies of existing data — image flips, crops, and color jitter; audio time-stretch and noise; text paraphrase and back-translation. It improves accuracy and reduces overfitting, especially with limited data.
- Diffusion modelDenoising Diffusion Probabilistic Model (DDPM)25%
- A generative model trained in two phases: forward diffusion adds Gaussian noise to data step by step, and a learned reverse process removes that noise to reconstruct a sample, letting the model generate new images starting from pure random noise.
- FIDFréchet Inception Distance25%
- A metric that compares the feature-space statistics of generated images against real ones. Lower FID means the generated images are statistically closer to real images — the direction candidates most often get backward.
- GANGenerative Adversarial Network25%
- A generative model built from two networks trained against each other: a generator that creates samples and a discriminator that tells real from fake, trained as a min-max game. GANs produce high-quality samples quickly but with weaker sample diversity than diffusion, and use no denoising step.
- Riva25%
- NVIDIA's GPU-accelerated, multilingual speech microservices for real-time conversational AI, chaining automatic speech recognition (ASR), an NLP/LLM component, and text-to-speech (TTS). It also offers neural machine translation and can scale in Kubernetes via a Helm chart.
- Context embedding15%
- A conditioning vector — typically CLIP's text encoding of a prompt — that steers what a diffusion model generates. Testing and refining the embedding is how a candidate is expected to reach a desired image output.
- Shared embedding space15%
- A common vector space, built by a model like CLIP, in which representations from different modalities are positioned so that cross-modal similarity is meaningful — the property that makes zero-shot classification and text-to-image conditioning possible.
- Autoencoder15%
- A network that compresses input to a bottleneck representation (encoder) and reconstructs it (decoder), trained on a reconstruction loss. Because it learns normal patterns well, a high reconstruction error on new input flags it as an anomaly.
- Mixed-precision training10%
- Training with FP16 arithmetic alongside FP32, cutting memory and speeding computation with minimal accuracy loss — achieved via an FP32 master weight copy, loss scaling to prevent small gradients underflowing in FP16, and FP32 accumulation of products.
- Quantization10%
- Reducing the numeric precision of weights and activations, typically to INT8, to shrink memory and speed up inference. It does not itself improve accuracy; quantization-aware training (QAT) recovers more accuracy than post-training quantization (PTQ) by simulating the reduced precision during training.
- U-Net15%
- An encoder-decoder convolutional network shaped like a U, with skip connections carrying fine spatial detail from encoder layers to matching decoder layers. It functions as a standalone autoencoder and, run repeatedly, as the denoising backbone inside a diffusion model.
- ACEAvatar Cloud Engine15%
- NVIDIA's platform for building digital avatars, combining Riva ASR (speech-to-text), Riva TTS (speech synthesis), Audio2Face (facial animation and lip sync), and a NeMo LLM (understanding and generating responses) into one pipeline.
- Content authenticity5%
- The multimodal-specific trust concern that generative models can produce convincing synthetic images, audio, and video. Building trustworthy generative systems includes provenance and watermarking of AI-generated media, disclosure that content is synthetic, and detection of manipulated content.