NVIDIA · NCP-AAI

The resilience thread

Keeping an agent alive under failure. Opens in M2 with the Retry and Circuit Breaker state machines, runs through M3's profiling-vs-evaluation split and the accuracy/latency tradeoff, through M4's distributed-load profiling and cost-vs-availability balance, and closes in M8 where monitoring, per-step tracing, and online task-success regression catch what none of the earlier stages could predict in advance.

NCPA-T1 · 23 lessons across 4 modules

  1. M2M2-01Prompt chains, dynamic branching, and refining agent decision-making
  2. M2M2-02Integrating multimodal and generative models across text, vision, and audio
  3. M2M2-03Building and connecting custom tools, APIs, and functions
  4. M2M2-04The Retry pattern: transient faults, backoff, and idempotency
  5. M2M2-05Retry vs. Circuit Breaker: two resilience-pattern state machines and when each applies
  6. M2M2-06Streaming conversation flows and real-time feedback
  7. M3M3-01Evaluation pipelines and task benchmarks
  8. M3M3-02Profiling vs. evaluation: two different questions
  9. M3M3-03Observability for evaluation: tracing an agent run step by step
  10. M3M3-04The accuracy-vs-latency trade-off
  11. M3M3-05Structured feedback and the data flywheel
  12. M3M3-06Analyzing results to guide targeted optimization
  13. M4M4-01NVIDIA NIM as an agent's inference endpoint: latency budgets and failure handling
  14. M4M4-02Scaling with containers, Kubernetes, and load balancing
  15. M4M4-03Profiling performance and reliability under distributed load
  16. M4M4-04MLOps and governance: CI/CD, monitoring, and audit
  17. M4M4-05Balancing deployment cost against high availability
  18. M4M4-06The serving stack in context: NIM, TensorRT-LLM, Triton, and Kubernetes
  19. M8M8-01Monitoring dashboards and reliability metrics
  20. M8M8-02Logs, errors, anomalies, and per-step tracing
  21. M8M8-03Online task-success regression: benchmarking a live agent against prior versions
  22. M8M8-04Automated agent/policy versioning: tool contracts and prompt versions in production
  23. M8M8-05Uptime, transparency, and trust in live deployments

Part of the throughlines running across the NCP-AAI prep course.