TensorFlow vs. PyTorch vs. Keras: The Three Deep Learning Frameworks Explained

Identity statement: TensorFlow and PyTorch are the two dominant deep learning frameworks; Keras is a high-level API built on top of TensorFlow, not a separate, competing framework. [GROUND TRUTH] (Sources/nca-genm/domain-1-core-ml-ai.md) states this directly: "Objective 1.10 calls out TensorFlow and PyTorch as the two dominant deep-learning frameworks... Keras is a high-level API (on TensorFlow)."

All three exist to solve the same underlying problem: building and training a neural network requires representing multi-dimensional numeric data (tensors), computing gradients of a loss function with respect to potentially millions of parameters (automatic differentiation, or autograd), and running the resulting arithmetic fast enough to be practical (GPU acceleration). [GROUND TRUTH] (Sources/nca-genm/domain-1-core-ml-ai.md) names all three of these as shared capabilities: "Both provide tensors, automatic differentiation (autograd), GPU acceleration, and building blocks for layers/optimizers."

When it matters: any scenario naming a specific framework and asking what it is, whether it competes with another named tool, or which one a described use case would favor.

The one differentiating fact the exam most wants you to know is the type of computation graph each framework historically favors. [GROUND TRUTH] (Sources/nca-genm/domain-1-core-ml-ai.md) states it plainly: "PyTorch is favored in research for its dynamic (define-by-run) graphs."

A dynamic, define-by-run graph means the computation graph — the sequence of operations connecting inputs to outputs — is built on the fly, as the code actually executes, rather than being defined once in advance and then run repeatedly. This matters for research because it lets a model's structure change from one run to the next based on ordinary code logic (a loop, a conditional), which fits naturally with the kind of rapid, exploratory experimentation research work usually involves — you can insert a debugger, print an intermediate tensor's shape, or change the network's structure mid-experiment without redefining a separate, static graph object first.

This is a relative framing, not an absolute one: it names what PyTorch is favored for, not a claim that TensorFlow is incapable of dynamic behavior or that PyTorch cannot be used in production. Reading it as "PyTorch = research only, TensorFlow = production only" overstates a real but narrower fact about historical default use and workflow convenience.

Keras is not a rival to TensorFlow the way PyTorch is — it sits on top of TensorFlow, offering a simpler, higher-level way to define common network structures (stacking layers, choosing an optimizer, calling .fit()) without writing the lower-level TensorFlow operations directly. Thinking of the relationship as three peers — TensorFlow, PyTorch, Keras — rather than two peers plus one layer built on the first is the single most common misreading this topic produces, and it is worth correcting explicitly before it becomes a wrong exam answer: Keras and TensorFlow are not alternatives to each other; choosing Keras is choosing a more convenient way to use TensorFlow.

Section 1 lists GPU acceleration as one of the three capabilities TensorFlow and PyTorch both provide, but it is worth being precise about what that phrase means mechanically, because a later NVIDIA-specific SDK sits directly underneath it. Neither framework talks to a GPU's hardware directly for every individual operation it runs; instead, both call into lower-level, GPU-accelerated primitive libraries that implement the actual arithmetic — convolutions, pooling, and similar operations — as highly optimized GPU kernels. [VENDOR SPEC] (Sources/nca-genm/domain-6-software-development.md) names this library explicitly for NVIDIA hardware: cuDNN (CUDA Deep Neural Network library) provides "GPU-accelerated primitives (convolutions, pooling) under frameworks," and the source material calls out directly that cuDNN "is a low-level GPU primitives library used by frameworks" — not a model, and not a server in its own right.

The practical layering, from the top down: you write PyTorch or TensorFlow code describing a network's structure; the framework translates that structure into a sequence of lower-level tensor operations; and for the operations that run on an NVIDIA GPU, cuDNN supplies the actual optimized implementation the framework calls into. This is why "GPU acceleration" is listed as a shared capability of both frameworks rather than a distinguishing feature between them — both frameworks are, for the NVIDIA-hardware case, ultimately built on the same class of underlying primitive library, even though each framework wraps that acceleration in its own API and its own graph-execution model.

Frameworks versus the serving layer, a distinction this lesson's scope stops short of

It is worth flagging one more relationship explicitly, even though its full treatment belongs to a later module: a framework (TensorFlow, PyTorch) is what you use to build and train a model, which is a different job from what serves a trained model in production. [VENDOR SPEC] (Sources/nca-genm/domain-6-software-development.md) names Triton Inference Server as the tool that serves models "across frameworks (TensorRT, PyTorch, TF, ONNX) with dynamic batching, concurrency, versioning, metrics" — meaning Triton is explicitly designed to serve models that were originally built in any of several different frameworks, TensorFlow and PyTorch both included. The exam-relevant boundary to hold onto here, ahead of its full treatment later in this course, is that "framework" and "serving layer" name two different jobs in a model's lifecycle, and Triton's multi-framework support is itself evidence that TensorFlow and PyTorch are not mutually exclusive choices at the level of an organization's overall tooling — a team can train in one framework and still serve through the same infrastructure as a team that trained in the other.

Section 1's shared-capability list from [GROUND TRUTH] (Sources/nca-genm/domain-1-core-ml-ai.md) ends with a fourth item worth calling out on its own: "building blocks for layers/optimizers." Tensors, autograd, and GPU acceleration are the infrastructure a framework provides; layers and optimizers are the higher-level pieces built on top of that infrastructure that let you actually assemble a network without writing the underlying tensor arithmetic by hand every time.

A layer is a pre-built, reusable unit of computation — a fully connected layer, a convolutional layer, an attention layer — that a framework ships ready to use, so a model architecture can be assembled by composing named layers rather than writing raw matrix operations from scratch for every network. An optimizer is the algorithm that takes the gradients autograd computed and decides how to update the model's weights — stochastic gradient descent (SGD) and Adam are the two most commonly named optimizers, and both TensorFlow and PyTorch ship several optimizer implementations ready to use out of the box, rather than requiring you to implement the update rule yourself.

This is precisely the level at which Keras's convenience becomes visible: Keras's Sequential and functional APIs are, in large part, an especially streamlined way of composing TensorFlow's own layer and optimizer building blocks, which is one more angle on why Keras is correctly described as sitting on top of TensorFlow rather than existing as an independent alternative to it — the "building blocks" TensorFlow already provides are the same building blocks Keras arranges into a friendlier interface.

Rather than a deep architectural comparison — out of scope for an associate-level exam — the comparison worth holding is a short table of what each tool is and what it is known for.

Two rows of that table carry essentially all of the testable content. First, TensorFlow and PyTorch are both full frameworks providing the same three core capabilities from section 1 — tensors, autograd, GPU acceleration — so a question that frames them as fundamentally incompatible in what they can do is testing a false premise; the real distinction is workflow style and ecosystem convention, not raw capability. Second, Keras's row is not really a peer comparison at all — its "graph style" column reads "inherits TensorFlow's underlying graph" precisely because it is not a separate execution engine, it is a friendlier way to write code that TensorFlow ultimately executes.

A related distractor worth naming explicitly: a question may describe a team switching a project from TensorFlow to PyTorch (or the reverse) partway through development and ask what has to happen to the model's learned weights. The correct answer is that raw framework-native weight files do not transfer directly — each framework serializes its own tensors in its own format — but this is a tooling and file-format fact, not evidence that the two frameworks compute anything differently in principle. Exporting to a framework-neutral format such as ONNX is the standard bridge, and it is the same bridge Triton uses to serve models trained in either framework side by side, which is exactly the point section 6's Q&A makes about the serving layer.

Why the research-versus-production framing is a tendency, not a hard rule

[GROUND TRUTH] (Sources/nca-genm/domain-1-core-ml-ai.md) frames PyTorch's research favor specifically around its dynamic graph property, and it is worth being precise about what that claim does and does not say. It does not say TensorFlow cannot be used for research, and it does not say PyTorch cannot be deployed to production — both frameworks have matured to support use cases outside their historically favored niche. What the exam is testing is the mechanism behind the historical association: a dynamic, define-by-run graph is more convenient for the kind of exploratory, frequently-changing experimentation research work involves, which is why PyTorch earned that reputation in the first place, not an absolute restriction on what either framework is capable of running.

⭐ THE EARNED INSIGHT The three-way comparison this topic seems to invite is a trap in miniature: TensorFlow and PyTorch are genuinely two peers solving the same problem with different default workflows, but Keras is not a third peer at all — it is a convenience layer sitting on top of one of the two. Answering a question about "the three frameworks" as though all three sit at the same level is the single fastest way to lose a point here, because the correct mental model has two peers and one API built on top of one of them, not three siblings.

A team describes their setup: "We wrote a training loop in PyTorch where the number of layers the input passes through changes based on the input's own content, decided by an if statement inside the forward pass." What does this describe?

Key detail: the network's structure changes at runtime, based on ordinary
code logic (an if statement), not a graph defined once in advance.

-> This is exactly the dynamic, define-by-run behavior PyTorch is
   favored for. The scenario is illustrating the mechanism, not
   asserting a specific measured benchmark.

Contrast with: "We defined a fixed sequence of layers using Keras's Sequential API and called .fit()." This describes using Keras's high-level API to define a straightforward, unchanging architecture — the kind of setup Keras's convenience layer is built for, running on top of TensorFlow underneath.

A platform team describes their production setup: "Our research group trains vision models in PyTorch and our NLP group trains language models in TensorFlow. Both teams export their trained models to be served through the same Triton Inference Server instance, and both rely on cuDNN-accelerated convolution and pooling operations during training." Identify what each named tool is doing.

PyTorch     -> the FRAMEWORK the vision team trains in (dynamic graph,
               favored for the vision team's research-style iteration)
TensorFlow  -> the FRAMEWORK the NLP team trains in (a different team's
               tooling choice, not a "wrong" one — both are dominant frameworks)
cuDNN       -> the low-level PRIMITIVES LIBRARY both frameworks call into
               for GPU-accelerated convolution/pooling during training —
               neither team is calling cuDNN directly, both frameworks are
cuDNN       -> UNDERNEATH both frameworks, not a competitor to either one
Triton      -> the SERVING layer downstream of training, explicitly built
               to serve models from multiple frameworks side by side

This is a constructed scenario, illustrative of the relationships rather than a real deployment. Notice that nothing in this description forces either team to switch frameworks to interoperate with the other — Triton's multi-framework serving support and cuDNN's role as a shared underlying primitives library are exactly what make a PyTorch-trained model and a TensorFlow-trained model coexist smoothly in the same production infrastructure. A scenario that asks "which framework should the NLP team switch to, to match the vision team" is testing whether you understand that no such switch is required — that pressure would be needed if TensorFlow and PyTorch could not both feed into the same serving and acceleration stack, but they can.

Each row names a specific symptom against a specific, checkable fix.

[GROUND TRUTH] (Sources/nca-genm/domain-1-core-ml-ai.md) names this material under objective 1.10, inside Core Machine Learning and AI Knowledge at 20% exam weight. The scope note for this domain is explicit that the exam expects recognition rather than hands-on implementation — you are not asked to write PyTorch or TensorFlow code, only to correctly identify what each tool is and how the three relate. The most common question shape names one of the three tools and asks either what it is, or how it relates to one of the other two, with the Keras-is-not-a-peer relationship being the single highest-value fact to have memorized cold.

A second recognizable shape draws on the cross-domain connection to Domain 6's SDK material: a scenario names cuDNN or Triton alongside TensorFlow or PyTorch and asks you to correctly place each tool at its layer — framework, primitives library, or serving layer. The keyed answer for this shape is almost always a straightforward layer-identification exercise once you hold the mental model from section 4 firmly: framework on top, primitives library underneath, serving layer downstream of both.

What the distractors typically look like

The reliable distractor families here: offering Keras as a genuine third alternative alongside TensorFlow and PyTorch, rather than a convenience layer on one of them; describing the research-versus-production association as an absolute limitation rather than a historical tendency tied to a specific mechanism (the dynamic graph); and placing cuDNN or Triton at the wrong layer — for instance, describing cuDNN as a competing framework, or Triton as a training tool rather than a serving tool.

No. Keras is a high-level API built on top of TensorFlow, offering a simpler interface for defining and training common network architectures. It is not a competing framework the way PyTorch is; using Keras means using TensorFlow through a more convenient layer, not choosing a third independent option.

Yes. Serving is a separate job from training, handled by a different layer of the stack — Triton Inference Server is explicitly designed to serve models trained in multiple different frameworks (including TensorRT, PyTorch, TensorFlow, and ONNX) side by side, with shared infrastructure for batching, concurrency, versioning, and metrics. Choosing PyTorch for one project and TensorFlow for another does not create an interoperability problem at the serving layer; it only means each model was trained with a different framework's workflow before both were handed off to the same downstream serving tool. The framework choice matters during training and experimentation, where a team's workflow preferences and the dynamic-versus-static graph tradeoff are genuinely relevant; it stops mattering once a model has been exported and handed to a serving layer designed, from the start, to be framework-agnostic.

• TensorFlow and PyTorch are the two dominant deep learning frameworks, both providing tensors, automatic differentiation, and GPU acceleration.
• PyTorch is favored in research for its dynamic, define-by-run computation graphs — the network's structure can change at runtime based on ordinary code logic.
• Keras is a high-level API on top of TensorFlow, not a third independent framework — never treat it as a peer competitor to TensorFlow or PyTorch.
• Both frameworks provide layers and optimizers as reusable building blocks, not just raw tensor and autograd primitives — this is what lets a network be assembled from named components rather than hand-written arithmetic.
• cuDNN sits underneath both frameworks as a shared GPU-primitives library, and Triton serves models trained in either framework — a team's framework choice does not create an interoperability problem downstream.

This module now turns from the tools to the mechanics running inside them. Next: M1-05 covers neural network basics — neurons, activation functions, and the training loop's five load-bearing terms: loss function, gradient descent, learning rate, backpropagation, and optimizer.