TrenTorch Makes You Build PyTorch Instead of Importing It

TrenTorch is an open source curriculum that walks you through building a working machine learning framework on plain NumPy in 20 progressive steps, from basic math up to a real language model. The video scrolls through the project's GitHub README, which pitches it with one line: "Most people learn ML frameworks by importing them. We built ours by hand, then made it hit harder."

What the Project Is

TrenTorch is a rebuilt and extended version of TinyTorch, the teaching framework from Harvard's CS249r Machine Learning Systems course. The README states this directly: "This is our implementation of TinyTorch (Harvard CS249r), rebuilt in our own style and pushed further. Same bones, more muscle." The repo badges visible in the video show Python 3.10+, an MIT license, five contributors, and a dependencies badge that reads "just NumPy." There is no PyTorch or TensorFlow anywhere in the stack. The point is to recreate the machinery those libraries hide, not to wrap it.

How the Curriculum Is Structured

The work is split into 20 modules grouped into four missions, shown in the README's "What You'll Build" section:

• Mission: Image covers real computer vision on standard benchmarks, with Conv2d, pooling, and CNNs written from scratch on NumPy.
• Mission: NLP covers tokenization, embeddings, and multi-head attention, all hand-rolled.
• Mission: LLM has you build full GPT-style transformer blocks with real self-attention and real language generation.
• Mission: Inference covers profiling, quantization, KV-cache, and memoization, so the model runs fast rather than just training.

The narrator frames this as recreating historic milestones, starting at the original perceptron and ending at the modern transformer architecture.

The Design Philosophy

The README's "The Bricks" section explains how TrenTorch differs from its TinyTorch base: it keeps the same fundamentals but adds cleaner internals, sharper performance instincts, and implementations pushed past the original spec. Three design goals are listed: small enough to read in one sitting, with every op traceable back to raw NumPy; big enough to run the real architectures that production frameworks run on; and fully owned, meaning rebuilt, refactored, and hardened by the maintainers themselves. The section closes with "No black boxes. No import torch. Just the machinery, exposed."

Getting Started

The Quick Start shown on screen is a standard Python setup: clone the repo, create a venv, pip install -r requirements.txt, then pip install -e . followed by tren setup. The tren CLI gets added to your PATH, so after setup any terminal can run tren from any directory without activating the venv first. Instructions are provided for both macOS/Linux and Windows PowerShell.

Key Takeaways

• TrenTorch teaches ML by making you implement every layer of a PyTorch-style framework yourself, using only NumPy.
• It is based on TinyTorch from Harvard's CS249r course, then rebuilt and extended by its own maintainers.
• The 20 modules progress through four missions: computer vision, NLP, GPT-style language models, and inference optimization.
• You end up recreating real architectures, from the perceptron to the transformer, rather than toy examples.
• The project is MIT licensed and installs with a standard clone, venv, and pip workflow plus a tren CLI.

Resources

• TrenTorch -- the 20-module build-your-own-ML-framework curriculum shown in the video, NumPy only
• TinyTorch -- the Harvard CS249r teaching framework TrenTorch is based on
• Harvard CS249r ML Systems book -- the open source Machine Learning Systems textbook TinyTorch accompanies