A 176 KB C Engine Ran the 2.78 Trillion Parameter Kimi K3 on One CPU in 8 GB of RAM

A project called kimi-k3-in-c ran Kimi K3, a 2.78 trillion parameter mixture-of-experts model, on a single CPU with a measured peak of 8.24 GB of RAM. The engine is 176 KB of portable C99 with no GPU, no BLAS, and no framework. The core idea is that 93% of the model consists of experts that never load into memory. They stay on the SSD and get read only when a token routes to them, which shrinks a 5.5 TB full-precision model to an 8.24 GB resident footprint with byte-identical output.

What Was Actually Demonstrated

The video is built around screenshots of the kimi-k3-in-c GitHub README and a viral post from Alvaro Cintas (@dr_cintas). The README banner reads "A 2.78-trillion-parameter model. One CPU. 8 GB of RAM." and lists the headline stats: 2.78T parameters, a 1.56 TB checkpoint on disk, and 8.24 GB peak RSS, measured. A terminal capture shows the binary running with a laptop preset on the prompt "The capital of France is" and producing 8 tokens in 261.5 seconds, about 32.7 seconds per token, with peak RSS for the whole run at 8.24 GB. The claim is inference correctness, not speed. The output is the same bytes you would get with the full model in memory.

How the Memory Reduction Works

Kimi K3 is a mixture-of-experts model, and most of its weights sit in experts that any single token never touches. Only 16 of 896 experts run per token, so the engine leaves roughly 1.45 TB of expert weights on the SSD and streams in only the experts the router selects, one layer at a time. An on-screen chart titled "How a model far too big for any desk ends up running on one" walks the numbers down in four steps: 5,560 GB for every weight at full precision (a multi-GPU server cluster), 1,560 GB for the 4-bit file you actually download (a 2 TB hard drive), 113 GB for what must stay in memory (a high-end workstation), and 8.24 GB for what was actually measured (an ordinary laptop). That is a 675x reduction, and the chart notes it answers with the same words at every step.

The Speed Cost

The trade is severe. The creator quotes 33 seconds per token, and he stresses the units: 33 seconds per token, not 33 tokens per second. A quoted post from Jun Song makes the same joke ("Not 33tok/s, it's 33s/tok"), and a reply captures the practical reality: "At this rate my kid will see the output of my first prompt years after my death." A benchmark table in the video, measured on a single 124-core workstation with RSS capped, shows how RAM buys speed without changing the answer: 26.5 s/token at 8 GB, 24.2 s/token at 32 GB, 19.8 s/token at 64 GB, and 5.6 s/token at 128 GB or more. More RAM only makes it faster. The output stays the same every time.

The Fine Print

Beyond the latency, the requirements are real. You need a 1.56 TB SSD to hold the quantized checkpoint, the engine is Linux only on x86-64, and the checkpoint download takes hours. The creator notes that one YouTuber built the project and admitted on camera that he never generated a single token because he did not have enough disk space. The on-screen text is blunt about the scope of the claim: 8 GB of RAM, not 8 GB of anything else.

Why It Matters Anyway

The creator frames the project as a proof rather than a product. The on-screen text summarizes the thesis: the memory wall was never about model size, it is about which bytes you keep. If expert weights can live on disk and stream in on demand with byte-identical results, then RAM stops being the hard ceiling on which models you can run locally, and becomes a knob that trades money for latency. He ties this to the broader momentum behind open-source models, since tricks like this only work when you can get the weights in the first place.

Key Takeaways

kimi-k3-in-c runs the 2.78T parameter Kimi K3 on one CPU with 8.24 GB peak RAM, using a 176 KB C99 engine with no GPU or framework.
The method exploits mixture-of-experts sparsity: only 16 of 896 experts run per token, so about 93% of the weights stay on the SSD and load on demand.
Output is byte-identical to full in-memory inference at every stage of the 5,560 GB to 8.24 GB reduction.
Speed is the cost: roughly 26 to 33 seconds per token at 8 GB of RAM, improving to 5.6 s/token at 128 GB or more.
Hard requirements remain: a 1.56 TB SSD, Linux on x86-64, and hours of checkpoint download time.
The result is a proof of concept about memory management, showing that RAM limits which bytes you keep resident, not which model sizes you can run.

Resources

Kimi (Moonshot AI) -- the model family the K3 checkpoint comes from
r/LocalLLaMA -- the subreddit where the result circulated, linked in a quoted post in the video