Colibrì: A 744B Parameter Model Running in 25GB of RAM, in Pure C

Colibrì is an inference engine that runs GLM-5.2, a 744-billion-parameter mixture-of-experts model, on a consumer machine with about 25GB of RAM. It is written in pure C with zero dependencies. A model of this size normally requires hundreds of gigabytes of memory or a rack of GPUs. Colibrì gets around that by keeping only the dense part of the model in RAM and streaming the routed experts from disk as they are needed.

The author is upfront about the cost: on his machine it decodes at roughly a tenth of a token per second. The point is not speed. It is a frontier-class model answering correctly on hardware that, as the video puts it, costs less than an H100's fan.

How It Fits 744B Parameters in 25GB

The trick is exploiting the structure of a mixture-of-experts model. GLM-5.2 activates only about 40B parameters per token, and only around 11GB of those change from token to token (the routed experts). The README shown in the video breaks the model into two tiers:

The dense part (attention, shared experts, embeddings, roughly 17B parameters) stays resident in RAM at int4, about 9.9GB.
The 21,504 routed experts (75 MoE layers times 256 experts, plus the MTP head, about 19MB each at int4) live on disk, about 370GB total, and are streamed on demand. A per-layer LRU cache, an optional pinned hot-store, and the OS page cache as a free L2 keep the hot experts close.

Colibrì GitHub README showing the pixel-art hummingbird logo and the core idea: run GLM-5.2's 744B MoE on a consumer machine with ~25GB of RAM in pure C with zero dependencies

The engine itself is a single C file, c/glm.c, around 2,400 lines plus small headers. No BLAS, no Python at runtime, no GPU required. An opt-in CUDA tier exists for pinned experts.

README section "The idea" detailing the dense part resident in RAM at int4 (~9.9GB) and 21,504 routed experts streamed on demand from ~370GB on disk

What Is Implemented

The video scrolls through the "What's implemented" section of the repo, which lists more than a minimal forward pass:

A faithful GLM-5.2 forward pass, validated token-exact against a transformers oracle.
MLA attention with a compressed KV-cache: 576 floats per token instead of 32,768, a 57x reduction.
A DeepSeek-V3-style sigmoid router with shared expert and first-3-dense layers.
Native MTP speculative decoding using GLM-5.2's own multi-token-prediction head (layer 78), measured at 2.2 to 2.8 tokens per forward pass while staying lossless.
True sampling with temperature and nucleus, defaults tuned for int4 (0.7 / 0.90).

README "What's implemented" section listing the faithful GLM-5.2 forward pass, MLA attention with compressed KV-cache, sigmoid router, and MTP speculative decoding

The Honest Numbers

The repo publishes a benchmark table under the heading "Honest numbers," measured on WSL2 with 12 cores, 25GB of RAM, and NVMe via VHDX:

Metric	Value
Model on disk (int4)	~370 GB
Resident RAM (dense, int4)	9.9 GB
Load time	~30 s
Peak RSS during chat	~20 GB (auto-capped)
Cold decode cost	~11 GB disk reads/token
Decode speed	~0.05-0.1 tok/s cold
MTP speculation	2.2-2.8 tok/forward

Honest numbers benchmark table showing ~370GB model on disk, 9.9GB resident RAM, and roughly 0.05-0.1 tokens per second cold decode

The README states it plainly: this is not fast. It is a 744B frontier-class model answering correctly on a machine that costs less than one H100 fan. Warm cache, pinned hot experts, and MTP push the useful-response latency down; the physics of the disk does the rest.

Why "Colibrì"

The name is Italian for hummingbird. The README explains the metaphor: a hummingbird weighs a few grams, hovers in place, and visits a thousand flowers a day. This engine keeps a 744-billion-parameter giant alive on hummingbird rations: 25GB of RAM, twelve CPU cores, and a lot of disk patience. The repo (JustVugg/colibri) sits at 2.2k stars and 144 forks in the frames shown, is Apache 2.0 licensed, and GLM-5.2 weights are released by Z.ai under MIT.

Key Takeaways

MoE structure makes this possible: only ~40B of 744B parameters activate per token, and only ~11GB of weights change token to token, so the rest can live on disk.
The dense core fits in under 10GB of RAM at int4; the 21,504 routed experts stream from ~370GB on disk through an LRU cache.
The whole engine is one ~2,400-line C file with no BLAS, no Python, and no GPU requirement.
Speed is the tradeoff: roughly 0.05-0.1 tokens per second cold, improved by MTP speculative decoding at 2.2-2.8 tokens per forward pass.
The project is a proof that frontier-model inference on cheap consumer hardware is possible, not that it is practical for interactive use.

Resources

JustVugg/colibri on GitHub -- the inference engine shown in the video
GLM-5.2 -- the 744B MoE model by Z.ai, weights released under MIT