Soup CLI: Fine-Tuning an 8B Model on a 4 GB GPU, Fact-Checked
By Matt Cullerton, founder of Mavric and Apso
Soup is an Apache-2.0 command-line tool that fine-tunes language models from one YAML file and a soup train command. Its "layer streaming" mode lets a 4 GB laptop GPU fine-tune an 8B model with LoRA. A TikTok from @ai.with.andrew gets the core idea right, but misspells the author's name, calls him a solo founder, and credits his own pre-repair benchmark to "a developer."
What is Soup?
Soup lives at MakazhanAlpamys/Soup on GitHub. It handles supervised fine-tuning, DPO, GRPO, PPO and LoRA adapter work, according to its GitHub topics. The install matches the video's "one config, one command" pitch:
pip install "soup-cli[train]"
soup init --template chat
soup train
It is open source under Apache-2.0. The about page says "There is no paid tier and nothing held back for a commercial edition."
Soup the CLI is unrelated to "model soups," a 2022 technique from Wortsman et al. that averages the weights of several fine-tuned models. Soup does have a soup merge --adapter command, but that folds a LoRA adapter into a base model.
Who built Soup?
The video says "His name is Alpamis Makajan." The GitHub account, the Zenodo paper and the README copyright line all spell it Alpamys Makazhan.
The video opens with "A solo founder from Kazakhstan." Soup's about page lists two co-founders: "Rafik Mamedov, Co-Founder & CEO" and "Alpamys Makazhan, Co-Founder & CTO." It says "Soup is developed and maintained by MePlay, Inc.," incorporated in Delaware on October 1, 2025.
The solo framing holds in one narrow sense. The README says Soup "is built and maintained in the open on a single 4 GB laptop, which is why every performance number in these docs is measured rather than claimed."
What is layer streaming?
The video calls the technique "as-layered streaming." The README, docs and YAML key all call it "layer streaming," turned on with stream_layers: true and marked BETA. The formal paper title on Zenodo is Exact Layer Streaming, published August 13, 2026 under CC BY 4.0.
The word "exact" matters. The paper's contribution is a correctness check "gated on bit-exactness against a resident reference," because, as the about page puts it, "a streamed run that has quietly severed its autograd graph still produces a falling loss curve and looks healthy."
The video says Soup works "by loading the model one layer at a time, training it, then swapping the next one in." That is close. Per the about page, the frozen base model is split into one shard per decoder layer, held in CPU RAM or on NVMe, and copied one layer at a time into small VRAM buffers so the next layer loads while the current one computes. "Only the LoRA adapters stay resident." The base layers are never trained. Only the LoRA adapters take gradients.
Can Soup really get 119 tokens per second on a 4 GB GPU?
The video says "A developer ran it on a 4 gigabyte laptop GPU and got 119 tokens per second." The README shows whose run that was: "Measured on an RTX 3050 Laptop 4 GB: Llama-3.1-8B-Instruct + NF4 at 119.6 tok/s at 952 MHz, 3.32 GB peak (pre-repair)." That is the maintainer's own laptop.
An outside developer did replicate it, with a different result. The README continues: "The post-repair run, on another RTX 3050 Laptop 4 GB (@umran666, #1103), reads 208.6 tok/s at 1935-1957 MHz in a 2.40 GB peak." The README says the gap between 119.6 and 208.6 "is the clock, not a speedup," since the second GPU ran at roughly double the clock speed.
Both of the video's figures are also out of date. The README says 119.6 and the H100 figure of 113.00 tok/s "were measured on v0.72.2, before the v0.73.0 correctness repair that cost -4.8% at 32B," and that pre- and post-repair rows "are not directly comparable."
Does an 8B model normally need 16 GB of VRAM?
Only at 16-bit precision. The video says "Normally, loading a full 8 billion parameter model like LLAMA needs at least 16 gigs of VRAM." Eight billion parameters at two bytes each is 16 GB, before optimizer state or activations.
QLoRA, published May 2023, changed that. Its abstract says it "reduces memory usage enough to finetune a 65B parameter model on a single 48GB GPU while preserving full 16-bit finetuning task performance."
Soup's own README agrees. Its VRAM guide lists 8 GB as enough for "~7B" models under "QLoRA 4-bit," with Llama-3.1-8B as an example. So the realistic starting point is about 8 GB, and layer streaming brings it to 3.32 GB (pre-repair) or 2.40 GB (post-repair). That is roughly a halving, compared with the five-fold cut the video implies.
Is Soup new, and did it go viral?
The video says Soup "just made it possible." The GitHub API dates the repository to February 20, 2026, almost eight months before the video.
The Show HN post, "Fine-tune an 8B model on a 4 GB laptop GPU," went up on August 4, 2026, submitted by Makazhan himself. It reached 139 points and 28 comments. In his top comment he explained that "during LoRA the base is frozen, read, never written. It doesn't need to live in VRAM, it needs to arrive before the matmul that uses it."
Does the GPU cost less than a PlayStation?
The video's hook compares "a laptop GPU that costs less than a PlayStation." The GPU is the GeForce RTX 3050 Laptop GPU, which is soldered into laptops and has no standalone retail price, so the comparison cannot be checked. On the console side, Sony's PlayStation Blog set US prices from April 2, 2026 at $649.99 for the PS5, $599.99 for the Digital Edition and $899.99 for the PS5 Pro.
Bottom line
Soup is a real, free, Apache-2.0 fine-tuning CLI from Alpamys Makazhan and co-founder Rafik Mamedov at MePlay, Inc. Layer streaming does fit an 8B LoRA fine-tune into a 4 GB laptop GPU. The 119.6 tok/s figure is the maintainer's own pre-repair number, the outside replication measured 208.6 tok/s, and the real VRAM saving is from about 8 GB under QLoRA, not 16 GB.
Sources
- MakazhanAlpamys/Soup on GitHub: repository, license, creation date and topics.
- Soup README: benchmark figures, repair notes, VRAM guide and install commands.
- About Soup and MePlay, Inc.: the two co-founders, the company and the layer streaming description.
- Exact Layer Streaming, Zenodo: the paper, author and measured figures.
- Show HN: Fine-tune an 8B model on a 4 GB laptop GPU: the August 4, 2026 launch post.
- Model soups, arXiv:2203.05482: the unrelated weight-averaging technique.
- QLoRA, arXiv:2305.14314: the 4-bit fine-tuning baseline.
- PlayStation Blog price changes, March 27, 2026: PS5 US prices.
Published October 8, 2026. Based on a TikTok by @ai.with.andrew. Full video notes (markdown)