Soup Fine-Tunes an 8B Model on a 4 GB Laptop GPU With One YAML File

Alpamys Makazhan, a developer from Kazakhstan, built Soup, an open-source CLI that fine-tunes large language models on consumer hardware. The core claim: layer streaming lets you train an 8 billion parameter model like Llama in about 3.3 GB of VRAM, so a 4 GB laptop GPU can do work that normally requires a 16 GB card or a rented cloud GPU. You write one YAML file, run soup train, and Soup handles batch size, memory management, and LoRA adapters.

The Problem Soup Targets

Fine-tuning has been expensive and fiddly. The typical path was renting cloud GPUs, writing custom training code, and debugging for days. Many developers gave up and used someone else's pre-trained model instead. Soup compresses that workflow into a config file and a single command.

How the Workflow Looks

1. Describe what you want the model to do in a YAML config.
2. Point the config at your training data.
3. Run soup train.

Soup figures out batch size, memory allocation, quantization, and LoRA adapter setup on its own. The repo's README (visible in the video) shows tiered installs: pip install soup-cli for the light core (CLI, config, data), pip install "soup-cli[train]" to add the training stack (torch), and pip install "soup-cli[all]" for training plus serving and UI.

Layer Streaming, the Key Technique

Loading a full 8B parameter model normally needs at least 16 GB of VRAM because every layer sits in GPU memory at once. Layer streaming keeps the frozen base model out of VRAM and feeds the GPU one decoder layer at a time: load a layer, train against it, swap in the next. Peak memory drops to about 3.3 GB, which fits inside most gaming laptops.

Reported Performance

Metric	Value
Model	Llama-3.1-8B-Instruct (NF4 quantization)
Test GPU	4 GB laptop GPU (RTX 3050 Laptop class)
Peak VRAM	~3.3 GB
Throughput	119 tokens per second

The project documentation states results are bit-exact against a normal fully-resident run, with an independent reproduction on an H100 at the same memory footprint. Treat single-benchmark numbers as a starting point and test on your own hardware and dataset.

What the Video Shows On Screen

The video opens with the caption "HE MADE AI FINE-TUNING ALMOST FREE" over footage of the Kazakhstan flag, then cuts between the creator talking, GPU and PC build b-roll, an attention-mechanism visualization, and the Soup GitHub repo. The repo view shows active development (folders like src/soup_cli and tests updated within hours of recording) and recent releases in the v0.71.x line, including a reward verifier generator and a regression gate for CI on model weights.

Key Takeaways

• Soup is an open-source CLI for fine-tuning LLMs from one YAML file and one command, soup train.
• Layer streaming swaps decoder layers through the GPU one at a time, cutting peak VRAM for an 8B model from 16+ GB to about 3.3 GB.
• A test on a 4 GB laptop GPU hit 119 tokens per second on Llama-3.1-8B-Instruct.
• Soup automates batch size, quantization, and LoRA adapter configuration, removing most custom training code.
• If cloud GPU costs are what stops you from fine-tuning, this puts the workflow on hardware you already own.

Resources

• Soup on GitHub -- the open-source repo by Alpamys Makazhan; README covers install tiers, quick start, and layer streaming details
• Soup CLI on Product Hunt -- launch page with community discussion