<- all tokdocs
2026.07.26@maybeworthbuilding

The 99 Percent Discount Just Reached Frontier Scale

AI
Watch on TikTok

View on TikTok ->

Dave Blundin, founder and GP of Link Ventures, makes a simple argument on the Moonshots podcast: a hobbyist GitHub project has spent years cutting the cost of training GPT-2 by 99 percent, and the release of Kimi K3 is the first proof that those same efficiency tricks hold at frontier scale. If that holds, every billion-dollar model someone builds now has a 1 percent-cost copy coming behind it.

The Speedrun That Nobody Watched

The Keller Jordan speedrun is a public GitHub repo where AI researchers compete to train a GPT-2-class model as fast and as cheap as possible. The baseline is Andrej Karpathy's work: his llm.c GPT-2 replication hits the target validation loss in about 45 minutes on an 8xH100 node, and the speedrun descends from that PyTorch trainer. Over years of incremental tricks the community pushed the same result down to roughly 90 seconds.

Blundin frames the cost reduction as 99 percent. The result now costs about one percent of the original. His point in the clip is that almost nobody paid attention, because it was "just GPT-2," a model that is five years old and no longer interesting on its own.

Why GPT-2 Was Never the Point

The reason the speedrun mattered was never the model. It was the open question underneath it: do efficiency gains found on a small, old model transfer to a giant, current one? Cutting the cost of a 124M-parameter model is a lab curiosity. Cutting the cost of a 10-trillion-parameter frontier model is an industry event.

Until recently there was no proof either way. You could stack optimizations on GPT-2 all day without knowing whether the same math survives when you scale to the models Elon Musk builds on a 16 billion dollar Colossus 2 data center. That uncertainty is exactly what kept the speedrun a niche pursuit.

Kimi K3 Closed the Question

Blundin's claim is that Kimi K3 answered it. The description dates the model's full open-weight release to July 27, with 2.8 trillion parameters going public. A frontier-class model released with open weights lets outside researchers see and reproduce what used to be locked inside a lab, which is what makes the transfer question testable at scale for the first time.

His conclusion: when a lab spends billions to build a 10 or 20 trillion parameter model, there is now a one percent-cost path to "effectively the same thing." The economics of training are no longer a moat you can buy your way into.

What This Means For Model Economics

Item Blundin's framing
The speedrun Community effort cutting GPT-2 training cost by 99 percent
The baseline Karpathy's llm.c GPT-2 replication, roughly 45 minutes on 8xH100
The open question Whether small-model tricks hold at frontier scale
The proof Kimi K3, 2.8T parameters, open weights, July 27
The consequence A 1 percent-cost copy for every billion-dollar frontier model

The takeaway is not that training gets a little cheaper. It is that the cost advantage of being first and best-funded may not last. If a 1 percent-cost reproduction is reliable, the value shifts away from who can afford the largest training run and toward distribution, data, and product.

Key Takeaways

  • The Keller Jordan speedrun cut GPT-2 training cost by 99 percent through years of community optimization on Karpathy's nanoGPT and llm.c lineage.
  • The real question was always whether those gains transfer to frontier-scale models, not GPT-2 itself.
  • Blundin argues Kimi K3's open-weight release is the first clear evidence that they do.
  • If true, every billion-dollar frontier model now has a roughly 1 percent-cost copy coming behind it.
  • Training-cost scale stops being a durable competitive moat.

Resources

Published July 26, 2026. Writeup generated from a favorited TikTok.