SkillOpt: Training a Closed Model's Skill File Like It Was Weights

You cannot fine-tune a closed model like GPT or Claude. The weights are sealed. The one thing you do control is the skill file the model reads at runtime. This video walks through a method, called SkillOpt, that treats that text file the way training treats weights: run the agent, measure where it failed, and let a second model propose small edits that have to earn their place. The reported result is roughly +25 points on GPT-5.5 without touching the model itself.

The whole video is animated diagrams over narration. There is no demo or code on screen, but each section maps a training idea onto plain text editing.

The Setup: Frozen Weights, Editable Text

The model's weights are "sealed." You can't change them. What you can change is skill.md, the instruction document the agent reads. The video's framing question is direct: what can you actually change? The answer is the document.

Today those skill files get written one of two ways: by hand, or generated in one shot by an LLM. Both leave performance on the table. The chart in the video labels the gap between current skills and what's possible as "unrealized," rising across epochs.

The Core Idea: Gradient Descent in Text

The trick is to treat the document like weights and optimize it the same way training optimizes a network.

• Sentences become the parameters. The text is what gets tuned.
• Edits become the steps. Each change moves you through the space, like a step of gradient descent.
• The benchmark score is the loss. You're descending toward a minimum, which means a higher benchmark score.

The video shows a ball rolling into the bottom of a curve, labeled "minimum," to make the analogy concrete.

The Optimizer Loop

SkillOpt runs a cycle. Each pass is an epoch with a current score.

1. Run. Execute the agent using the current skill file.
2. Score. Measure where it failed on the benchmark.
3. Propose. A second model suggests a tiny edit: add a line, cut a line, or rewrite a line.
4. Patch. Apply the edit to skill.md and loop again.

The animation shows this as a circle: propose, run, score, patch, repeat, with the score sitting at 66 in the example epoch.

What Keeps It From Breaking

The guardrail is validation on a held-out set. An edit is not accepted just because the proposing model likes it. It has to beat the prior score on data it has not seen.

If a proposed change does not beat the held-out score, it gets reverted and is not retried. This forces small steps only and stops the document from drifting into changes that look good locally but hurt overall. The result is that the skill effectively teaches itself, edit by edit, without any human rewriting it.

What It Buys You

The reported gain is about +24.8 points on GPT-5.5 in a Codex loop, compared to a no-skill baseline. The video lists what SkillOpt beats:

Approach	Status per the video
Hand-written skills	Beaten
One-shot LLM-generated skills	Beaten
TextGrad	Beaten
GEPA	Beaten
EvoSkill	Beaten

Because the output is just a text file, it transfers to other models with no retraining. You optimize the document once and carry it across model families.

Key Takeaways

• Closed models cannot be fine-tuned, but their skill file can be optimized, and that file is the real lever you control.
• SkillOpt maps training onto text: sentences are parameters, edits are steps, and the benchmark score is the loss.
• The loop is run, score, propose a tiny edit, patch, repeat, with a second model proposing the edits.
• Every edit must beat a held-out score or it is reverted, which keeps steps small and prevents regressions.
• The reported gain is about +25 points on GPT-5.5, and because it is a plain file, it transfers across models with no retraining.

Resources

• TextGrad - prior method for optimizing text via feedback, named as a baseline SkillOpt beats.
• GEPA - reflective prompt and instruction optimizer, named as a baseline.