Microsoft's SkillOpt-Sleep "Zero to Perfect in One Night" Is a 3-Task Held-Out Score on Claude Haiku 4.5, a Model Anthropic Now Lists as Legacy

The headline result the video attributes to Microsoft, a weak skill going "from zero to perfect in one night" on Claude Haiku, comes from a table in the plugin README that names Claude Haiku 4.5, scores a brief-writer seed with a local rule judge, and reproduces with --limit-holdout 3, so "perfect" means three of three held-out tasks passed a check for a risks section and a confidence level, on a model whose own Anthropic page now reads "Legacy" with Claude Haiku 5.5 as the current Haiku. The artifact is a 105.60-second vertical clip, 720x1280 at 9:16, in an MP4 container (ffprobe reports mov,mp4,m4a,3gp,3g2,mj2), video in HEVC Main as hvc1 at 92,682 bps and 30 fps across 3,168 frames, audio in HE-AACv2 at 44,100 Hz stereo and 64,112 bps, 163,501 bps overall, 2,158,295 bytes on disk. TikTok's metadata puts the post at 2026-10-14 18:10:07 UTC from uploader dripthecup, channel nickname "Dripthecup", with a 92-character description and the audio credited as "Sci-fi ambient with a desolate and lonely atmosphere" by harryfaoki. At capture on 2026-10-19 01:52 UTC the counters read 1,130 views, 42 likes, 0 comments, 22 reposts, and 70 saves. I read all 53 extracted frames (2-second intervals) and all 340 transcript words; audio.mp3 runs 105.67 seconds and the last Whisper segment ends at 102.4 seconds, so I discarded nothing. There is no human presenter. Every frame is a dark starfield with a crescent moon, a round latte-cup mascot badge labeled "CAPP" at lower left, a walking paper-cup mascot (tagged "DRIP" in a few frames) at lower right, a monospace chapter label at top left that steps through "// the repo", "// a sleep cycle", "// every morning", "// skillopt", "// one night", "// the gate", "// 3 steps", "// tested", "// 2 tips", and "// learned", and a top-right ticker reading "NAME IN A SECOND…", then "MICROSOFT · MIT", then "OUR TEST + THEIRS", then "DRIPTHECUP.COM/LINKS". Frames 1 to 3 show a GitHub card "microsoft / [blank]" with an "MIT · OPEN SOURCE" badge and a blank star count; frame 15 fills it in as "microsoft / SkillOpt" with a "MICROSOFT RESEARCH" chip and "★ 0", and frame 16 animates the count to "★ 18,328". Frames 4 to 6 and 14 show a "CLAUDE.md / CLAUDE'S INSTRUCTION FILE" card with orbiting task chips "update the README", "fix the tests", and "write the commit", plus a sticky note "+ Commit subjects: imperative, max 50 chars" stamped "WRITTEN DOWN FOR YOU ✓". Frames 7 to 10 show MON, TUE, WED 9:00 chat bubbles "Remember: no em dashes." answered "Got it! First time hearing that.", labeled "ILLUSTRATION", under the caption "EVERY SESSION STARTS FROM THE SAME FILE". Frames 17 to 22 show a "ONE night" stepper (1·READ, 2·FIND, 3·PRACTICE, 4·DRAFT), a mined task "fix the failing tests" tagged "×3 THIS WEEK", and two draft cards, "CLAUDE.md + Run the tests before every commit" and "SKILL.md + Check the failing test's error first", labeled "DRAFTS ONLY · ILLUSTRATION". Frames 23 to 27 show "THE gate" as a bar with a "HELD BACK · NEVER PRACTICED" grid reading "6/10 PASS", a "KEPT" stamp, a "THROWN OUT" stamp, and a prompt "NOTHING GOES LIVE UNTIL YOU TYPE /skillopt-sleep adopt". Frames 30 to 36 build three command cards: "git clone https://github.com/microsoft/SkillOpt", "/plugin marketplace add ./plugins/claude-code", "/plugin install skillopt-sleep@skillopt-sleep", "/skillopt-sleep dry-run", "/skillopt-sleep run", and "/skillopt-sleep adopt --legacy". Frames 37 to 41 show two result cards: "MICROSOFT'S TEST · THEIR NUMBERS / Claude Haiku, 1 night / a deliberately weak skill (brief-writer) / BEFORE 0.00, AFTER 1.00 / held-out score · 3 tasks · gbrain-evals", and "OUR TEST · i7-10700F + RX 9060 XT / Claude Haiku, 2 nights / built-in coding tasks (already decent) / 0.63 / 16 edits drafted · 0 kept / NOTHING CHANGED". Frames 42 to 47 show "TIP 1 The default is a free practice run (no AI calls) / For real learning, switch the backend to Claude. It uses your own Claude usage. / /skillopt-sleep run --backend claude" and "TIP 2 Real mode sends session snippets to the model / Skip it on sensitive projects." Frames 48 to 53 close on the CLAUDE.md card with two adopted notes and "dripthecup.com/links · THE 3 STEPS · github.com/microsoft/SkillOpt · SEND TO SOMEONE WHOSE CLAUDE KEEPS FORGETTING".

The repo, the license, the lab, and the star count all check out

The GitHub REST API for microsoft/SkillOpt on 2026-10-19 returns an MIT license, Python as the primary language, creation at 2026-05-08T06:41:01Z, a last push at 2026-10-18T17:03:51Z, 18,548 stargazers, 1,763 forks, 72 subscribers, and 55 open issues. Frame 16 shows 18,328 stars, which is 220 below the figure five days after the post. "Over eighteen thousand stars" was accurate on the day.

"From Microsoft Research" also holds. The paper, arXiv 2605.23904, "SkillOpt: Executive Strategy for Self-Evolving Agent Skills", was submitted 22 May 2026 (v2 on 25 May) by fifteen authors from Yifan Yang through Chong Luo. Microsoft Research published a feature titled "SkillOpt: Agent skills as trainable parameters" on June 30, 2026, bylined by seven Microsoft Research staff including Yifan Yang (Senior Researcher) and Chong Luo (GM Research Sciences). The Claude Code plugin's marketplace.json lists its owner as "Yifan Yang" with a microsoft.com address.

One distinction the video flattens: SkillOpt the paper trains a skill document against a benchmark with a separate optimizer model, and the thing the video installs is SkillOpt-Sleep, which the repo calls the "deployment-time companion". The README's news log dates the Sleep preview to 2026-06-15 and its first packaged release to v0.2.0 on 2026-07-02; PyPI confirms 0.1.0 on 2026-06-02 and 0.2.0 on 2026-07-02 as the only releases. The Sleep documentation opens with a "Preview" notice stating that "interfaces and defaults may change". The video never uses the word preview. The creator's own links page does.

The plugin README also credits where the "sleep cycle" idea came from. It names three sources: SkillOpt's validation-gated edits, "Claude Dreams" for offline consolidation with review-then-adopt, and the agent-sleep idea. Anthropic's Dreams documentation describes a research-preview feature of Claude Managed Agents that reads a memory store plus 1 to 100 past sessions and writes a new, reorganized store while leaving the input untouched. The video presents the overnight-learning framing as Microsoft's; Microsoft's own docs present it as a synthesis that includes an Anthropic feature.

The three install steps match the plugin README, with one click and one flag the narration skips

The commands in frames 30 to 36 are the same commands in the plugin README: git clone https://github.com/microsoft/SkillOpt.git, /plugin marketplace add ./plugins/claude-code, /plugin install skillopt-sleep@skillopt-sleep, then /skillopt-sleep dry-run, /skillopt-sleep run, and /skillopt-sleep adopt --legacy. The marketplace name and the plugin name are both skillopt-sleep in plugins/claude-code/.claude-plugin/marketplace.json, which is why the install id doubles up. The README adds /skillopt-sleep status as a verification step and states the requirement the video omits: Python 3.10 or newer, plus an authenticated claude executable on PATH for the real backend.

Claude Code's own marketplace documentation confirms the mechanism: a marketplace is any directory with a .claude-plugin/marketplace.json, /plugin marketplace add ./path registers it from inside a session, and /plugin install name@marketplace "opens the plugin's details in the /plugin panel, where you install it." Step two therefore ends with a click in a panel, which the video's animation does not show.

The narration says "type adopt". The frame and the README both say adopt --legacy, and the README explains why: adopt --legacy applies the reviewed managed proposal, while nights that use per-skill fan-out need adopt --skill NAME or adopt --all-skills. The plugin's command file goes further and instructs Claude to "never imply that bare adopt means adopt everything." The backup claim is correct: the README states "Every adopt backs up the prior file first" and the command file confirms backups land under the staging directory's backup/.

Cloning instead of pip install skillopt is also the right call, and the README says why. PyPI's latest release is 0.2.0 from July; the README's version note says handoff mode, --preferences, multi-skill fan-out, and reviewed subset adoption "require a source checkout from main until the next release", and the bundled runner needs no pip install from a checkout. The scheduling line checks out too: /skillopt-sleep schedule --hour 3 --minute 17 installs a managed cron entry, and "the scheduled run stages proposals unless --auto-adopt is explicitly selected." So "nothing goes live until you type adopt" is true by default and has a documented opt-out the video does not mention.

"Zero to perfect in one night" means three of three held-out tasks on a rule check, using Claude Haiku 4.5

The transcript says "Microsoft's own test with Claude Haiku took a deliberately weak skill from zero to perfect in one night." The source is the plugin README's results table: backend "Claude (Haiku 4.5)", seed "brief-writer", held-out score 0.00 to 1.00, 1 night; the Codex backend reached the same score in 2 nights. The README describes the seed as a brief-writer "with no risks section / no confidence level" and the scorer as "a local rule judge (no judge-API, no way to grade its own homework)". The reproduce command runs with --nights 1 --limit-replay 3 --limit-holdout 3. Three held-out tasks, scored on whether the required sections appear, is what 0.00 to 1.00 measures.

Frame 37's small print discloses this ("held-out score · 3 tasks · gbrain-evals"); the narration does not, and the creator's links page states it plainly: "was scored on 3 held-out tasks." The benchmark is Garry Tan's gbrain-evals skillopt-v1, whose own write-up (dated 2026-06-03, fixture updated 2026-08-31) lists four deliberately flawed seeds, including a brief-writer that "never includes a risks section or a confidence level", scored by an easy section-heading checker plus an independent quality judge, with all four seeds going 0.00 to 1.00 at a cost of $0.88 to $0.98 per variant.

I am correcting the video's unversioned "Claude Haiku". Microsoft's table names Haiku 4.5. Anthropic's models overview, fetched 2026-10-19, lists the current lineup as Claude Fable 5.1, Claude Opus 5.5, Claude Sonnet 5.5, and Claude Haiku 5.5 (claude-haiku-5-5, from $0.10 per million input tokens and $0.50 per million output tokens), and files Claude Haiku 4.5 under "Legacy models (still available)". The Haiku 4.5 page reads "Legacy. Released October 15, 2025", model ID claude-haiku-4-5-20251001, $1 per million input tokens and $5 per million output tokens, retirement "not sooner than October 15, 2026". The reproduce command passes --model haiku to the Claude CLI, so I cannot tell from the docs which Haiku a rerun today would use.

Microsoft's own RESULTS.md puts the gbrain result in context the video leaves out. Its "Honest scope" section says the method helps on "recurring tasks with a checkable correctness signal and real headroom", is "flat within run-to-run noise" on saturated tasks with strong models, and rests on a single seed, so differences under about 1.5 points should be treated as noise. Its headline experiment is about safety rather than lift: GPT-5.4-nano on SearchQA fell from 0.554 to 0.026 over five nights with the gate off, a 52.8-point collapse, and held at 0.570 with the gate on.

The creator's null result agrees with Microsoft's stated limits, and the hardware label is beside the point

Frames 40 and 41 and the line "On our PC, Haiku had nothing better to learn, so the gate changed nothing" report the creator's own run: Claude Haiku, 2 nights, "built-in coding tasks (already decent)", 0.63 before and after, 16 edits drafted and 0 kept. The creator's links page gives more precision than the video: "Our test on Oct 14, 2026 (Claude Haiku, the built-in practice tasks): 0.625 to 0.625 over 2 nights and 0.861 to 0.861 over 3, with no edits kept. The rules and task cards in the video are illustrations." So the on-screen 0.63 is 0.625 rounded, a second 3-night run at 0.861 exists that the video omits, and the page calls the tasks "practice tasks" where the frame calls them "coding tasks". These numbers are the creator's and I could not verify them independently.

The outcome is what RESULTS.md predicts for a saturated task set: the gate rejected every one of 16 drafted edits because none lifted the held-out score. That is the gate working as documented, and the video says as much.

The "i7-10700F + RX 9060 XT" label on the card implies the local machine affected the result. With --backend claude it did not. The plugin README says a real CLI backend "requires its corresponding claude or codex executable on PATH and authenticated", and the data-boundary section says the backend sends prompts to the selected provider. The CPU and GPU ran the harvest, the mining, and the local rule judge; the model ran on Anthropic's servers.

Both tips hold up, and the docs add three cautions the video skips

Tip one is accurate. The README states the default backend is mock, "deterministic, no API spend", and that --backend claude or --backend codex switches to "model-driven mining and optimization on your own budget". The plugin's command definition preserves a --backend option and passes it through to the bundled scripts/sleep.sh, so frame 44's /skillopt-sleep run --backend claude is a valid invocation. The README also documents a third path the video does not mention: /skillopt-sleep-handoff run, where the current Claude Code session answers the model calls with no claude -p subprocess and no API key, which the README calls subscription-friendly.

Tip two is accurate and, if anything, understated. The README's data-boundary section says a real backend "sends truncated transcript excerpts and derived tasks to the selected provider for mining, replay, judging, and reflection" and that "outbound prompts are not currently guaranteed to be secret-free." The Sleep documentation suggests harvesting to a task file, inspecting and redacting it, and marking it "reviewed": true before replaying on a sensitive project.

Three cautions from the same docs did not make the video. First, each stateful night writes a local evidence.jsonl containing "best-effort-redacted, per-field-truncated copies of miner, replay, judge, and reflection prompts and replies", which the docs say to "treat as sensitive local data". Second, the per-night token budget max_tokens_per_night defaults to 400,000 and "is not a hard cap"; a started step always finishes, there is no wall-clock budget, and a real-backend dry-run still incurs provider calls and spend. Third, the scheduler accepts --auto-adopt, so an unattended nightly run can rewrite CLAUDE.md if you opt in, which is the one configuration where "nothing goes live until you type adopt" stops being true.

Key Takeaways

Verified: microsoft/SkillOpt is MIT-licensed, created 2026-05-08, with 18,548 stars and 1,763 forks on 2026-10-19; frame 16's 18,328 was accurate for the week of posting.
Verified: The paper (arXiv 2605.23904, 22 May 2026) and the June 30, 2026 Microsoft Research blog post are bylined by Microsoft Research staff, and the plugin's marketplace manifest is owned by first author Yifan Yang.
Verified: The six commands on screen match the plugin README exactly, including adopt --legacy, the mock default backend, --backend claude, the cron scheduler, and backups on adopt.
Correction: The "zero to perfect in one night" result used Claude Haiku 4.5, which Anthropic now labels a legacy model behind Claude Haiku 5.5, and it measures three held-out tasks judged by a local rule check for a risks section and a confidence level.
Clarification: SkillOpt-Sleep is labeled a preview in its own docs, has had no packaged release since 0.2.0 on 2026-07-02, and credits Anthropic's Dreams feature as one of its three source ideas.
Unverified: The creator's own numbers (0.625 to 0.625 over 2 nights, 0.861 to 0.861 over 3, 16 edits drafted, 0 kept) are self-reported, and the hardware on the card did no model inference under the Claude backend.

Resources

microsoft/SkillOpt on GitHub establishes the MIT license, the 2026-05-08 creation date, the 18,548 stars and 1,763 forks on 2026-10-19, the 2026-10-18 last push, the v0.1.0 and v0.2.0 release dates, and the README's news log and paper citation.
SkillOpt-Sleep Claude Code plugin README establishes the install commands, the mock default, --backend claude, the handoff mode, the Haiku 4.5 brief-writer table, the --limit-holdout 3 reproduce command, the backup-on-adopt rule, and the scheduler.
docs/sleep/README.md establishes the preview label, the data boundary, the evidence.jsonl log, the 400,000-token default budget and its soft-cap behavior, and the three credited source ideas.
docs/sleep/RESULTS.md establishes the gate-off collapse from 0.554 to 0.026, the "Honest scope" limits, and the single-seed caveat.
arXiv 2605.23904 establishes the paper title, the fifteen authors, and the 22 May 2026 submission date.
Microsoft Research: SkillOpt: Agent skills as trainable parameters establishes the June 30, 2026 publication date and the Microsoft Research bylines.
skillopt on PyPI establishes 0.1.0 (2026-06-02) and 0.2.0 (2026-07-02) as the only releases, with MIT license.
Anthropic models overview establishes the current lineup and Claude Haiku 5.5 pricing, and files Claude Haiku 4.5 under legacy models.
Claude Haiku 4.5 model page establishes the "Legacy" status, the October 15, 2025 release, the model ID, and the $1 and $5 per million token prices.
Claude Code: Create a marketplace establishes the .claude-plugin/marketplace.json requirement, the /plugin marketplace add ./path syntax, and that /plugin install name@marketplace opens a panel for the final install.
Claude Managed Agents: Dreams establishes that Dreams is a research-preview feature that reads a memory store plus up to 100 sessions and writes a new store without modifying the input.
gbrain-evals skillopt-v1 benchmark note establishes the four flawed seed skills, the brief-writer defect, the heading-based scorer plus quality judge, and the per-variant cost.
dripthecup.com/links establishes the creator's own test figures, the "3 held-out tasks" disclosure, the "illustrations" disclaimer, and the Python 3.10 requirement.