Ponytail's Benchmarks Hold Up, and Opus Gains Even More Than Haiku

Ponytail is an open source repo that claims to make Claude Code leaner: roughly 30% faster and 20% cheaper. Those published numbers came from runs on Haiku 4.5, so this video reruns the same benchmarks on both Haiku 4.5 and Opus 4.8 to see if they hold. They do, and on Opus the gains are larger than what the repo advertises.

What Ponytail Claims

The repo page shown on screen describes Ponytail with the tagline "He says nothing. He writes one line. It works." Listed stats: 40k stars, release v4.7.0, works with 14 agents, MIT license. The headline claims are "~54% less code (up to 94%) - ~20% cheaper - ~27% faster - 100% safe."

The footnote clarifies the methodology. The ~54% figure is the median across Claude Code sessions editing real tasks, measured with Haiku 4.5 at n=4. The skill reaches 94% where an agent overbuilds (the example given is a date picker) and drops near zero where the code is already minimal. The core idea: tell Claude Code that if something already exists, whether in a library, a dependency, or the native app, use it instead of reinventing it.

Rerunning the Line-of-Code Benchmark

A three-column comparison table tracks line-of-code reductions per task across three runs:

• THEIR PUBLISHED (Haiku 4.5, n=4): aggregate -54%
• OUR RUN (Haiku 4.5, n=2): aggregate -56%
• OUR RUN (Opus 4.8, n=2, new): aggregate -71%

Per-task numbers swing widely. A few from the Opus column: 309.5 to 11.5 (-96%), 454 to 109.5 (-76%), 269 to 83 (-69%). The reproduced Haiku run (-56%) lands essentially on top of the published -54%, and Opus pushes the reduction to -71%.

A Skill-vs-Baseline Chart

One frame shows a grouped bar chart, "Every metric vs the no-skill baseline (Claude Code, Haiku 4.5, 12 tasks)," comparing four variants: baseline (gray, 100%), caveman (orange), ponytail (green), and yagni-oneliner (purple). Lower is leaner, cheaper, and faster.

Ponytail's bars: LOC 46%, tokens 78%, cost 80%, time 73%. Caveman actually rises above baseline on tokens (107%), cost (102%), and time (102%). On the adversarial safety tier (path-traversal, SQLi, token forgery, malformed input, rate-limit), baseline, caveman, and ponytail all score 100%, while yagni-oneliner hit 95% because it "dropped a guard once."

Cost: Fewer Lines Turn Into Real Money on Opus

A per-run cost view ("BEYOND LINES -- COST PER RUN ($, FROM THE CLI)") carries the title "Fewer lines to real money -- but only fully on Opus." It puts Raw Claude against Ponytail per task.

• Haiku 4.5 aggregate cost: -25%
• Opus 4.8 aggregate cost: -53%

On Haiku, a couple of tasks went the wrong way (multi-step wizard +3%, count items +21%), which is why the aggregate stays modest. On Opus nearly every task drops hard: command palette $1.38 to $0.48 (-65%), multi-step wizard $1.39 to $0.38 (-73%), date picker $0.67 to $0.22 (-68%).

Speed: Same Pattern

A matching time chart shows Haiku 4.5 at -31% aggregate and Opus 4.8 at -71% aggregate. Haiku again has a few regressions (search by title +18%, bulk-delete +22%). Opus is faster almost everywhere: command palette 213s to 68s (-68%), multi-step wizard 245s to 54s (-78%), date picker 163s to 20s (-88%).

Why Opus Benefits More

The takeaway from the creator is that larger, more capable models gain more from Ponytail because they are more verbose by default. Opus in particular likes to build things from scratch that it does not have to build. Ponytail's instruction to reuse what already exists cuts the most fat where there is the most fat to cut, so the bigger model sees the bigger drop in lines, cost, and time.

Key Takeaways

• Ponytail's published Haiku 4.5 result (-54% lines of code) reproduced cleanly at -56% on an independent rerun.
• On Opus 4.8, the line reduction jumped to -71%, well above the advertised number.
• Cost savings scaled with model size: -25% aggregate on Haiku versus -53% on Opus.
• Speed followed the same split: -31% aggregate on Haiku versus -71% on Opus.
• On Haiku a handful of tasks regressed (slower or more expensive), but on Opus nearly every task improved.
• The mechanism is verbosity: bigger models over-build more, so a "reuse what exists" skill removes more waste from them.