<- all tokdocs
2026.10.12@ai.honeycove

Cloudflare open sourced security-audit-skill on 18 June 2026, 116 days before this video, and the 7,245 findings it credits to that skill were produced by a separate internal harness

AI
Watch on TikTok

View on TikTok ->

A 40.13-second MP4 (QuickTime/MOV container, mov,mp4,m4a,3gp,3g2,mj2), 1080x1920 portrait, HEVC video at 30 fps carrying 1,204 frames and 401 kbps, AAC stereo audio at 44.1 kHz and 64 kbps, 472 kbps overall, 2,368,788 bytes on disk, format id bytevc1_1080p_472175-1. Posted at epoch 1791779759, which is 2026-10-12T04:35:59 UTC, by handle ai.honeycove (nickname "AI Honeycove", uploader id 7615522612223722526). Audio credit is "Million Dolla Hip Hop" by Brentin Davis. The description runs 309 characters and ends with the tags #AI #AITools #Cybersecurity #Cloudflare #TechInnovation. At the metadata capture on 2026-10-19 00:31 local time the counters read 85,700 views, 4,490 likes, 33 comments, and 1,157 reposts. I read 15 of the 20 extracted frames: 001, 002, 003, 004, 005, 006, 007, 008, 010, 012, 014, 016, 017, 019, and 020, covering the clip at roughly two-second intervals. The transcript holds 134 words across 16 Whisper segments. Frame 001 (about 0 s) splits the screen: the Cloudflare cloud logo over an animating word rendered mid-transition as "UDFL", a headline card reading "ai hunts" above "SECURITY BUGS", and below it a presenter, a middle-aged man in a black shirt and heavy black glasses seated against a pale wall with a green banker's lamp on his left and a potted plant on his right, with the spoken-word caption "open" overlaid. Frames 002 and 003 (about 2 s and 4 s) show a mocked browser window, chrome pill labelled "Claude", tab titled "StoreDesk Dashboard", URL bar reading admin.storedesk.io/ModernGoodsCo/dashboard, sidebar entries Home, Orders (badge 12), Products, Customers, Discounts, Shipping & Delivery, Settings, and a Shipping & Delivery panel with "Free delivery thresholds" fields at $75.00 domestic, $100.00 Canada, $150.00 international, captioned "finds" then "coded". Frame 004 (about 6 s) is a screen recording of the Cloudflare blog post "Build your own vulnerability harness", datelined "JUNE 18, 2026", bylined "Dan Jones, Alexandra Godoi, and Grant Bourzikas", marked "21 MINUTE READ", with the opening paragraph referencing "Project Glasswing" and "threats posed by frontier AI", captioned "originally". Frames 005 and 006 (about 8 s and 10 s) show a counter animating to "7,000+" under the header "Cloudflare's own codebase" and the label "security issues found", above a fabricated terminal titled security-audit listing entries such as edge/router/headers.rs:348 IDOR on record ID, workers/ai/prompt.ts:318 IDOR on record ID, api/v4/dns/records.go:369 Prompt injection, and edge/cache/purge.rs:41 Path traversal; frame 004 lists four consecutive rows with the identical label "IDOR on record ID" against four different file paths, and edge/router/headers.rs appears tagged "Open redirect" in one frame and "SQL injection risk" in the next, which marks this as designed b-roll rather than captured output. Frame 007 (about 12 s) is a stylised dark card reading "cloudflare / security-audit-skill" with a "Public" pill, a motion-blurred star counter mid-count-up, and two illegible tiles labelled "forks" and "watching", captioned "20,000". Frame 008 (about 14 s) shows the Claude wordmark animating in as "Claud" with the caption "install". Frame 010 (about 18 s) shows three orange pixel-sprite characters captioned "team". Frame 012 (about 22 s) stacks three cards, "Logins" with a flagged password field, "Database" showing a row 1 | ana@app.com | hunter2 with the password column highlighted red, and "APIs" showing GET /users, POST /login, and a flagged DELETE /admin, captioned "AP". Frame 014 (about 26 s) shows a bug card reading "Plain-text passwords". Frame 016 (about 30 s) shows a card titled "Skeptic agent" stamped "Real bug · confirmed" with the checks "Hashed? No", "Encrypted? No", "Test data? No". Frame 019 (about 36 s) shows a "Security report" card badged "3 issues" listing "Critical Plain-text passwords Verified", "High Unprotected admin route Verified", and "Medium No login rate limit Verified". Frame 020 (about 38 s) adds a "How to fix" row pointing at src/db/users.ts:42 with the diff - user.password = password; and + user.password = await hash(password);, captioned "exactly". No frame I read shows the real github.com repository page, a real terminal session, or a real run of the tool against a real codebase. Every artifact on screen is a designed mock except the Cloudflare blog screen recording in frame 004. The spoken argument runs as follows: Cloudflare has open sourced a free AI skill that finds security holes in vibe-coded apps, it was built internally where it found over 7,000 issues in Cloudflare's own codebase, it has crossed 20,000 GitHub stars, and once installed into Claude Code it maps the app, dispatches hunter agents across logins, databases, APIs, and prompt injection, hands each candidate bug to a separate agent that tries to disprove it, and finally emits a report of verified issues with fixes. On the transcript-end check: the final Whisper segment ends at 40.26 s, and audio.mp3 measures 40.150204 s, so the last segment overruns the audio by 0.11 s. That gap is rounding on the closing word rather than invention, and the segment text, "with every verified issue and exactly how to fix it", is corroborated by the on-screen caption "exactly" in frame 020 and the report card beside it. No transcript content is hallucinated past the end of audio.

"Cloudflare just open sourced" is 116 days out of date

At 0.00 s the video opens with "Cloudflare just opensource an AI skill that finds security holes in your vibe-coded app, and it's completely free." The GitHub REST API reports the repository cloudflare/security-audit-skill with created_at of 2026-06-18T14:08:44Z, and the commit log shows the initial commit authored by Dan Jones at 2026-06-18T16:20:36Z. The Cloudflare blog post that accompanied it, "Build your own vulnerability harness", carries a datePublished of 18 June 2026 and a modified date of 15 July 2026. The video was posted at epoch 1791779759, which is 2026-10-12T04:35:59 UTC. That is 116 days after the release.

The repository was not even recently touched at the time of posting. The API reports pushed_at of 2026-09-14T19:29:02Z, and the most recent of the 14 commits on main is "Clarify guidance and full audit modes" by Dan Jones on 14 September 2026. Nothing shipped in the 28 days before the video went up.

The one part of this claim that holds is "completely free". The repository carries an MIT license, confirmed in both the API response (spdx_id: MIT) and the LICENSE file at the repository root. The README closes with "MIT -- see LICENSE."

The 7,245 findings came from the internal harness, not from the skill that was released

At 6.54 s the video says "They originally built it for themselves" and at 8.18 s "It found over 7,000 security issues in their own code base." The 7,000 figure traces to a real number, but it belongs to a different system than the one the video tells you to install.

The Cloudflare post states: "Out of 20,799 raw candidates generated by VDH, only about 12,057 survived validation." It later states: "Ultimately this left 7,245 actionable findings for engineering teams to act on." VDH is the Vulnerability Discovery Harness, paired with the Vulnerability Validation System, and the post describes that pair operating across 128 distinct repositories, with a central pool brought to 13,841 findings across 145 repositories and a Dedup agent folding away 5,442 duplicates.

The released skill is the seed, not the harness. The post says "We started with a ~450-line security-audit skill that we ran on a single repository", and on the release itself: "Along with this blog post, we're releasing the initial skill we used to develop the harness, it's been slightly cleaned up before release so it's easier to understand and integrate, but the skill itself remains substantially the same." The repository README states the same relationship in its own words: "This is the skill that seeded Cloudflare's vulnerability discovery harness... The harness grew into a multi-stage, fleet-wide system; this skill is the single-repo starting point it evolved from."

So the chronology the video gives is directionally right. Cloudflare did build this for itself first. The error is one of attribution rather than order. What you install is the single-repository prototype. The 7,245 number is the output of the fleet-wide system that Cloudflare built on top of it and did not release.

"Crossed 20,000 stars" was true as a floor and understated the real count by roughly 12,000

At 11.68 s the video says the project "has already crossed 20,000 stars on GitHub." Frame 007 renders a cloudflare / security-audit-skill card with the star counter motion-blurred mid-animation and the caption "20,000" below it.

The GitHub API returned stargazers_count of 32,473 at my capture on 2026-10-19, alongside 2,028 forks and 129 subscribers. The star-history.com page for the repository reported 31.8k stars on the same date with a global rank of #1128 and 2.9k stars added that week, and noted a best rank of #1 on 17 September 2026 with fourteen days trending. Extrapolating backwards from that weekly delta puts the count near 29,000 on 12 October, the video's post date.

I could not source a dated star count for 12 October itself. GitHub caps stargazer pagination well below 32,473 entries, so the per-star timestamps are not reachable through the API, and I could not find a first-party snapshot for that specific day. The 29,000 figure is therefore an estimate derived from a secondary source, not a verified reading, and I flag it as such.

What is verifiable is the shape of the error. "Crossed 20,000" is technically satisfied by a count of 29,000 or 32,473 in the same way that "crossed 1,000" would be. It presents a threshold the project passed in September as if it were the current state.

One detail is worth putting on the record without assigning intent. The blog post pairs 7,245 actionable findings with 20,799 raw candidates in the same sequence of paragraphs. The video pairs "over 7,000 security issues" and "crossed 20,000 stars" in a single sentence at 8.18 s to 14.36 s. The 20,799 candidate figure is a count of vulnerability leads, and the star count is a count of GitHub users. They are unrelated quantities that happen to round to the same number, and both appear in the one source the video is summarising.

The adversarial validation claim is accurate; "only the real issues make it to the finish line" is not

At 26.60 s the video says "Then every bug they find is handed to a separate agent whose only job is to disprove that bug exists." This is the most accurate claim in the script. The repository README describes Phase 3 as "Candidate validation -- give every unique candidate to a fresh verifier that tries to disprove it", and lists among its design principles: "Adversarial validation. The agent that checks a finding is never the agent that found it." Phase 5 adds a second pass, "Independent record verification -- fresh agents verify final source claims. Material replacements receive another independent verifier." Frame 016 visualises this with a "Skeptic agent" card and a "Real bug · confirmed" stamp, which matches the mechanism even if the agent is not named that in the source.

At 31.84 s the video continues "So only the real issues make it to the finish line." That compresses a three-way verdict into a binary. The README states the output contract plainly: the skill writes "confirmed, needs_validation, and rejected records to findings.json", and "The verdicts are distinct: confirmed has a complete source trace and bounded observed result, needs_validation has an exact unresolved fact and no severity, and rejected records a disproved candidate." VALIDATION-AND-REPORTING.md is explicit that the middle category must not be presented as a finding: for the NEEDS VALIDATION table it instructs "Do not assign severity or call it a confirmed vulnerability", and for NEEDS-VALIDATION.md it instructs "Keep these as prioritized leads without severity."

The skill's own design principles push harder in the opposite direction than the video does. One reads "Only confirm established boundary failures. Keep a source-grounded blocked lead as needs_validation with its exact unresolved fact." Another reads "Defense-in-depth gaps are not vulnerabilities. If Layer A prevents the attack, the absence of Layer B is a hardening note." A third reads "Multiple runs improve coverage. In our test runs, a single run found roughly half of the vulnerabilities that repeated runs found in total." A single pass is expected to find about half of what the tool can find, which the video does not mention.

The closing claim at 34.84 s, "it gives you a detailed report with every verified issue and exactly how to fix it", does hold. VALIDATION-AND-REPORTING.md requires each confirmed finding in REPORT.md to carry "repository source location, lower-trust principal, target-native bounded reproduction, conditions, actual result, impact, priority rationale, and smallest source fix", and requires FINDINGS-DETAIL.md to include "source-level remediation and regression case" for every confirmed medium, high, or critical record. The schema reserves the remediation field for confirmed records only. Frame 020's mocked diff, - user.password = password; replaced by + user.password = await hash(password);, is a fair illustration of that output shape.

The video narrows the install path to Claude Code and drops the sandbox requirement

At 14.36 s the video says "Just install it into your cloud code and it'll map out your entire app." "Cloud code" is Whisper mishearing "Claude Code", which frame 008 confirms by animating the Claude wordmark under the caption "install".

The README does not require Claude Code. Under Requirements it lists "A coding agent with a model that supports tool use and parallel sub-agents" and "Node.js for the zero-dependency findings and coverage-ledger validators." The documented install path is the agent-neutral Skills CLI at skills.sh: npx skills add https://github.com/cloudflare/security-audit-skill --skill security-audit, with --global for a user-level install. The repository tree is a single skills/security-audit/ directory holding SKILL.md, fourteen attack-class and phase companion documents, report-schema.json, and two zero-dependency CommonJS validators with their test files.

The third requirement is the one the video drops entirely, and it is the one that changes whether the tool works. The README states: "An OS-enforced sandbox for target-controlled builds, tests, processes, browsers, emulators, fuzzers, and fixtures. It must disable external networking, use a sanitized allowlisted environment, enforce resource limits, and allow writes only to assigned scratch paths. Without these controls, the workflow keeps the lead as needs_validation instead of executing target code." Someone who installs this on an unsandboxed laptop and asks it to audit their app will get leads rather than confirmed findings, which is precisely the category the video told them would not reach the finish line.

Two more omissions are worth naming. On runtime, the blog post states "A full scan of a complex repo can take hours; the worst run took just over 14 hours." On cost, it states "Running hundreds of agents over a fleet of repos is not cheap, but at least the shape of the spend is predictable", and discloses no dollar or token figure. The skill is free to download under MIT. The model inference it drives is not.

The scope is broader than "vibe-coded apps", and the hunt list is longer than four items

The video's framing at 2.54 s is "finds security holes in your vibe-coded app", and the TikTok description repeats it as "scans vibe-coded apps for security flaws". The phrase "vibe" does not appear in the README. The repository description reads "A coding-agent skill for multi-phase security audits with independently verified, machine-readable findings", and the usage examples are generic: "security audit this codebase", "find security vulnerabilities in ./src", "do a security review, output to ~/audits/my-project". Cloudflare ran its derivative system across 128 of its own production repositories, which are not vibe-coded.

At 21.04 s the video lists the hunting surface as "your logins, database, and APIs, even prompt injection if your app uses AI under the hood." The repository ships thirteen attack-class documents beyond the core set, reaching well past web application code into memory safety, binary and kernel targets (MEMORY-SAFETY-AND-BINARY.md), supply chain, CI, release signing and plugin paths (SUPPLY-CHAIN-AND-RELEASE.md), IAM, infrastructure-as-code and container configuration (CLOUD-AND-DEPLOYMENT.md), tenant isolation and data lifecycle (DATA-ISOLATION-AND-LIFECYCLE.md), and native app, webview and local IPC surfaces (DESKTOP-MOBILE-AND-LOCAL-IPC.md). Some arrived after the initial release: the commit "Add memory-safety, binary, and kernel hunting classes for native targets" landed on 26 June 2026 from an outside contributor.

The 40-second format forces compression, and the four surfaces the video names are reasonable picks for a general audience. The claim that lands wrong is the framing, which presents a general-purpose multi-language audit harness as a safety net for one narrow category of code.

Key Takeaways

  • Correction: The video says at 0.00 s that "Cloudflare just opensource an AI skill". The GitHub API reports created_at of 2026-06-18T14:08:44Z and the accompanying Cloudflare blog post is dated 18 June 2026. The video posted on 2026-10-12, 116 days later, and the repository's last push was 2026-09-14.
  • Correction: The video says at 8.18 s that the skill "found over 7,000 security issues in their own code base". The blog post attributes the 7,245 actionable findings to the Vulnerability Discovery Harness and Vulnerability Validation System running across 128 repositories. The released artifact is the "~450-line security-audit skill that we ran on a single repository" that seeded that harness.
  • Partial correction: The video says at 11.68 s that the repository "has already crossed 20,000 stars". The GitHub API returned 32,473 stars on 2026-10-19, with 2,028 forks and 129 watchers. The stated threshold was passed well before the video and understates the count at post time.
  • Unverified: I could not source a dated star count for 2026-10-12. star-history.com reported 31.8k on 2026-10-19 with 2.9k added that week, which implies roughly 29,000 on the post date. That is an estimate from a secondary source, not a verified reading. GitHub's stargazer pagination caps far below 32,473, so per-star timestamps are unavailable.
  • Verified: The adversarial validation claim at 26.60 s is accurate. The README's Phase 3 reads "give every unique candidate to a fresh verifier that tries to disprove it", and its design principles include "Adversarial validation. The agent that checks a finding is never the agent that found it."
  • Verified: "Completely free" is accurate on licensing. The repository is MIT licensed per both the API spdx_id and the LICENSE file.
  • Verified: The closing claim at 34.84 s about a report with fixes holds. VALIDATION-AND-REPORTING.md requires each confirmed finding to carry the "smallest source fix" in REPORT.md and "source-level remediation and regression case" in FINDINGS-DETAIL.md.
  • Correction: The video says at 31.84 s that "only the real issues make it to the finish line". The skill emits three verdicts, confirmed, needs_validation, and rejected. The reporting doc forbids assigning severity to needs_validation records or calling them confirmed vulnerabilities, so unresolved leads do reach the report as a separate, explicitly unscored table.
  • Clarification: The video says at 14.36 s to "install it into your Claude Code". The README requires only "A coding agent with a model that supports tool use and parallel sub-agents" plus Node.js, and documents the agent-neutral npx skills add path.
  • Clarification: The video omits the README's third requirement, an OS-enforced sandbox with external networking disabled. Without it, "the workflow keeps the lead as needs_validation instead of executing target code", which blocks the confirmed findings the video promises.
  • Context: The blog post states "A full scan of a complex repo can take hours; the worst run took just over 14 hours" and that "Running hundreds of agents over a fleet of repos is not cheap". No token or dollar figure is disclosed. The skill is free; the inference it drives is not.
  • Context: The README states "In our test runs, a single run found roughly half of the vulnerabilities that repeated runs found in total." A single pass is expected to miss about half of what the tool can find.
  • Context: The three bylined authors shown in frame 004, Dan Jones, Alexandra Godoi, and Grant Bourzikas, each have an author page on the Cloudflare blog, and Dan Jones is the committer of record on the repository's initial commit. The spellings on screen are correct.
  • Context: The lineage runs through "Project Glasswing: what Mythos showed us" by Grant Bourzikas, published 18 May 2026, one month before the skill and harness post. The video does not mention it, though it is visible as a link in the blog screen recording at frame 004.
  • Clarification: The terminal listings in frames 004, 005, and 006 are designed b-roll, not captured output. Frame 004 shows four consecutive rows labelled "IDOR on record ID" against four different file paths, and edge/router/headers.rs is tagged "Open redirect" in one frame and "SQL injection risk" in the next. No frame I read shows a real run of the tool.

Resources

Published October 12, 2026. Writeup generated from a favorited TikTok.