Cloudflare's Security Audit Skill Is the 450-Line Seed, Not the Harness That Found 7,245 Bugs

The GitHub repository cloudflare/security-audit-skill gained 2,874 stars in a day, according to the lead sheet The Terminal works from, and shows 46.4k stars in total on its page. What it contains is a coding-agent skill that runs a six-phase audit on one codebase. What Cloudflare reports finding with its larger system, 7,245 actionable findings across 145 repositories, comes from a harness the company has said it will release "shortly" and has not.

That distinction decides whether the repository is a security tool or a template. It is MIT-licensed, and its README calls it the seed of Cloudflare's vulnerability discovery harness.

What the skill does and what it needs

The README says the skill "orchestrates isolated agents through reconnaissance, coverage-led hunting, candidate validation, structured output, independent record verification, and target-neutral reporting." Install is one command: npx skills add https://github.com/cloudflare/security-audit-skill --skill security-audit.

The phases produce files. Reconnaissance writes architecture.md and coverage-ledger.json. Hunters check ledger units while critics look for gaps. A fresh verifier tries to disprove each candidate, and results land in findings.json and then REPORT.md, FINDINGS-DETAIL.md and NEEDS-VALIDATION.md.

It needs a coding agent with tool use and parallel sub-agents, plus Node.js for the validators. Target code runs only in an OS-enforced sandbox. Without one, leads stay in needs_validation and nothing from the target is executed, which is the right default for software you did not write. The README's design principle reads "Only confirm established boundary failures."

The page shows 14 commits, 33 open issues and 63 open pull requests. Stars measure attention, not adoption, and none of those figures says how many teams run it.

What Cloudflare's own post says about it

Grant Bourzikas published "Build your own vulnerability harness" on Cloudflare's blog on June 18, with a July 15 update. The page gives no job title for him. The skill began as a roughly 450-line prompt set run against one repository, and the team tuned it until it surfaced real bugs.

The post is candid about the limit: "a single run finds only about half the bugs you'd catch across multiple runs," and the bugs found skewed toward simpler ones. The README repeats the half figure. Anyone installing the skill should plan on repeated runs.

The two documents also disagree on small details. The post describes the skill as a seven-phase audit in one session, while the README lists six phases. The difference is probably how the stages are counted, but it is a reminder that the blog post describes the system as it stood in June.

The numbers belong to the harness

Cloudflare's figures are point-in-time and vendor-supplied. The company's Vulnerability Discovery Harness produced 20,799 raw candidates, of which about 12,057 survived validation. A second system, the Vulnerability Validation System, runs on a different model so the two check each other. It holds 13,841 findings across 145 repositories.

1. Actionable7245 findings
2. Folded away as duplicates5442 findings
3. Wrong repo, other or not a risk1154 findings

Source: Cloudflare blog, Build your own vulnerability harness, accessed 2026-10-25 (vendor-supplied)

The three buckets add up to the pool: 7,245 plus 5,442 plus 1,154 is 13,841. Duplicates alone account for 39 percent of what the system produced, which is why the post says deduplication needs its own agents, since string matching cannot tell whether two logic flaws share a root cause.

The post says validation rejection fell from 40 percent to 11 percent and the share of high-integrity findings rose from 35 percent to 58 percent. The fleet scanner covered 128 repositories, the worst scan took just over 14 hours, and a typical repository of about 30,000 lines yielded roughly 100 initial findings that compressed to 80 distinct bugs in about 3 hours.

What the post does not claim

Cloudflare says plainly that it does not claim a false-negative rate, because no labelled set of all real bugs exists. Progress is measured by whether re-runs keep surfacing new bugs. It also does not name the models. The findings come from what the post calls an isolated research experiment rather than active unpatched production vulnerabilities, and Cloudflare says its security teams are applying fixes.

One operational detail is worth more than the headline counts. The post says the hunters invoked Semgrep, a conventional static analysis tool, zero times in a month, preferring to read and run code. The most-used tool was a "Wishlist" that let a hunter request evidence it was missing, written 25,472 times. Each finding ships with a proof-of-concept test run against unpatched code and a proposed diff, and a validator cannot file findings itself. A human reviews every branch.

What this means for anyone installing it

The skill gives a team the loop and the discipline: separate hunters from verifiers, never let the finder confirm its own bug, refuse to execute target code outside a sandbox. It does not give the producer-consumer pipeline, the deduplication system or the second model that checks the first. Expecting the Cloudflare numbers from the repository would be a mistake of scale.

Sandboxing is the other half of the problem. NVIDIA's OpenShell sandbox reached 19,500 stars with telemetry on by default, a reminder to read what an agent tool sends home. Finding bugs is also only the start: a PHP2Shell patch round left a third flaw unfixed. Plex2Shell's discoverer, Zach Main, wrote that he used language models in a four-step loop of recon, hunt, verify and chain, close to the structure Cloudflare describes.

The next thing to watch is the harness release Cloudflare promised in the post. Until it appears, the 7,245 figure is a claim about a system outsiders cannot run.