Claude Generates 180M Lines of Proof Code to Find Bugs Missed by Trillions of Tests

Is Claude about to eliminate global software bugs?

In just 48 hours, the world witnessed 27 major cyberattacks. Now, people are buzzing with talk of "The Darkest October."

Early this morning, Anthropic’s Frontier Red Team, in collaboration with Theorem Labs, released a heavyweight report—

They directed Claude to dive deep into low-level assembly instructions (RISC-V), launching a comprehensive assault on real-world, production-grade open-source codebases.

In just a few weeks, Claude achieved formal verification for 239 test entry points across 123 open-source repositories, generating a staggering 180 million lines of Rocq mathematical proof code, all machine-verified!

Hidden vulnerabilities that Google’s platform missed after trillions of tests, along with high-risk logical blind spots lurking for over a decade, and even genuine defects in x86 processor hardware itself, were all exposed during this massive purge.

Even more astonishingly, within a popular PDF reader, the security vulnerabilities Claude discovered in one week matched its typical annual reporting volume.

This time, Claude integrated bug discovery, fixing, and verification into a single workflow, executing a "mathematical-grade" bug sweep.

The global cybersecurity crisis might actually have a solution.

Bugs Missed by Trillions of Tests

Forced Out by Mathematics

For the past decade, one of the most relied-upon defenses in the open-source world has been OSS-Fuzz, launched by Google in 2016.

This method is known as fuzz testing: it floods program entry points with batches of semi-random inputs to see which ones cause crashes. It has accumulated trillions of test runs.

However, fuzz testing ultimately relies on luck within a maze. Its inputs are typically limited to under 1MB; as long as there is an unexplored path, bugs can remain hidden there indefinitely.

The research team reframed the testing entry point as a mathematical proposition. Under specified conditions, any input smaller than 64 MiB will not trigger the defined crashes, including assertion failures and errors detectable by AddressSanitizer (ASan).

Claude’s task was to prove this. A single successful run is insufficient; every possible input within the specified range and every reachable execution path must be proven free of these errors.

If the proof stalls, it means either Claude hasn’t found a valid proof strategy yet, or there is an actual bug in the program. Upon finding a bug, Claude patches it itself and then continues the proof.

Consequently, bugs hiding in corners unreachable by traditional tests were exposed.

For instance, a decompressor contained a counter underflow that could write data beyond memory boundaries. This issue only triggered when fed inputs of approximately 20 MB with a rare layout.

The research team stated that the mere process of attempting proofs uncovered hundreds of bugs.

To quantify the difference between testing and proving, researchers deliberately planted seven bugs in a Python interpreter written in Rust by Claude.

Random testing missed five of them, while formal verification caught all seven.

391 Lines of Code, 630,000 Lines of Proof

These proofs were conducted at the lowest level.

The object of Claude’s verification was RISC-V machine code actually executed on the CPU after compilation. Source code and compiled binaries are not necessarily identical; only by verifying machine instructions can one prove properties of the program as it actually runs.

To achieve this, Theorem first modeled RISC-V instructions and Linux process behavior mathematically. Current proofs are built upon this model, restricted to environments with single-threaded execution, static linking, and fixed memory budgets.

Verifying at this layer caused the engineering effort to explode. For the JSON parsing library cJSON, a single parsing entry point consisting of only 391 lines of C code required Claude to generate 630,000 lines of Rocq proof.

This thousand-fold expansion occurs because the proof must cover the entire process. The startup sequence of the C program, the memory allocator, printf, and even the runtime of the memory detection tools themselves all had to be verified. Only about one-tenth of the proof related directly to cJSON itself.

Across the entire project, Claude generated approximately 1 billion lines of proof, of which about 180 million passed verification. More than half of these were automatically generated by tools Claude wrote for itself.

The final gatekeeper was the Rocq proof checker, which validates each step according to logical rules. Verifying a single proof requires anywhere from 30 minutes to several hours of CPU time. While the checker confirms that the proof adheres to the specification, humans must still review whether the specification and the machine model accurately reflect real-world requirements.

No matter how elegantly the model is written, if it fails the checker, it counts for nothing. Generating proofs of this scale was previously considered an unimaginable engineering burden.

The legendary microkernel seL4 took two person-years to write but required twenty person-years for formal verification. For decades, security guarantees of this level were affordable only for life-critical systems like Airbus flight controls.

Yet this time, Claude processed 123 open-source libraries in just a few weeks.

Claude Wrote Mathematics for Chips

And Uncovered Hardware Bugs Along the Way

To understand how deep Claude’s proofs can dig, look no further than cpp-httplib.

To rule out stack overflow due to excessive recursion depth, Claude delved into the red-black tree implementation of the C++ standard library (libc++). It proved at the machine-code level that the height of this tree remains bounded logarithmically. By establishing that the tree height is controlled, it ensured an upper bound on recursion depth during destruction, thereby proving that stack space would not be exhausted.

It first formalized the zlib compression format specification, then proved that the zlib implementation included in the Linux kernel fully complies with it.

In this instance, the verification was performed on x86 machine code. Notably, the mathematical model of the x86 instruction set was written entirely autonomously by Claude.

How precise is this model? By stepping through instructions one by one within the model, it can fully boot Ubuntu’s user space on the Linux kernel.

During the process of writing the model, Claude identified hundreds of errors in the official x86 reference documentation and even discovered a few bugs hidden within the CPU hardware itself.

From software all the way down to chips, AI is already helping humanity re-verify the foundation of the entire computer.

Proof speed doubles every two months

A $134 million verification roadmap

The biggest barrier to formal verification is cost; a single proof often burns through hundreds of millions of tokens. However, this bill is shrinking rapidly.

Anthropic tested 36 small-scale verification tasks, finding that Opus 5.5 required approximately 40% fewer tokens than Opus 5 to complete half of them.

Theorem tested seven frontier models released between May and September of this year, observing that verification speed roughly doubles every two months.

Moreover, proofs are reusable.

After agents shared workspaces, the same set of tasks consumed 17% fewer tokens. Projects as unrelated as libcups, freerdp, and libzmq share underlying code such as runtime libraries. Consequently, large portions of proofs were copied from parts verified by others, slightly modified, and then reused.

Following this logic, Theorem calculated the total cost using the Nix software ecosystem as a sample.

In the Nix software ecosystem, just 9 MB of shared base libraries account for the dependencies of two-thirds of all machine code. OpenSSL alone appears in the runtime dependencies of nearly 16,000 software packages.

At current prices, fully verifying OpenSSL once costs approximately $220,000. Amortized across each downstream package, this amounts to only about $14 per package.

Theorem estimates that if verification capability continues to double every two months, and progress proceeds with an annual budget of $100 million, by the end of 2027, with a total cost of approximately $134 million, it will be possible to include 95% of the machine code in the Nix ecosystem under formal proof.

For comparison, GovAI estimates that global losses, response costs, and defensive expenditures related to cybercrime total approximately $500 billion annually.

Mathematics as the last line of defense during the darkest 48 hours in global cybersecurity

The Linux kernel is one of the most rigorously tested pieces of software in the world. A Debian kernel security advisory from September listed over 1,000 vulnerabilities. Frontier large models can now chain together several seemingly harmless minor bugs into a fatal attack chain.

As AI becomes increasingly powerful at finding bugs, humans patching them one by one can no longer keep up with its speed.

Aengus Lynch, one of the paper's authors, wrote, "These are terrifying days for the internet. But I believe they are also a beacon of hope."

He also believes that one day, Theorem will formally verify all critical software.

Currently, proofs for the first batch of 25 libraries and 60 entry points have been published on GitHub, allowing anyone to download them and manually verify the calculations.

With more machines writing code, humanity is paving a path to ensure that the critical code supporting the digital world comes with mathematical proofs.

References:

https://x.com/AnthropicAI/status/2113390212911083894?s=20

Chasing ASI in Seconds

⭐ Like, Share, and Watch — One Click Combo ⭐

Star us to lock in instant updates from New Intelligence Era!