Anthropic Claims AI Proof Generation

Anthropic has integrated its Claude model with partner Theorem to automatically verify existing software code and generate formal proofs of its correctness. This initiative, led by Anthropic's Frontier Red Team, attempts to solve a fundamental challenge in software development: proving that code runs without error under all possible conditions. While traditional formal verification is prohibitively expensive, requiring significant human resources, this approach aims to automate the process using artificial intelligence.

The core of the project involves using Claude to mathematically prove that existing code is correct. Historically, establishing such proofs has been a resource-intensive task. For example, a team of specialists required 20 person-years to formally verify the seL4 operating system kernel. Consequently, this rigorous method has been reserved for extreme cases like avionics software used by Airbus. Anthropic's approach seeks to make this level of assurance more accessible.

Working with Theorem, the Frontier Red Team directed Claude to generate proof work rather than new code. The model produced roughly 180 million lines of mathematical proof code in a few weeks. This work utilized test programs from 123 open-source libraries within Google's OSS-Fuzz security project. The process involves translating crash tests into mathematical problems, analyzing compiled machine code, and producing extensive proofs.

An independent verification program named Rocq automatically checks the proofs generated by Claude. According to Anthropic, this process requires almost no human review, though the underlying tools and formal foundations were built by humans at Theorem. The researchers refer to this as "fully autonomous" proof generation, highlighting the potential for large-scale, automated verification of software.

During the verification process, Claude encountered hundreds of bugs and was permitted to resolve them independently. Many of these issues would have evaded conventional testing because they only appear under specific conditions, such as unusually large input files or a particular sequence of access operations. For instance, in one week of verifying a widely used PDF reader, Claude found as many security bugs as are typically reported for that software in an entire year. In another case, it caught a logical error that had gone undetected for over a decade.

It is important to note the limitations of this method. Anthropic acknowledges that "formally verified" does not mean "error-free." The proofs are only valid under strict conditions, including a specific processor architecture, single-threaded programs, and inputs under 64 MB. In reality, these libraries operate on various hardware and in different configurations, meaning the proofs cannot be applied universally.

The compute cost for this work is substantial, with individual proofs running up hundreds of millions of tokens. Anthropic does not report total project costs. However, if AI can verify its own code, developers can trust AI-generated software more and spend less time reviewing it. This saves costs not just on verification but on the entire coding process.

Furthermore, a smaller test run indicated that proofs are reusable. Newer models are also becoming more efficient, with Opus 5.5 solving half the proofs with about 40 percent fewer tokens than Opus 5. This project strengthens the case for software automation by shifting the question from whether AI can program to whether AI can prove that the code it writes is safe. If the answer is yes, the implications could extend far beyond current coding tools.

Anthropic expects formal verification with AI to soon become as practical as standard testing. Theorem's roadmap already targets critical infrastructure like glibc, OpenSSL, and cURL. This suggests a future where ensuring software safety is a routine part of development, not a rare and costly endeavor. The goal is to make such verification as common as running standard tests on any piece of software.