Mistral's Leanstral 1.5 Finds 5 Unknown Bugs Free

Formal verification has spent decades as a tool for aerospace engineers and cryptographers - too expensive, too slow, too specialized for everyday software. Mistral AI's Leanstral 1.5, released on June 30, 2026 under an Apache 2.0 license, is a direct challenge to that assumption. The model doesn't just solve math competition problems. Mistral put it to work on real open-source Rust code and it found bugs the original developers hadn't reported on GitHub.

The original Leanstral from March beat Claude Sonnet 4.6 on formal proof benchmarks at a fraction of the cost. Leanstral 1.5 picks up where that left off - now saturating the miniF2F benchmark completely and posting state-of-the-art numbers on graduate-level math tasks.

What Leanstral 1.5 Actually Does

Lean 4 is a proof assistant: a programming language where you write specifications with code, and a mechanical checker verifies every logical step. If you can write a formal spec and prove your code meets it, you eliminate entire classes of bugs by construction. The catch has always been that writing Lean proofs requires expertise most engineering teams don't have.

Leanstral 1.5 handles the proof-writing. You describe what your code should do, and the model creates a Lean 4 proof, checking it against the compiler in a feedback loop until it either succeeds or times out.

From Benchmarks to Bug Hunting

The benchmark numbers are strong. Leanstral 1.5 solves 587 of 672 PutnamBench problems - competition-level university math - and scores 87% on FATE-H, a graduate algebra benchmark where most models stall. It saturates miniF2F completely at 100%.

The more interesting test was a practical one. Mistral's team ran Leanstral 1.5 through a Rust verification pipeline using Aeneas, which translates Rust code into Lean. Across 57 open-source repositories, the system flagged 47 violated properties. Eleven pointed to genuine bugs. Five of those had never been filed on GitHub.

The Bug Mistral Actually Found

One of the newly discovered issues is concrete enough to explain exactly. In the datrs/varinteger Rust library, the zigzag decoding function's sign logic uses an expression that, on input u64::MAX, increments before shifting. That increment overflows on unsigned 64-bit integers - wrapping to zero in release builds, crashing in debug mode. It's exactly the class of edge case that fuzzing misses because u64::MAX appears so rarely in random test data that statistically it almost never comes up.

Leanstral caught it by proving a formal property about the function's behavior across all possible inputs, not just sampled ones.

Leanstral 1.5 targets Rust codebases through an Aeneas pipeline that translates Rust into Lean 4 for formal verification. Source: unsplash.com

Under the Hood

Architecture

Leanstral 1.5 uses a mixture-of-experts design with 128 experts and 4 active per forward pass. Total parameter count is 119B, but only 6.5B activate per token - which keeps inference costs low enough that Mistral can offer the model free during beta. The 256k context window matters here: Mistral reports one AVL tree complexity proof ran for 2.7 million tokens across 22 iterations, pushing through the context limit with multi-turn memory management.

Training Pipeline

Training ran through three stages: mid-training on formal math and code corpora, supervised fine-tuning on Lean 4 proof data, and reinforcement learning using CISPO. The RL stage used two environments. The first was a multiturn proof setting where the model attempts theorems and receives Lean compiler feedback to refine its approach. The second was a code agent environment with filesystem access, bash commands, and real-time Lean language server access - basically the same setup a human developer would use.

Test-time compute scaling works as expected: performance improves as more budget is allocated, which makes the model well-suited to automated pipelines where difficult proofs can get additional compute.

Getting Started

The model is free through Mistral's Labs API under the identifier labs-leanstral-1-5. A minimal test:

from mistralai import Mistral

client = Mistral(api_key="<YOUR_API_KEY>")

response = client.chat.complete(
    model="labs-leanstral-1-5",
    messages=[
        {
            "role": "user",
            "content": "Prove that the length of a concatenated list equals "
                       "the sum of the individual lengths in Lean 4."
        }
    ]
)

print(response.choices[0].message.content)

For self-hosted deployments, weights are on Hugging Face under Apache 2.0 and the model runs on vLLM with standard configuration.

Requirements

Formal verification gives mathematical guarantees that code behaves correctly across all inputs - something fuzzing and unit tests can't provide. Source: unsplash.com

Benchmark Comparison

The FLTEval pass@8 comparison with Claude Opus 4.6 deserves a note: pass@8 means running eight attempts and counting a problem solved if any one succeeds - a weaker standard than pass@1. The numbers aren't directly comparable to a single-shot Claude evaluation. What they show is that with a reasonable compute budget, Leanstral 1.5 can reach correct proofs that other models don't.

All benchmark scores are vendor-reported by Mistral. No independent replication from Epoch AI or similar groups is available yet.

Where It Falls Short

The beta endpoint retires on September 30, 2026. Teams building workflows around labs-leanstral-1-5 will need to migrate to a permanent endpoint - which Mistral hasn't announced yet - or switch to self-hosted weights. Six weeks isn't a long window for a production evaluation.

The practical Rust pipeline requires Aeneas, and getting Aeneas to translate Rust cleanly isn't always automatic. Complex ownership patterns and trait implementations sometimes need manual hints. Mistral's 57-repository test was a curated selection, not a random slice of real-world code, so the 11-genuine-bugs-out-of-47-flags ratio shouldn't be generalized too broadly.

The vendor self-reporting issue applies across. Until third-party evaluators publish results, treat the benchmark numbers as directionally correct rather than conclusive.

And the model is truly experimental in the ML sense: built to push the frontier on formal verification benchmarks, not as a polished developer tool. Expect sparse documentation and a non-trivial learning curve if your team doesn't already know Lean 4.

Mistral has published the weights, kept the license clean, and shown that the model can catch bugs that fuzzing misses. For teams doing safety-critical work in Rust, the cost of evaluation is effectively zero until October. That's a hard set of conditions to ignore if a silent u64::MAX overflow in your codebase is unacceptable.

Sources:

• Leanstral 1.5: Proof Abundance for All - Mistral AI
• Mistral AI Releases Leanstral 1.5 - MarkTechPost
• Mistral's open-source Leanstral 1.5 aces formal math benchmarks - The Decoder
• Leanstral 1.5 Model Card - Mistral Docs
• Mistral releases Leanstral 1.5 - TestingCatalog