Microsoft's New Cyber AI Scores 96% - Leaderboard Disagrees

Nine days ago, Project Perception was a rumor built from leaked org charts and reporting on Microsoft's security layoffs. On July 27, Microsoft made it official: a new cybersecurity-specialized model called MAI-Cyber-1-Flash, an agentic platform to go with it, and a benchmark number Microsoft says beats every competitor on the market.

The number is 95.95% on CyberGym, the industry's go-to test for AI vulnerability work. Microsoft is calling it a win over Claude Mythos 5, GPT-5.5-Cyber, and Google's Gemini 3.5 Flash Cyber. What Microsoft isn't mentioning is that CyberGym's own public leaderboard doesn't list that score, and a competitor posted a verified, leaderboard-ranked number the exact same day.

What Actually Shipped

Strip away the benchmark fight and there's a real product launch underneath it. MDASH is Microsoft's internal vulnerability-management pipeline, and MAI-Cyber-1-Flash is the model built to run most of it cheaply.

The Model: A Cheap Specialist, Not a New Flagship

MAI-Cyber-1-Flash is a 137B-parameter mixture-of-experts model with roughly 5B active parameters and a 256K context window, fine-tuned from Microsoft's MAI-Code-1-Flash, itself descended from the MAI-Thinking-1 reasoning line. It's text-only, trained on Microsoft's own exploit and remediation records, and it isn't meant to out-think a frontier model on the hardest problems. It's meant to be cheap enough to run constantly.

The routing logic is the actual innovation here: MAI-Cyber-1-Flash handles roughly 90% of MDASH's workload on its own, escalating the hardest 10% to OpenAI's GPT-5.4. Microsoft says that combination cuts costs by half compared to its prior MDASH setup, which leaned on GPT-5.4, a GPT-5.4-mini variant, and GPT-5.3-Codex together. No token pricing or call-volume numbers back up the 50% figure; Microsoft's announcement doesn't disclose them.

Project Perception's Three Agent Teams

The platform layered on top, Project Perception, organizes work into three agent groups: red teams that simulate attacks, blue teams that triage what the red teams find, and green teams that write and deploy fixes. High-impact actions still require a human sign-off.

Microsoft's Redmond campus, where the security organization now run by EVP Hayete Gallot is based. Source: wikimedia.org

Hayete Gallot, the security chief whose executive reshuffle drove our earlier coverage, described it this way:

"Project Perception brings together signals, context, models and specialized agents into a continuously learning system that can reason, prioritize and act at machine speed while keeping humans firmly in control."

Corporate VP David Weston framed the goal in blunter terms: work that used to take "hours and hours of manual work from multiple specialized folks" should now take minutes. Project Perception enters public preview on August 3, starting inside Microsoft Defender, with the model itself available on Azure AI Foundry to vetted customers and billed through a consumption-based system Microsoft calls Security Compute Units.

The Benchmark Claim Doesn't Hold Up The Way It's Presented

Here's the part Microsoft's launch materials gloss over.

Self-reported AI security benchmarks are common. Independent verification against a public leaderboard isn't. Source: pexels.com

What CyberGym Actually Measures

CyberGym's Level 1 task isn't "find a vulnerability." It's narrower: given a text description of a known flaw and the pre-patch codebase, the agent has to produce a working proof-of-concept that triggers the bug on the vulnerable version and not on the patched one. That's closer to confirming a report already sitting in a triage queue than discovering an unknown flaw from scratch. The benchmark's harder sibling, CyberGym-E2E, strips out the description and requires blind discovery plus a patch - and even the best systems there clear only about 20% of tasks. Microsoft's 95.95% is a Level 1 number, not an E2E one.

What They Didn't Mention

The 95.95% is also a system score for the full MDASH pipeline, including the GPT-5.4 escalation path, not a number for MAI-Cyber-1-Flash running alone. On several harder, more adversarial evals, the standalone model's numbers are far less flattering, including a flat zero across three ExploitGym categories.

The Hacker News reported that CyberGym's public leaderboard didn't list Microsoft's 95.95% result as of July 28, 2026, more than a day after the announcement.

Wiz Showed Up The Same Day, With A Verified Number

While Microsoft was posting a number nobody could check, Wiz launched Project Atlas, a multi-agent vulnerability researcher built by combining several existing models rather than training one. Atlas currently sits at the top of CyberGym's actual public leaderboard with 90.9%, a lower number than Microsoft's claim but one anyone can look up.

Wiz argues system design, not a bigger base model, is what actually moves benchmark results like these. Source: pexels.com

Wiz's own comparison puts Mythos at 83% and GPT-5.5-Cyber at 85% on the same leaderboard, close to but not identical with the 83.2%-85.6% range Microsoft cites for the same two systems - a reminder that even the baseline numbers shift depending on who's running the test. Atlas has reportedly surfaced more than 200 previously unknown vulnerabilities in open-source projects, including one GitHub bug that earned a $100,000 bounty.

Katie Norton, an IDC analyst tracking the space, put the bigger picture in perspective:

"Frontier AI models can already reason about vulnerabilities," but "discovery is outpacing the industry's ability to coordinate remediation and disclosure."

That's the part none of these launches solve. Finding bugs faster doesn't help if the patching and disclosure pipeline behind it can't keep pace, and every vendor in this race is currently optimizing for the first half of that problem.

Should You Care?

If you're a security team assessing any of this, the routing architecture is the legitimately interesting part. Splitting cheap triage work from expensive escalation is sound engineering, and it's the same logic OpenAI's Codex Security and Anthropic's Project Glasswing are both betting on in different forms. Cheaper continuous scanning, if it holds up, is worth having regardless of who ships it first.

The 95.95% headline is a different matter. Treat any self-reported AI security benchmark as provisional until it shows up on the leaderboard it claims to lead, and treat "beats Mythos" claims from Microsoft, Wiz, and everyone else in this fight with the same skepticism, since none of them are running each other's evaluations. Microsoft has two weeks between now and its August 3 preview to either post that number where CyberGym's maintainers can check it, or quietly let it fade from the pitch deck.

Sources:

• Microsoft launches its first cybersecurity model, plus a new agentic cybersecurity system - TechCrunch
• Introducing MAI-Cyber-1-Flash inside MDASH - Microsoft AI
• Microsoft's first cybersecurity model powers new Project Perception agents - SiliconANGLE
• Microsoft Says New Cybersecurity AI Model Helps MDASH Score 95.95% at Half the Cost - The Hacker News
• Microsoft AI Releases MAI-Cyber-1-Flash - MarkTechPost
• Introducing Atlas: Wiz's AI vulnerability researcher - Wiz Blog
• Wiz And Google Have An Answer To Mythos, And It's Not A Model - Forbes
• CyberGym: Evaluating AI Agents' Real-World Cybersecurity Capabilities at Scale - arXiv