OpenAI Buys the Tool Used to Test Its Own Models

OpenAI announced on Monday it'll acquire Promptfoo, the open-source AI security platform that has become the closest thing the industry has to a standard red-teaming toolkit. The deal folds the most widely used independent AI testing tool directly into the company whose models most people use it to test.

TL;DR

OpenAI picks up Promptfoo, used by 300,000+ developers and 30-plus Fortune 500 companies
Promptfoo will integrate into OpenAI Frontier, the enterprise AI agent platform
Anthropic and Google teams also used Promptfoo - now their security scanner belongs to a competitor
OpenAI pledges to keep the open-source project alive under its current license
Terms not disclosed; Promptfoo raised $23.4M total before the acquisition

The Deal

Promptfoo was founded in 2024 by Ian Webster and Michael D'Angelo. Webster had spent years running the LLM engineering team at Discord - shipping AI to 200 million users - and grew frustrated that the security tools available were designed for a different era of software. He built Promptfoo as a side project, then launched it commercially with $5 million in seed funding from Andreessen Horowitz. Discord's own CTO, Stanislav Vishnevskiy, backed it personally, as did Shopify CEO Tobi Lütke.

The commercial pitch gained traction quickly. A $18.4 million Series A from Insight Partners followed in July 2025, bringing total pre-acquisition funding to roughly $23.4 million.

What Promptfoo Does

Rather than relying on manual penetration testing, Promptfoo sends specialized models against a customer's AI application through its own chat interface or APIs. Those models behave like users, or specifically like attackers. When an attack succeeds, the platform records it, analyses why it worked, and iterates through a reasoning loop to find deeper vulnerabilities.

The risks it covers span prompt injection, data leakage, jailbreaks, and what the company calls application-level failures: AI systems that promise users things they can't deliver, reveal database contents in response to a customer service query, or drift into political opinion inside a homework tutor. It runs tests against 50+ vulnerability types, drawing on adversarial research from Microsoft, Meta, and academic groups.

The Numbers

The open-source command-line tool has been downloaded by more than 300,000 developers. The commercial version has 125,000 active users and over 30 Fortune 500 customers. According to Insight Partners, Promptfoo's growth follows a classic open-source path: developers adopt it for free, then companies pay for the managed reporting, traceability, and compliance features the enterprise tier adds.

A developer running security tests against an AI application at a terminal Promptfoo automates the adversarial testing process - running specialized attack models against AI applications to surface vulnerabilities before deployment. Source: unsplash.com

The Problem With the Acquirer

Promptfoo was vendor-neutral by design. Its documentation explicitly compares GPT, Claude, Gemini, and Llama models in head-to-head security tests. Anthropic and Google teams used it to red-team their own systems. Researchers from independent labs cited it as a shared baseline.

That independence is now gone.

OpenAI says it'll continue developing the open-source project under its current license - a pledge that matters because much of Promptfoo's credibility comes from the community that built test cases around it. But maintaining a neutral open-source project while owning the company that monetises it within a competing enterprise platform is not a tension-free arrangement. The full Promptfoo team joins OpenAI, and the commercial product integrates into OpenAI Frontier.

"Promptfoo brings deep engineering expertise in evaluating, securing, and testing AI systems at enterprise scale," said Srinivas Narayanan, CTO of B2B Applications at OpenAI.

The statement says nothing about what happens to customers running Promptfoo against Claude or Gemini. That question hasn't been answered publicly.

Where Frontier Comes In

OpenAI Frontier is the company's enterprise platform for launching AI agents - "AI coworkers" in OpenAI's framing - across production systems. Its early customers include Uber, State Farm, Intuit, and Thermo Fisher Scientific. Unlike ChatGPT, Frontier connects agents to CRM platforms, internal data warehouses, and operational tools, which notably expands the attack surface.

Risk Category	Examples
Prompt injection	Malicious input in CRM data redirecting agent actions
Data exfiltration	Agent reading and leaking files outside its scope
Tool misuse	Agent calling APIs in ways that violate policy
Jailbreaks	Users bypassing guardrails on the chat interface
Application failures	Agent making promises it can't technically fulfill

The integration announcement frames the acquisition as a direct response to this expanded surface. Promptfoo's automated red-teaming runs during development, before a Frontier agent goes live, and the compliance reporting feature helps enterprises document what tests they ran and what the results showed.

This isn't OpenAI's first move into security tooling. Codex Security launched in research preview just two days ago, scanning code repositories for vulnerabilities using AI. Promptfoo is the other side of the same bet: if OpenAI controls both the platform and the security testing layer, it can offer enterprise buyers a single contract that covers development, deployment, and compliance.

A padlock representing AI security and access control Access control and security testing for AI agents are becoming core enterprise requirements as platforms like Frontier connect agents to production systems. Source: unsplash.com

The Open Source Question

Promptfoo has 248 contributors on GitHub, accumulated over two years of community pull requests. The project's value isn't just the code - it's the database of attack patterns, test case templates, and shared benchmarks that the community built. OpenAI has pledged to maintain it under the current open-source license.

For context: when Elastic moved its license away from open source in 2021, AWS forked it and launched OpenSearch the same week. The developer community has demonstrated it'll act when a beloved project stops being neutral. OpenAI knows this, which explains the pledge.

Whether the pledge holds under pressure from a commercial roadmap is a separate question. Ian Webster hasn't said publicly what Promptfoo's open-source arc looks like inside a company that creates most of its revenue from proprietary API access.

For those wanting independent options, our review of AI red teaming and cybersecurity platforms covers 15 alternatives, several of which have no commercial tie to model providers.

The acquisition reflects a genuine market pressure. Enterprise AI deployment requires security testing, and enterprises want that testing integrated into the same platform where they build and run their agents. OpenAI is solving a real customer problem. The part worth watching isn't whether Promptfoo's technology improves inside OpenAI - it probably will. The part worth watching is whether the 300,000 developers who built their security workflows around a neutral tool choose to stay.

Sources: