Best LLM Red Teaming Tools 2026: 6 Scanners Compared

Four of the best-known names in AI red teaming got bought out in the past two years, and not by failing companies looking for an exit. OpenAI acquired Promptfoo in March, a tool with over 150,000 developer users that OpenAI itself was already running internally. Check Point is buying Lakera in a deal reported at roughly $300 million. Palo Alto Networks closed on Protect AI last year, and Cisco bought Robust Intelligence for a reported $400 million before that. Every major security vendor decided it needed to own a way to methodically attack language models before someone else does it for them, in production.

That's the category this article covers: tools that probe an LLM application for jailbreaks, prompt injection, data leakage, and unsafe outputs before it ships. It's a different job from the LLM firewall products we tested last month. A firewall sits in the request path and blocks bad traffic at runtime. A red teaming tool runs offline, throws thousands of adversarial prompts at your model or agent, and hands you a report of what broke. You need both, but they're not interchangeable, and vendors selling one increasingly want to sell you the other too - which is exactly what's driving the acquisitions above.

What "red teaming" means for an LLM

Traditional red teaming means hiring people to break into a network. AI red teaming means methodically generating adversarial inputs - jailbreak phrasing, encoded payloads, multi-turn manipulation, malicious tool calls - and checking whether the model or the application wrapped around it does something it shouldn't. Most tools in this space test against the OWASP Top 10 for LLM Applications, which covers prompt injection, sensitive information disclosure, supply chain risks, and excessive agency, among others.

The tools split cleanly into two groups. One group is open source, runs from your terminal or CI pipeline, and costs nothing but engineering time to operate. The other is a hosted platform with a dashboard, scheduled scans, and a sales team. Both groups can find real vulnerabilities. They differ in who does the work of triaging results and keeping the attack library current.

Quick comparison

Promptfoo

Promptfoo started as an LLM evaluation CLI and added red teaming as a first-class feature. It's the most starred tool in this comparison at 24,174 GitHub stars, MIT-licensed, and according to its own repository description it's already used internally by OpenAI and Anthropic to test their own models. You define a promptfooconfig.yaml, point it at a target - an API endpoint, a local model, an agent - and it runs a battery of adversarial plugins covering jailbreaks, PII exfiltration, excessive agency, and more, then scores the results.

The dashboard groups findings by plugin and severity and shows the exact attack string that succeeded with the model's response and the grader's reasoning for flagging it. That level of detail matters when you're trying to convince an engineering team a finding is real and not a scoring artifact.

Promptfoo's vulnerability report groups findings by severity and risk category, with per-category pass rates. Source: promptfoo.dev

Pricing stays simple: the open-source core is free and includes red teaming, evaluation, and CI/CD integration with no feature gate. A $50/month Team tier adds cloud collaboration and shared dashboards. Enterprise is custom-quoted and adds SSO, RBAC, audit logging, and on-prem deployment. OpenAI has said Promptfoo will stay open source under its current license after the acquisition closes, and the company will continue supporting existing customers - a commitment worth revisiting in six months, since acquired open-source tools have a mixed track record on staying that way.

NVIDIA Garak

Garak is narrower and more research-oriented than Promptfoo. NVIDIA's AI Red Team built it as, in the project's own description, "the LLM vulnerability scanner" - the nmap-for-LLMs framing shows up constantly because it fits. You point Garak at a generator backend (it supports more than 20, including OpenAI, Hugging Face, and local models), select probe modules, and it fires adversarial payloads while detector modules score the responses for data leakage, toxicity, and policy violations.

Garak's probe library is deep: encoding-based injection (Base64, Morse, ROT13, Braille, hex), DAN-style jailbreaks, malware generation attempts, and hallucination checks, organized into more than a dozen categories with multiple sub-probes each. It has 8,782 GitHub stars and ships under Apache 2.0, with no paid tier - NVIDIA maintains it as research infrastructure, not a product.

Garak's CLI output during an encoding-injection probe run - failure rates vary sharply by encoding scheme. Source: github.com/NVIDIA/garak

The tradeoff is usability. There's no managed dashboard, no hosted scheduling, and interpreting a full run means reading HTML or JSONL reports yourself. Garak is the right tool for a team that owns the model - fine-tuning, quantizing, assessing a new base model - and wants a scanner they fully control, not a tool that's the right fit for someone who just wants a pass/fail gate in CI.

Microsoft PyRIT

PyRIT - the Python Risk Identification Toolkit - is the odd one out here because it isn't really a scanner with a fixed probe list. It's an orchestration framework. Microsoft's AI Red Team built it out of scripts they'd been using internally since 2022 and open-sourced it in February 2024, and it has 4,285 stars and 117 contributors under an MIT license.

What PyRIT does well is multi-turn and multi-modal attacks. It ships orchestrators for single-turn probing, multi-turn conversational escalation, Crescendo attacks, and Tree of Attacks with Pruning, plus a converter pipeline that can transform a prompt through Base64, leetspeak, homoglyph substitution, or cross-modal conversion to slip past filters. It can target text, image, audio, and video systems, and it can drive browser-based targets through Playwright, not just API endpoints.

Microsoft is explicit that PyRIT is "not a replacement for manual red teaming" - it augments human analysts by surfacing which areas warrant deeper investigation, and Microsoft's own writeup describes exercises that used to take weeks of manual probing completing in hours. That framing matters: PyRIT assumes a security engineer is driving it, not that it will produce a clean report for a product manager.

DeepTeam

DeepTeam, from Confident AI, is the newest and smallest tool here at 2,439 GitHub stars, but it's the one built specifically to slot into an existing LLM testing workflow. It's the sibling project to DeepEval, which we covered in our LLM eval tools comparison, and it shares the same pytest-friendly, locally-run philosophy.

DeepTeam ships more than 50 vulnerability types and 20-plus attack vectors spanning data privacy, responsible AI, and security, with built-in alignment to the OWASP LLM Top 10 and the NIST AI Risk Management Framework. It handles single-turn and multi-turn attacks and can test agents, RAG pipelines, and chatbots with persistent memory and tool use, not just single-shot prompts. Grading is delegated to whatever LLM you configure as a judge, which keeps the tool itself lightweight - Apache 2.0, runs locally, no infrastructure to stand up.

The catch is that DeepTeam's results live wherever you run them unless you pair it with the Confident AI platform, which is a separate commercial product with its own pricing. For a solo developer or small team already comfortable with pytest-style testing, that's a feature. For a security team that needs a persistent audit trail across scans, it's an extra purchase.

Lakera Red

Lakera built its red-teaming credibility on Gandalf, a public game where players try to trick an AI into revealing a password - Lakera says it has drawn more than a million players and tens of millions of attack attempts across 100-plus languages, and that data feeds directly into Lakera Red's attack library. The product tests across three risk lenses - safety, security, and responsible AI - covering context extraction, instruction override, content injection, service disruption, and indirect poisoning through RAG or external data sources.

Lakera Red integrates in a closed loop with Lakera Guard, the company's runtime detection API that we reviewed in our LLM firewalls comparison: findings from a Red scan can feed directly into Guard's blocking rules. Pricing is straightforward at the entry level - a free Community tier covers up to 10,000 API requests per month with an 8,000-token prompt cap - but enterprise pricing isn't published and requires talking to sales. Check Point's pending acquisition doesn't appear to have changed the product roadmap yet, but Lakera's Zurich office is set to become Check Point's AI security R&D hub once the deal closes, which is worth watching if you're standardizing on the platform for the long term.

Mindgard

Mindgard is the most enterprise-focused product in this lineup. It runs automated adversarial campaigns against LLMs, traditional NLP models, and multi-modal systems on a schedule rather than as a one-off scan, and it rolls out through CI/CD pipelines, Burp Suite, or a single click depending on how integrated a team wants it.

Mindgard's findings view tracks severity and risk category over time rather than presenting a single scan snapshot. Source: mindgard.ai

The feature that separates Mindgard from the open-source tools is compliance mapping: reports are structured against OWASP, NIST AI RMF, MITRE ATLAS, and EU AI Act requirements out of the box, which matters a lot if a compliance team, not just an engineering team, needs to sign off on an AI deployment. Mindgard doesn't publish pricing tiers - every engagement starts with a demo request - so budget for a sales cycle before you get a number.

Open source versus paid, honestly

Here's the part vendors won't say directly: an open-source scanner like Promptfoo, Garak, or DeepTeam will find most of the same vulnerability classes a $20,000-a-year platform will find. What you're paying Lakera or Mindgard for isn't a different kind of attack, it's someone else keeping the attack library current, a dashboard your compliance team can read without a CLI, and a report format that satisfies an auditor. If your team already has security engineers comfortable with YAML configs and Python, start with the free tools and see how far they get you before paying for a platform.

• Feb 2024 - Microsoft open-sources PyRIT, built from internal red team scripts used since 2022.

• Aug 2024 - Cisco announces intent to acquire Robust Intelligence, reported at roughly $400 million.

• Apr 2025 - Palo Alto Networks announces intent to acquire Protect AI to boost Prisma AIRS.

• Late 2025 - Check Point agrees to acquire Lakera in a deal reported at around $300 million.

• Mar 2026 - OpenAI announces it is buying Promptfoo, committing to keep it open source.

Best picks

Best overall: Promptfoo, for the combination of a real free tier, a large plugin library, and CI/CD integration that doesn't require a platform contract.

Best for deep model research: Garak, if a security or ML team wants full control over probe selection and is comfortable reading raw scan output.

Best for agentic and multi-modal attacks: PyRIT, when the target isn't a simple prompt-response API but a multi-turn agent or a system that spans text, image, and audio.

Best paid platform: Mindgard, for teams that need scheduled scanning and compliance reports mapped to specific frameworks rather than a one-time CLI run.

None of these tools replace a human security review before a high-stakes AI deployment - Microsoft says as much about its own tool. What they do is catch the vulnerability classes that are tedious and repetitive enough that no security team should be finding them by hand in 2026.

Sources

• Promptfoo GitHub repository
• Promptfoo is joining OpenAI
• OpenAI acquires Promptfoo to secure its AI agents - TechCrunch
• NVIDIA Garak GitHub repository
• Microsoft PyRIT GitHub repository
• Announcing Microsoft's open automation framework to red team generative AI systems
• DeepTeam GitHub repository
• Lakera AI red teaming
• Check Point to acquire Lakera
• Palo Alto Networks completes acquisition of Protect AI
• Cisco announces intent to acquire Robust Intelligence
• Mindgard best tools for red teaming
• OWASP Top 10 for Large Language Model Applications