NVIDIA's Nemotron 3.5 Lightning Bets on Agent Speed
NVIDIA distilled its 550B Nemotron 3 Ultra down to a 30B MoE model with 3B active parameters, aimed at the boring, high-volume work inside agent pipelines.
5 min read
NVIDIA shipped a new open-weight model on August 11 that isn't trying to win a leaderboard. It's trying to win a stopwatch.
Nemotron 3.5 Lightning is a 30B parameter mixture-of-experts model with only 3B active parameters per token, distilled down from NVIDIA's own Nemotron 3 Ultra 550B-A55B. The pitch is narrow and specific: this isn't a model meant to plan an agent's next move. It is meant to be the thing that actually does the move, over and over, thousands of times a day, without burning a frontier model's inference bill to do it.
Key Specs
| Spec | Value |
|---|---|
| Total / active parameters | 30B / 3B (MoE) |
| Architecture | Hybrid Mamba-2 + MoE + Attention layers |
| Context window | 1M tokens |
| Distilled from | Nemotron 3 Ultra 550B-A55B |
| Throughput vs class | Up to 4x faster output |
| Task completion | 30% faster than comparable models |
| License | OpenMDW-1.1 (open weights) |
| Checkpoints | BF16, NVFP4 |
A Distillation Play, Not a Fresh Build
NVIDIA already has a full Nemotron 3 lineup: a 4B nano model for edge devices, a 120B agentic model, and the 550B Ultra flagship. Lightning slots underneath all of them, and it didn't start from scratch. NVIDIA took Ultra's capability and compressed it down through distillation, the same technique covered in our guide to finetuning and distilling language models, rather than training a new model on a fresh data mix.
The Architecture Underneath
The hybrid design interleaves Mamba-2 state-space layers with MoE blocks and a handful of standard attention layers. Only 3B of the 30B total parameters activate on any given token, which is why NVIDIA can claim compute costs close to a dense 3B model while keeping the capacity of something six times larger. Speculative decoding with multi-token prediction is baked into pretraining itself rather than bolted on afterward, and NVIDIA shipped two purpose-built draft models, DSpark and DFlash, alongside the main checkpoint to accelerate that decoding further.
What Got Trained In
NVIDIA also released an open dataset called Nemotron-RL Agentic Terminal Pivot with the model, and the weights come with support for further tuning through NeMo Automodel (LoRA or full supervised fine-tuning) and NeMo RL for reinforcement learning. That matters more than it sounds: NVIDIA's own case study with the code review company CodeRabbit reports that supervised fine-tuning plus RL took exact route agreement on a routing task from 75.8% to 80.4%, while cutting output tokens by 63.4% at roughly half the cost of the baseline API calls CodeRabbit was using before.
How It Stacks Up
NVIDIA's benchmark of choice is PinchBench, a 10,000-task agentic benchmark spanning coding, research, and file management that the company uses to plot accuracy against completion time.
| Model | PinchBench Accuracy | Relative Completion Time |
|---|---|---|
| Nemotron 3.5 Lightning | ~86% | Fastest in class |
| Qwen 3.6-35B-A3B | Comparable accuracy | ~30% slower |
| Google Gemma 4 26B | Lower accuracy | Slower still |
"Nemotron 3.5 Lightning is an open 30B mixture-of-experts model with 3B active parameters built for that execution layer of always-on agents."
On the Artificial Analysis Intelligence Index, NVIDIA says the model sits on the accuracy-versus-speed Pareto frontier for small open models, meaning nothing else in its weight class currently beats it on both axes at once. It's a narrower claim than "best open model," and NVIDIA is careful to frame it that way. See our agentic AI benchmarks leaderboard for how these numbers compare against other agent-focused evaluations.
The Other Half of the Announcement: Switchyard
NVIDIA didn't release Lightning in isolation. It shipped with an open-source routing library called NeMo Switchyard, which sits in front of a mix of open, proprietary, and NVIDIA models and sends each request to whichever one fits the task, without developers rewriting their applications. NVIDIA's internal benchmarks claim a "system of models" setup, where a frontier model plans and Lightning executes the repetitive parts, gets frontier-level accuracy while cutting task completion cost to roughly a third of what running a frontier model like Opus 4.8 alone would cost.
That framing tells you what NVIDIA actually thinks the future looks like: not one model doing everything, but a router deciding, task by task, which model is cheap enough to bother with.
Where You Can Actually Run It
Lightning is built to run anywhere from a laptop to a data center. NVIDIA lists NVIDIA RTX PCs, DGX Spark, DGX Station, and Jetson devices as local targets, with the usual cloud and data center paths. Weights are on Hugging Face and ModelScope under the permissive OpenMDW-1.1 license, with day-zero support across vLLM, SGLang, TensorRT-LLM, llama.cpp, Ollama, LM Studio, and Unsloth. It's also live as an NVIDIA NIM microservice on build.nvidia.com and testable through OpenRouter, and inference providers including Baseten, DeepInfra, FriendliAI, and GMI Cloud turned on day-0 hosting.
NVIDIA's DGX Spark is one of the local targets NVIDIA lists for running Nemotron 3.5 Lightning outside the data center.
Source: nvidianews.nvidia.com
NVIDIA's own PinchBench chart plots Nemotron 3.5 Lightning ahead of Qwen3.6-35B and Gemma 4 26B on the accuracy-versus-speed frontier.
Source: developer.nvidia.com
What To Watch
- The benchmarks are NVIDIA's own. PinchBench and the CodeRabbit case study both come from NVIDIA or its direct partners. Independent third-party numbers haven't landed yet.
- "Agentic" is doing a lot of work in the marketing. Lightning is explicitly positioned as an execution layer, not a planner. Teams still need a stronger model upstream to decide what Lightning should do.
- OpenMDW-1.1 isn't as widely recognized as Apache 2.0 or MIT. It is permissive, but enterprises with strict license review processes will want to read the actual terms before deploying it in production.
- The Nemotron family is getting crowded. Nano, Super, Ultra, Cascade, and now Lightning all target different points on the size-versus-speed curve, which makes picking the right one its own small research project.
Whether Switchyard actually gets adopted will matter more than Lightning's benchmark scores. A fast, cheap execution model is only useful if something upstream is reliably deciding when to call it instead of the expensive one.
Sources:
- NVIDIA Nemotron 3.5 Lightning Delivers Fast, Accurate Specialized Task Execution for Long-Running Agents - NVIDIA Technical Blog
- NVIDIA Nemotron 3.5 Lightning and NeMo Switchyard Deliver Faster, Smarter, More Efficient Agentic AI - NVIDIA Blog
- Introducing NVIDIA Nemotron 3.5 Lightning - Baseten
- nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 - Hugging Face
- NVIDIA DGX Spark Arrives for World's AI Developers - NVIDIA Newsroom
