Item: Nemotron 3.5 Lightning
Author: Elena Marchetti

NVIDIA shipped Nemotron 3.5 Lightning on August 11, and the pitch is refreshingly narrow for a company that usually leads with superlatives. This 30-billion-parameter open-weight model, distilled from NVIDIA's own 550-billion-parameter Nemotron 3 Ultra, isn't trying to out-think anything. It's trying to run the boring parts of an agent loop, tool calls, file reads, retries, thousands of times a day, faster and cheaper than the model class it competes in. I spent the past several days going through NVIDIA's own materials alongside every independent test I could find, and the honest verdict sits somewhere between the marketing and the skepticism.

TL;DR

7.4/10 - a genuinely fast, cheap open model for the execution layer of agent pipelines, not a general-purpose upgrade
Independent testing confirms the speed claims; roughly 670 tokens per second, close to double Gemini 3.5 Flash-Lite's throughput
Trails Qwen 3.6-35B-A3B on raw intelligence and long-context accuracy despite a 1M-token context window
Use it as the fast worker behind a stronger planner; skip it if you need one model to handle both reasoning and execution

A Distillation, Not a Fresh Build

Lightning replaces Nemotron 3 Nano as the small, fast option in NVIDIA's Nemotron 3 family, which now spans Nano, Lightning, Super, and Ultra. Architecturally it keeps the family's signature: interleaved Mamba-2 state-space layers, Mixture-of-Experts blocks, and a handful of standard attention layers, so long sequences stay cheap while attention handles precise recall. NVIDIA lists the model as 30 billion total parameters with 3 billion active per token. The German outlet The Decoder, working from the published configuration files, puts the exact figures at 31.6 billion and 3.6 billion. It's a minor gap, but it's the kind of rounding NVIDIA does consistently across its marketing, and it's worth knowing before you take any of the round numbers at face value.

What separates Lightning from a from-scratch small model is where its weights came from. NVIDIA distilled it down from Ultra rather than pretraining a new checkpoint on a fresh data mix, the same technique covered in our guide to finetuning and distilling language models. Multi-token prediction was baked into pretraining itself, not bolted on afterward, and NVIDIA shipped two purpose-built draft models, DSpark and DFlash, with the main checkpoint specifically to accelerate speculative decoding.

What Got Trained In

NVIDIA released an open dataset, Nemotron-RL Agentic Terminal Pivot, alongside the model, and support for further tuning through NeMo Automodel and NeMo RL. That customization pipeline is not theoretical. CodeRabbit, one of NVIDIA's private-preview partners, reports that post-training Lightning for its code review routing task took under three hours of experimentation and cost less than $100.

The Benchmarks NVIDIA Wants You to See

NVIDIA's headline chart plots PinchBench, its own 10,000-task agentic benchmark, against GPU-hours to completion. Lightning lands at roughly 86% accuracy while finishing the task set in about 30% fewer H100 GPU-hours than Qwen 3.6-35B-A3B, which scores similarly. NVIDIA calls this the "efficiency frontier," and on that specific axis, the claim holds up.

The independent picture from Artificial Analysis tells a more useful, less flattering story. Lightning scores 24 on the Artificial Analysis Intelligence Index, a nine-point jump over the Nano generation it replaces, and it ties OpenAI's gpt-oss-120b (high) despite running at a fraction of the parameter count. But on the accuracy side, it sits well behind Qwen 3.6-35B-A3B (32) and Google's Gemma 4 31B (30). It also can't touch proprietary small models: Gemini 3.5 Flash-Lite scores 37 while running at comparable speed.

Chart plotting Artificial Analysis Intelligence Index against output speed for open-weight models under 40B parameters, showing Nemotron 3.5 Lightning far ahead on speed but mid-pack on intelligence Nemotron 3.5 Lightning is the clear speed leader among small open models, at roughly 670 tokens per second, but its Intelligence Index score of 24 trails Qwen 3.6-35B-A3B (32) and Gemini 3.5 Flash-Lite (37). Source: developer.nvidia.com

Where Lightning does look truly strong is on agent-specific evaluations. On GDPval-AA v2, it reaches an Elo rating of 824, ahead of gpt-oss-120b's 800 and even NVIDIA's own larger Nemotron 3 Super at 698. On Terminal-Bench v2.1, it jumps from Nano's 7% to 24.3%, nearly matching gpt-oss-120b's 26.2%. These are the numbers that back up NVIDIA's "built for the execution layer" framing better than PinchBench does.

Where It Falls Behind

The single-turn benchmarks are less forgiving. Qwen 3.6-35B-A3B beats Lightning on MMLU-Pro (85.2 versus 81.94), GPQA Diamond (86.0 versus 75.44), and SWE-bench Verified (73.4 versus 51.56). That's not a close race on any of the three. The long-context story is similarly mixed: Lightning ships with a 1M-token window on NVFP4 hardware, but on NVIDIA's own long-context accuracy measure it scores 52.0, against 61.06 for Qwen and 57.56 for Gemma 4 26B. A big context window that retrieves less reliably than smaller windows on competing models is worth knowing before you plan around it.

Kingy.ai's review, which is upfront about not running the model itself, put it plainly: Qwen 3.6-35B-A3B "is materially stronger on hard science, software engineering, terminal work and long-context accuracy." That assessment lines up with NVIDIA's own published numbers rather than contradicting them, which is a useful sanity check on a vendor benchmark suite.

Hands-On, From People Who Actually Ran It

Kilo, which builds coding agent tooling, integrated both the "Instant" and "Thinking" variants into production workflows covering UI edits, git operations, and code generation. Their write-up reports 100% success on git tagging tasks with the Instant variant and 73% exact completion on a custom git benchmark, with 16 out of 16 perfect scores on operations they classify as critical. The catch, in their words, is that Lightning has "a highly opinionated" personality for a model this size. It performs best inside a structured harness with strict guardrails, and struggles when given open-ended planning latitude.

Thoughtworks ran a broader test covering domain adaptation, behavioral tuning, and inference speed. A LoRA adapter trained for legal use won 75% of blind comparisons against the base model and pushed CaseHOLD benchmark accuracy from 35% to 77%. A healthcare adapter won 60% of comparisons and cut prediction error by 24%. General capabilities held within 1.5 points of baseline in both cases, which suggests the specialization didn't come at the cost of breaking the model elsewhere. On speed, Thoughtworks measured a 1.46x to 1.96x throughput gain from the built-in speculative decoder, cutting GPU cost per token from $0.477 to $0.250.

"This wasn't a months-long project." - Thoughtworks, on adapting Lightning for a specialized domain

The CodeRabbit Case Study, With Caveats

CodeRabbit's routing experiment is the most detailed public account of fine-tuning Lightning, and it's worth reading in full rather than trusting NVIDIA's summary of it. Starting from 39,566 examples scraped from public repositories, supervised fine-tuning alone took exact route agreement on their code-review router from 75.8% to 80.4%. Adding reinforcement learning with verifiable rewards pushed that to 80.7%, but CodeRabbit's own writeup flags that the confidence interval on the RLVR-specific gain crossed zero, meaning they treat it as a non-regression rather than a proven improvement. The clearer RLVR win was on output agreement, where Cohen's kappa rose from 0.461 to 0.544.

Bar chart comparing peak serving cost for 1,000 tasks between a GPT baseline model at $2.34 and NVIDIA Nemotron 3.5 Lightning on a Baseten A100 at $1.16 CodeRabbit's own measurement: $2.34 per 1,000 tasks on their GPT baseline versus $1.16 on a fine-tuned Lightning running on a Baseten A100, a 50.4% reduction at saturated throughput. Source: coderabbit.ai

The tuned model generated 63.4% fewer output tokens than the baseline on the same task, which is most of where the cost savings come from. CodeRabbit is also careful to note what they haven't shown yet: the evaluation ran on a frozen 1,000-request test set, "leaving sustained-load and production-traffic checks for the next phase." Treat the 50% cost figure as a credible early signal, not a settled production result.

Running It Locally

Lightning fits on a single H100 or A100 in BF16, and the NVFP4 quantized checkpoint runs leaner still across Blackwell, Hopper, and Ampere GPUs. NVIDIA lists RTX PCs, DGX Spark, DGX Station, and Jetson devices as local targets, with Ollama, LM Studio, and llama.cpp supported from day one.

A third-party benchmark from local.ai, run specifically on DGX Spark hardware, plots intelligence against end-to-end task time on a log scale. Lightning is the only first-party NVIDIA model to land on the Pareto frontier in that chart; several community quantizations of Qwen and Gemma sit near it, but nothing NVIDIA-published beats it on both axes at once for local, on-device use.

Chart plotting local.ai Intelligence Index against end-to-end task time on DGX Spark, with Nemotron 3.5 Lightning sitting on the Pareto frontier between slower, smarter models and faster, less capable ones On DGX Spark specifically, Lightning sits on the local-deployment Pareto frontier, roughly two minutes per task, trading some accuracy against the slower Step 3.7 Flash and Qwen 3.6 quantizations above it. Source: developer.nvidia.com

One practical wrinkle Kingy.ai flagged: all 30 billion weights still need to be stored even though only 3 billion activate per token, and the unquantized MLX port runs 63.2 GB. Lightning is fast to run, but it isn't small on disk or in memory the way its active-parameter count might suggest.

NeMo Switchyard: The Real Bet

NVIDIA didn't ship Lightning alone. NeMo Switchyard is an open-source routing library that sits in front of a mix of open, proprietary, and NVIDIA models and sends each request to whichever one fits, without developers rewriting their applications. It includes a tuning-free LLM classifier with session affinity, a stage router that reads recent tool activity, and an escalation router that starts cheap and promotes only on sustained difficulty.

NVIDIA's own numbers here are specific enough to be interesting: an escalation router splitting traffic between Lightning and Claude Opus 4.8 cut total cost by 74% against a frontier-only baseline, sending just 7% of calls up to the frontier model, at roughly a 6-point accuracy tradeoff. That framing tells you what NVIDIA actually believes the near future looks like. Not one model doing everything, but a router deciding, call by call, which model is worth the money.

Hacker News commenters were split on the release. One thread noted Lightning "looks behind" the equivalent Qwen model on most benchmarks, which tracks with the published numbers above. Another strand of discussion welcomed a release that wasn't chasing the largest possible parameter count, on the reasoning that better inference hardware is still a couple of years out and the industry's incentives currently push the opposite direction.

Strengths

Independently confirmed speed advantage; roughly 670 tokens per second, close to double Gemini 3.5 Flash-Lite
Distilled from Nemotron 3 Ultra, inheriting some of a 550B model's judgment at 3B active parameters
Truly easy to fine-tune: CodeRabbit's full post-training run cost under $100 and three hours
Strong agent-specific benchmarks: 824 Elo on GDPval-AA v2, ahead of gpt-oss-120b and Nemotron 3 Super
Runs on a single H100 or A100, with day-one support across vLLM, SGLang, TensorRT-LLM, Ollama, and LM Studio
OpenMDW-1.1 license permits unrestricted commercial use, and pricing on OpenRouter undercuts its own predecessor at $0.05/$0.20 per million tokens

Weaknesses

Trails Qwen 3.6-35B-A3B by 3 to 11 points on MMLU-Pro, GPQA Diamond, and SWE-bench Verified
Long-context retrieval accuracy (52.0) lags behind Qwen (61.06) and Gemma 4 26B (57.56) despite the larger context window
Strongest capability claims (CrowdStrike, Harvey, Lila Sciences) remain vendor-reported, single-customer results with no independent reproduction
Text-only; no image, screenshot, or document input
Described by hands-on testers as "highly opinionated," making it a poor fit for open-ended planning without a structured harness around it

Verdict

Nemotron 3.5 Lightning does what NVIDIA said it would do, which is rarer than it should be for a model launch this loud. It's fast, independent testing confirms the speed, and it's cheap to run and cheap to customize. CodeRabbit turned it into a working code-review router for less than $100 in an afternoon. That's a real capability, not a benchmark trick.

What it isn't is a broad upgrade over the small open models it competes with. Qwen 3.6-35B-A3B beats it on nearly every benchmark that measures reasoning rather than throughput, and the 1M-token context window is less useful than it sounds once you look at the retrieval accuracy numbers. Lightning is a specialist built for the boring, repetitive layer underneath a stronger planning model, exactly as NVIDIA describes it. Judge it on that job, not on whether it can replace the model doing the planning, and the score holds up.

Score: 7.4/10 - a legitimately fast, cheap execution-layer model with confirmed real-world gains, held back by benchmark gaps its own maker doesn't hide.

Nemotron 3.5 Lightning Review: NVIDIA Bets on Speed