Cerebras Pushes GPT-5.6 Sol to 750 Tokens Per Second

OpenAI's GPT-5.6 Sol went live on Cerebras wafer-scale hardware this week at up to 750 tokens per second. That's not a benchmark number from a controlled demo. It's the production rate available right now to select users through the Cerebras API - and it's roughly ten times faster than any GPU-based deployment of a frontier model in production today.

To put that in terms developers actually feel: a 3,000-token agent turn that takes 43 seconds on a standard H100 cluster completes in about 4 seconds on Cerebras. A five-turn conversational loop that clocks in at over a minute drops to under seven seconds. Those aren't gradual improvements - they change what's architecturally feasible.

TL;DR

GPT-5.6 Sol runs at 750 tokens/second on Cerebras WSE-3 wafer-scale chips, roughly 10x faster than H100 GPU clusters (~70 tok/s)
A 3,000-token agent turn drops from 43 seconds to 4 seconds; a 5-turn loop from 70 seconds to 6.5 seconds
API is OpenAI-compatible - swap one base_url and pass a Cerebras API key, no other code changes needed
Initial access is limited to roughly 20 organizations; broader availability expected July 10-17
Speed matters for user-facing, latency-sensitive workloads - batch jobs and overnight pipelines see no benefit

The Speed Gap

Here is what the numbers look like for typical agent scenarios:

Scenario	H100 cluster (~70 tok/s)	Cerebras WSE-3 (750 tok/s)
1,000-token turn	14 seconds	1.3 seconds
3,000-token turn	43 seconds	4 seconds
5-turn conversation loop	70 seconds	6.5 seconds
4,000-token pull request review	~57 seconds	~5 seconds

The H100 figure here is the priority tier. Standard GPU inference at frontier scale is typically 20-35 tokens per second - the gap widens further if you aren't on a premium tier.

The Cerebras WSE-3 wafer-scale chip compared to conventional GPU silicon The Cerebras WSE-3 is a single chip that occupies nearly an entire 300mm silicon wafer. Source: servethehome.com

Inside the WSE-3

The speed advantage comes from hardware architecture, not software optimization. Understanding why Cerebras is faster also explains where it isn't - and where its current constraints will bite you.

The Wafer Approach

The WSE-3 is 46,225 mm squared - roughly 21.5 centimeters on a side. For comparison, a NVIDIA H100 is about 814 mm squared. The Cerebras chip is 57 times larger, holding 4 trillion transistors and 900,000 AI-optimized compute cores. An entire frontier-class model - layers, weights, activations - fits on a single substrate rather than being sliced across a cluster of discrete chips. OpenAI's deployment of Sol spans 70 to 100 of these wafer-scale chips, with roughly one model layer per wafer.

Memory Bandwidth Is the Real Story

GPU-based inference is bottlenecked by memory bandwidth. A H100 moves data between its HBM3e memory and compute cores at about 3.35 terabytes per second. The WSE-3 provides 21 petabytes per second of on-chip SRAM bandwidth - that's 6,000 times higher. Every compute core on the WSE-3 can reach 44 GB of on-chip SRAM without any DRAM latency, without HBM bandwidth limits, without waiting for data to arrive from another chip.

When you run inference on a GPU cluster, data has to be fetched from HBM on every layer forward pass. At scale, the memory system - not the compute - is what limits throughput. The WSE-3 eliminates that bottleneck by making the memory part of the chip itself.

No Inter-Chip Hops

On a multi-GPU setup, weight tensors are sharded across cards and reassembled via NVLink or InfiniBand at each layer. That inter-chip coordination adds latency that compounds through a deep model. On Cerebras hardware, each wafer holds contiguous layers. A token passes from wafer to wafer in order without shuffle-and-gather operations. The communication overhead that controls GPU cluster inference largely disappears.

Using Sol on Cerebras

The API is fully OpenAI-compatible. If you already call Sol through the OpenAI SDK, switching to the Cerebras-hosted fast tier requires changing two lines:

from openai import OpenAI
import os

client = OpenAI(
    api_key=os.environ["CEREBRAS_API_KEY"],
    base_url="https://api.cerebras.ai/v1"
)

response = client.chat.completions.create(
    model="gpt-5.6-sol",
    messages=[{"role": "user", "content": "Your prompt here"}],
    stream=True
)

for chunk in response:
    print(chunk.choices[0].delta.content or "", end="", flush=True)

Streaming behaves identically to the standard OpenAI endpoint. Existing agent frameworks - LangChain, AutoGen, custom orchestrators built on the OpenAI SDK - will work without modification.

WSE-3 vs NVIDIA H100 specification comparison WSE-3 vs NVIDIA H100 across key AI accelerator dimensions. On-chip SRAM bandwidth is the column that explains most of the throughput difference. Source: servethehome.com

What 750 Tokens Per Second Actually Changes

The throughput improvement is real. Whether it's meaningful for your workload depends on where latency lives in your architecture.

Agentic Pipelines

Multi-step agent workflows - plan, decompose, execute, verify, summarize - run those phases sequentially. Each phase creates hundreds to thousands of tokens. At GPU speeds, a five-step loop with 600 tokens per step takes roughly 43 seconds. On Cerebras, that same loop completes in about 4 seconds. Workflows that previously had to be async and batch-friendly can become synchronous and interactive. A coding agent producing a pull request in under six seconds is a qualitatively different product than one that takes a minute.

Real-Time Voice and Chat

Voice agents are the clearest beneficiary. Conversational turn latency has to stay under roughly 500 milliseconds for speech to feel natural. Standard GPU inference can't reliably hit that threshold for frontier-class models at any price. Cerebras can. Streaming the first 50 tokens takes about 66 milliseconds at 750 tok/s versus over 700 milliseconds on a priority H100 tier.

Requirements and Availability

Parameter	Details
Model	GPT-5.6 Sol
Throughput	Up to 750 tokens/second
Context window	Standard Sol context (same as API)
Pricing (Sol tier)	$5/$30 per million input/output tokens
Speed premium	Not yet announced for fast tier
API compatibility	OpenAI SDK - swap `base_url` only
Access	~20 orgs initially; broader rollout July 10-17
Contract	OpenAI-Cerebras $20B, 750MW compute, through 2028

OpenAI signed a $20 billion multi-year contract with Cerebras covering 750 megawatts of inference capacity - with expansion provisions reaching 2 gigawatts by 2030. This is the largest high-speed AI inference deployment in the world by a wide margin.

A coding agent that produces a 4,000-token pull request in five seconds is a fundamentally different product than one that produces it in a minute.

Where It Falls Short

The throughput numbers are genuine. The access situation isn't. Right now, roughly 20 organizations have access under initial vetting requirements. General API availability is expected this week (July 10-17), but capacity is finite and early demand will likely outstrip it.

The speed advantage is also strictly for user-facing, latency-sensitive workloads. If you are running overnight batch jobs, fine-tuning pipelines, or high-volume async document processing, Cerebras adds cost without adding value - you're paying for tokens per second when your workload doesn't need them. Background tasks that don't touch a user should stay on standard GPU inference.

The 44 GB of on-chip SRAM, while massive compared to a H100, still creates constraints for very long-context inference. Cerebras uses a MemoryX external memory system for models that overflow SRAM, which changes the latency profile for very large context windows. Expect the 750 tok/s headline number to degrade somewhat at multi-hundred-thousand-token contexts.

Finally, Cerebras' Q1 2026 earnings showed margin pressure from rapid scale-out - the company is spending heavily to build the capacity behind this OpenAI deal. The fast tier will carry a pricing premium. That premium has not been published, but it'll matter for workloads running millions of tokens per day.

The infrastructure story is sound. The bottleneck for most developers right now isn't chip architecture - it's getting access to the capacity that already exists.

The Cerebras deployment of GPT-5.6 Sol shows what wafer-scale inference can do when deployed at production scale. For real-time agent workflows and voice applications, 750 tokens per second is a meaningful architectural unlock. The question for the next three months is whether Cerebras can expand general access fast enough to turn this into a developer platform rather than a quota-limited premium tier. The Cerebras IPO gave them the capital. Now the build-out has to keep pace with demand.

Sources: