OpenAI has begun serving GPT-5.6 Sol, its flagship model, on Cerebras wafer-scale hardware at up to 750 tokens per second, roughly an order of magnitude faster than typical GPU inference. The move signals a shift that most AI buyers have not priced in yet: inference speed, not just intelligence or cost, is becoming a deciding factor in whether AI is actually usable.
For two years, the AI conversation has been about how smart a model is. Then, through 2026, it became about how cheap a model is to run at scale. A quieter third variable has now moved to the front: how fast the model can think out loud. GPT-5.6 Sol reached general availability on July 9, 2026 as OpenAI's most capable model, and the decision to run it on non-GPU, wafer-scale hardware is the clearest sign yet that the labs believe speed is where the next round of competition happens.
What Actually Happened
OpenAI confirmed that GPT-5.6 Sol, previewed in late June and now generally available, will run on Cerebras wafer-scale processors at up to 750 tokens per second. That number is the headline. Traditional GPU clusters serving a frontier-class model typically stream completions in the 40 to 120 tokens per second range, so wafer-scale inference is roughly an order of magnitude faster on the same weights. Cerebras chief executive Andrew Feldman has put the advantage at up to 15 times faster than GPU-based systems.
This is not a one-off demo. OpenAI and Cerebras formalized a multi-year agreement to deploy up to 750 megawatts of wafer-scale capacity dedicated specifically to low-latency inference, with rollout phased across 2026 through 2028. Sol is a premium product, priced at 5 dollars per million input tokens and 30 dollars per million output tokens at general availability, so this is a deliberate bet that some customers will pay for a flagship model delivered fast rather than a cheaper model delivered slowly.
The mechanism is worth understanding because it explains why the gap is so large. Wafer-scale chips keep an entire model on one enormous piece of silicon instead of splitting it across dozens of GPUs that must constantly pass data between them. Removing that chip-to-chip communication delay is a large part of why the same model generates tokens far faster.
Why Speed Suddenly Matters More Than It Used To
For a single chatbot reply, the difference between 100 and 750 tokens per second is barely perceptible. The answer arrives in a second either way. So why is this a strategic story rather than a benchmark curiosity?
The answer is that AI workloads have changed shape. Two shifts make latency the constraint that decides what is possible.
Agents chain many generations in sequence. A modern agent does not answer once. It plans a step, calls a tool, reads the result, re-plans, and repeats, sometimes dozens of times across a single task. Because each step waits on the one before it, total latency is additive. At 100 tokens per second a fifty-step agent workflow can take half a minute; at 750 it finishes in a few seconds. That is frequently the difference between a workflow users adopt as part of their day and one they quietly abandon, a dynamic we explored in the economics of long-running AI agents.
Reasoning models think in tokens. The frontier models that "reason" do so by generating long internal chains of thought before answering. That thinking is itself token generation, so a slow model spends real wall-clock time reasoning. Fast inference is what makes deep, multi-step reasoning practical inside an interactive experience rather than a background batch job.
Our take: Speed is not a nice-to-have layered on top of intelligence and cost. For a growing share of use cases, it is the variable that determines whether the other two even get to matter. A brilliant, affordable model that responds too slowly for a live conversation or an interactive agent simply does not get used, and the smartest answer in the world is worthless if the user has already clicked away.
The New Three-Axis Trade-Off
Buyers are used to weighing two things when they choose a model: how good it is and what it costs. The Cerebras deployment makes the case that there are now three axes, and they do not move together.
- Intelligence. How well the model handles hard, ambiguous, or novel work. Still the headline for complex reasoning and planning.
- Cost. Price per token. The variable that dominated the cheap-and-fast tier conversation as inference prices collapsed through 2026.
- Speed. Tokens per second, and the p95 latency your users actually feel. The axis this news puts in the spotlight.
The important insight is that you rarely get all three at once. Wafer-scale speed is scarce and priced at a premium. The cheapest tokens often come from slower, batched deployments. The most capable model is not always the fastest. Treating model selection as a single choice across all workloads leaves value on the table, because different parts of your product sit at different points on this triangle.
Turning that speed into a usable product is its own layer of work. Fast token generation only becomes a fast experience when the surrounding orchestration of multi-step agent workflows is engineered so the model is not stalled waiting on slow tools, databases, or retrieval calls. A 750-token-per-second model bottlenecked behind a two-second database query is still a two-second experience.
What This Means for Your Business
Most companies will not sign a wafer-scale inference contract, and they do not need to. The signal here is not "go buy exotic hardware." It is that latency deserves a seat at the table when you choose and deploy AI, right next to quality and price.
The practical move is to sort your AI workloads by interaction pattern before you sort them by model. A nightly report that summarizes ten thousand documents does not care whether inference runs at 100 or 750 tokens per second, so paying for speed there is waste. A customer-facing voice agent, a live coding assistant, or an interactive multi-step agent lives and dies on latency, and that is where fast inference earns its premium. This is the same production discipline that separates a working demo from a deployed system in voice AI and other real-time experiences, where the felt speed of the interaction is the product.
The other implication is measurement. Vendor headline numbers describe ideal conditions. What matters is the p95 latency your users experience on your real traffic, under your real concurrency, with your real retrieval and tool calls in the loop. A model that benchmarks fast can feel slow in production if everything around it is slow, and a slower model can feel fine if the interaction is designed to hide the wait.
How Businesses Should Respond
You do not need to react to this specific deal. You do need to make speed an explicit variable in how you evaluate AI. A few steps make sense now.
- Add latency to your selection criteria. When you evaluate a model for a workload, record tokens per second and p95 response time alongside quality and price. Most teams track two of the three and are surprised by the one they ignored.
- Segment workloads by interaction pattern. Separate batch and asynchronous work, where slow is fine, from interactive and agentic work, where speed drives adoption. Spend your speed budget only where the pattern demands it.
- Measure on your own traffic. Run latency tests on real production requests, not vendor demos, and include the tool calls and retrieval steps that add real-world delay.
- Watch for availability constraints. Scarce, premium capacity often ships to a limited set of customers first. Design so you can fall back to a slower tier when fast capacity is unavailable, rather than hard-wiring one lane.
Common Mistakes to Avoid
Paying for speed you cannot feel. If a workload runs in the background, faster inference changes nothing a user will notice. Match the spend to the interaction.
Optimizing the model while ignoring the pipeline. The fastest model in the world is throttled by a slow database, a slow retrieval step, or a slow external API. Latency is a whole-system property, not a model property.
Treating one headline number as the truth. Peak tokens per second under ideal load is not the latency your users get at your concurrency. Always validate against your own workload.
Key Takeaways
- OpenAI is serving its GPT-5.6 Sol flagship on Cerebras wafer-scale hardware at up to 750 tokens per second, roughly an order of magnitude faster than typical GPU inference.
- The deployment sits on a multi-year agreement for up to 750 megawatts of low-latency inference capacity, phased through 2028, signaling that speed is a long-term bet, not a stunt.
- Inference speed is now a third strategic axis alongside intelligence and cost, and the three rarely move together.
- Latency matters most for agentic and interactive workloads, where sequential steps and reasoning tokens make delay compound; it matters little for batch jobs.
- The durable response is to add latency to your selection criteria, segment workloads by interaction pattern, and measure p95 speed on your own production traffic.
Not sure where inference speed fits in your AI roadmap? Book a discovery call and we will help you figure that out, no strings attached.