Subquadratic models such as state space models and the newly launched SubQ process text closer to linear time instead of the quadratic time transformers require. For businesses, that means inference costs grow far more slowly as documents and conversations get longer, which can make long-context workloads meaningfully cheaper to run at scale.
For most of the past decade, the answer to "which architecture powers your AI" was a single word: transformer. May 2026 complicated that answer. While the frontier model race paused for breath, alternative architectures stepped into the spotlight, and they carry direct implications for what your AI workloads cost to run.
What Actually Changed in May 2026
The headline trend was not a bigger model. It was a quieter structural shift. One industry roundup framed the month as a moment when the frontier took a breath and architecture took the stage. The clearest signal was a startup betting its whole existence on moving past the transformer.
On May 5, a Miami company called Subquadratic launched with 29 million dollars in seed funding and a single claim: its model, SubQ, is not a transformer. The company says SubQ uses a mechanism it calls Subquadratic Sparse Attention, which scales closer to linearly with context length, and that the model ships with a 12 million token context window. According to the company, SubQ runs roughly 52 times faster than standard FlashAttention at one million tokens and costs about a fifth of what frontier models charge for comparable long-context workloads.
Those are vendor claims, and they deserve a skeptic's eye. When Subquadratic also asserted a roughly 1,000 times reduction in attention compute, independent researchers publicly demanded independent proof before accepting the figure. The healthy posture is to treat headline efficiency numbers as hypotheses to benchmark, not facts to budget around.
The trend is not limited to one startup. The state space model line of research kept advancing too. The Mamba-3 work presented at ICLR 2026 reports roughly seven times faster long-sequence inference than transformers while edging out transformer baselines on language-modeling benchmarks, and it keeps memory roughly constant per step rather than growing a KV cache. None of this means transformers are obsolete. It means the menu of architectures is widening, and the differences between items on that menu now show up on your bill.
Why the Math Matters for Your Budget
The transformer's core mechanism, self-attention, compares every token in your input to every other token. That comparison scales quadratically: the well-known O(n squared) behavior. Double the length of a document and the attention cost roughly quadruples. Quadruple it and the cost grows by sixteen times. For short prompts this is invisible. For a 200-page contract, a quarter of transcripts, or a long-running agent conversation, it becomes the dominant line item.
The numbers are stark at long context. One 2026 analysis of inference economics found raw hardware cost rising from about 0.34 dollars per million output tokens at 4,000 tokens of context to roughly 19.84 dollars per million at 128,000 tokens on the same model. The same report noted that a 70-billion-parameter model on a single high-end GPU could serve dozens of concurrent users at short context but only a handful as context stretched toward its limit. Long context is not a little more expensive. It is dramatically more expensive, and the quadratic curve is why.
Subquadratic architectures attack exactly this curve. State space models and sparse-attention designs process sequences closer to linear time. Double the input and you move toward doubling the cost rather than quadrupling it. The longer your inputs, the larger the gap between the two curves becomes.
This is not a marginal efficiency tweak. It changes which use cases are economically viable. A workflow that processes thousands of long documents per day can sit comfortably inside budget on a near-linear model while being prohibitively expensive on a quadratic one. The architecture decision and the unit-economics decision are the same decision.
Where This Lands for Real Businesses
Long-context workloads are the clearest win. Document review, legal and compliance analysis, call and meeting transcript summarization, and multi-hour agent sessions all involve long inputs. These are precisely the cases where quadratic scaling hurts most and where a subquadratic model's advantage is largest. If your most expensive AI line item involves long inputs, this is where to look first.
Latency improves alongside cost. Near-linear processing does not just lower the price of long inputs; it usually returns answers faster on them too. For customer-facing or real-time workflows, that can be the difference between a usable feature and one people abandon.
The trade-off is real and worth naming. Subquadratic models compress or replace the full attention mechanism, and that compression can cost some precision on tasks that require pinpoint recall of a specific detail buried deep in a long input. Industry analysis in 2026 increasingly points to hybrid designs, which blend attention and state space layers, as the pragmatic middle ground. The honest framing is not "better" but "different cost and quality profile." Choosing well means matching the architecture to the workload, which is the same discipline we describe in our guide to choosing the right AI model for your business. The companies that get the most from this shift treat architecture selection as part of a deliberate AI strategy rather than a default, benchmarking candidates against their own data instead of leaderboard averages.
How This Fits the Broader Cost Story
The architecture shift does not arrive in isolation. It compounds a price trend already underway. Aggressive provider discounting, the dynamic we covered in what the DeepSeek effect means for your AI budget, has already pushed the cost of a token down sharply. Subquadratic architectures pull a second lever: they reduce the number of expensive operations required per token of context in the first place.
Stacked together, these forces mean the cost ceiling for long-context AI is dropping from two directions at once. That should change how you scope projects. Use cases you shelved a year ago because the inference math did not close may now pencil out. This is a good moment to revisit the build versus buy decisions you made when assumptions about cost per long document were less favorable.
Our take: Most businesses should not rip out their transformer-based stack. The tooling, fine-tuning ecosystem, and operational knowledge around transformers remain far deeper, and that maturity has real value. The smart move is selective. Identify the one or two workloads where you process the longest inputs at the highest volume, benchmark a subquadratic or hybrid option there, and let measured cost and quality, not architectural fashion or vendor claims, drive the call.
How to Respond Without Overreacting
- Inventory your long-context spend. Find the workloads where inputs are longest and volume is highest. That is where quadratic scaling is quietly inflating your bill and where a switch would pay off most.
- Benchmark on your own data. Public benchmarks and vendor numbers rarely reflect your documents or your quality bar. Run a candidate model against a representative slice of your real workload and compare cost, latency, and accuracy directly.
- Watch the maturity gap. Transformer tooling for fine-tuning, evaluation, and serving is more established. Factor the integration and operational cost of a newer architecture into the comparison, not just the headline inference price.
- Keep architecture decisions reversible. Abstract the model behind a clean interface so you can swap architectures per workload as the landscape evolves. The pace of change in May 2026 is a reminder that today's best choice may not be next quarter's.
Common Mistakes to Avoid
The first mistake is treating this as a binary migration. It is not transformer versus subquadratic across your whole stack; it is the right architecture per workload. The second is trusting benchmark headlines and vendor efficiency claims instead of your own numbers, since the cost advantage of subquadratic models is concentrated in long-context cases and can be modest or absent in short-prompt ones. The third is ignoring the trade-off entirely and assuming near-linear scaling is free of quality cost. Test before you trust.
Key Takeaways
- Subquadratic models such as state space models and SubQ scale closer to linearly with input length, while transformers scale quadratically, so the cost gap widens as inputs get longer.
- May 2026 marked a visible shift toward architecture, headlined by Subquadratic's 29 million dollar launch and continued state space model progress, though several efficiency claims still await independent verification.
- The clearest business wins are long-context, high-volume workloads like document and transcript analysis, where quadratic scaling hurts most.
- Subquadratic models trade some pinpoint recall for efficiency, so the right move is selective adoption based on benchmarks against your own data, not a wholesale migration.
Navigating the shift to new AI architectures does not have to be a solo effort. Book a free discovery call and let's map out what subquadratic models could mean for your inference costs and your roadmap.