Tokenless, a Y Combinator-backed startup built by AI researchers from Google DeepMind, Princeton, and UC Berkeley, has launched a router product that fans out each LLM API request to a group of models, monitors their responses as they stream, selects the model clearly on track, and cancels the remaining generations so the customer pays only for the chosen output (Tokenless).

The router exposes an OpenAI-compatible and Anthropic-compatible API endpoint as a drop-in replacement for existing API calls, meaning teams can point their existing SDK calls at Tokenless without code changes. It offers at least two tiers: PRO, optimized for cost savings, and MAX, which routes each task to what Tokenless considers the best available model for maximum quality (Tokenless).

Benchmark numbers published by the company are striking in the spread they reveal. On TerminalBench 2.1, Tokenless PRO achieved 72% task completion at $0.32 per task. GPT 5.5 scored 76% at $0.74, Opus 4.8 hit 72% at $2.41, and Tokenless MAX reached 82% at $6.70 (Tokenless). In other words, PRO matched Opus 4.8's completion rate at roughly one-eighth the cost per task, and Tokenless claims PRO is 57% cheaper than the cheapest competing frontier model at comparable quality on that benchmark.

On LiveCodeBench, the picture tightens but holds. Tokenless PRO scored 89.0% completion at a total cost of $74.78, edging out GPT 5.5 (85.7% at $84.27) and Opus 4.8 (88.9% at $80.01). Tokenless MAX led the group at 90.4% and $85.33 (Tokenless).

The fan-out-and-cancel approach is not entirely new as a concept; speculative decoding and model cascading have explored adjacent territory. What Tokenless appears to operationalize is a production-grade version of parallel speculative execution at the API layer, where the "speculation" is simply: try multiple frontier models simultaneously, pick the winner, kill the rest. The cost model works because providers charge per token generated, and cancellation mid-generation limits the tokens you pay for on the discarded paths.

A cache-aware routing algorithm preserves prompt cache state when switching between models, so routing decisions do not invalidate cached prefixes (HN discussion). This matters because cache invalidation is one of the hidden tax mechanisms in multi-model setups; a naive router that picks the cheapest model per request can destroy cache hit rates and end up more expensive than sticking with a single provider. Tokenless also claims its router matches the performance of Claude Fable 5 at half the cost (HN discussion), though the TerminalBench and LiveCodeBench figures on the company's site are the more detailed performance disclosures available.

The company was founded by Rohit, alongside co-founders Andrew and Kev (HN discussion). Its landing page features an interactive savings calculator that references the Ramp AI Index for AI spend-per-employee data covering June 2025 through June 2026, letting prospective customers estimate savings against their current provider spend (Tokenless). New users receive $20 in free credit on sign-up (HN discussion).

Tokenless plans to add Kimi K3 to its router in the future and has published a technical blog post detailing how the router was built, along with a YouTube demo video (HN discussion).

The broader context for this product is the trajectory of LLM inference economics. As frontier model providers have held or raised prices while token throughput and context windows have expanded, aggregate API spend has become a material line item for any engineering organization building on these models. A router that can transparently cut per-task cost by half or more at parity quality addresses a real and growing pain point, particularly for agentic workloads where a single user-initiated task may chain dozens or hundreds of model calls.

The risk in any fan-out approach is latency: dispatching to multiple models simultaneously means the user waits for the slowest of the initial fan-out before cancellation can occur, and the monitoring overhead adds its own latency tax. Tokenless's benchmark claims are self-reported and benchmark-specific, and the degree to which TerminalBench and LiveCodeBench results generalize to production traffic patterns is an open question that only independent evaluation or production deployment can answer. What the company has demonstrated, at minimum, is that the fan-out-and-cancel model can produce cost-quality tradeoff curves that no single frontier model achieves alone.