OpenAI's Jalapeño Chip Beats Nvidia Blackwell on Independent Inference Benchmark

At the Hot Chips conference on August 25, 2026, OpenAI presented detailed technical specifications and the first benchmark results for Jalapeño, its custom inference chip codeveloped with Broadcom. The chip outperformed an Nvidia Blackwell system on SemiAnalysis's InferenceX benchmark, delivering more tokens per user and higher throughput per kilowatt than currently available state-of-the-art inference processors, according to TechCrunch.

Richard Ho, OpenAI's head of hardware, said the benchmark results show "a very significant performance advance over state of the art," allowing Jalapeño to serve more AI work per unit of power with lower latency. The comparison was made against an Nvidia Blackwell system on InferenceX, an independent, vendor-neutral benchmark from SemiAnalysis that measures fixed-sequence serving and AgentX, its long-context evaluation suite. SemiAnalysis states that InferenceX has been widely reproduced, validated, and supported by nearly every major compute buyer, including Google Cloud and Microsoft Azure. The benchmark's v2 release, published in February 2026, evaluated NVIDIA Blackwell against AMD and Hopper GPUs.

Jalapeño's architecture is built around minimizing latency in the phases of inference that OpenAI identifies as the most common bottlenecks: prefill and communication. Prefill is the initial phase of generating an AI response, where the model processes the input prompt before producing any output. In a blog post, OpenAI said the chip minimizes data movement and communication delays by explicitly placing and keeping model state — including the KV cache, a temporary store of intermediate results that the model reuses across tokens — local, while activating the right combination of compute, memory, and networking for each inference phase. The chip is a reticle-sized ASIC (application-specific integrated circuit), meaning it is designed to be as large as a single manufacturing pass allows, maximizing the amount of logic on one piece of silicon. It was built in an ultra-fast nine-month development cycle, as Tom's Hardware reported.

The chip was first announced in October 2025 and developed in close collaboration with Broadcom, with OpenAI's own models assisting in the development process. OpenAI's newsroom page, published in June 2026, introduced Jalapeño as a custom AI chip built for LLM inference — the process of running a trained language model to generate responses — designed to improve performance, efficiency, and scale across AI systems. At that time, OpenAI had received first silicon samples and was testing how the chip handled AI workloads. The company said early testing showed performance per watt substantially better than current alternatives, though final performance measurements were still underway.

Those measurements have now arrived, at least in preliminary form. The Hot Chips presentation marks the transition from OpenAI's June claims of promising early results to independently structured benchmark figures on a recognized industry framework.

Ho estimated that Jalapeño will deploy in very small volumes at the end of 2026, with more significant deployment coming in 2027. OpenAI also said it plans to make Jalapeño a multigenerational platform, allowing AI products, models, chips, and memory to be developed in concert rather than in sequence.

The architectural approach here is worth examining. Keeping the KV cache local and coordinating compute, memory, and networking per inference phase addresses a real and well-documented bottleneck in LLM serving. Prefill latency, in particular, has been a stubborn problem as context windows have grown; a chip designed from the ground up to minimize data movement during that phase could meaningfully shift the inference cost curve. The reticle-size design pushes the limits of what a single die can hold, trading manufacturing difficulty for density and fewer interconnects between separate chips.

The nine-month development cycle is notably compressed for a chip of this complexity. Traditional ASIC development from specification to first silicon typically runs 18 to 24 months. OpenAI's use of its own models in the development process, while described in general terms, points to an emerging pattern where AI-assisted design tools compress the human bottleneck in physical layout and verification.

The broader question for the industry is whether vertically integrated inference silicon changes the economics of AI deployment enough to alter the competitive landscape. Nvidia's Blackwell platform remains the default for most operators, with a mature software stack and broad ecosystem support. OpenAI's decision to benchmark against Blackwell rather than a weaker comparison point signals confidence in the results, though the company is also the sole operator of Jalapeño, which means real-world deployment data will come only from OpenAI's own infrastructure.

For now, the benchmark numbers stand on their own. Whether Jalapeño's advantages hold at production scale, across diverse workloads and model architectures, will become clearer as the chip moves from limited deployment into broader use in 2027.