Microsoft's Maia 300 Aims to Cut the Cost of Running AI

Microsoft announced Maia 300, its third-generation in-house Maia AI chip, and will start rolling it out to production in 2027. The company designed the chip to serve as the primary AI inference system for Microsoft products and services, according to details reported The Verge. Inference here means running a trained model to respond to users, not training the model in the first place.

Microsoft expects Maia 300 to deliver 40 percent more tokens per megawatt than leading GPU-based systems. Tokens are the small chunks of text the model reads and writes, so the figure measures useful work per unit of power, like fuel efficiency for data centers. In a full 96-accelerator configuration, the company positions the system at up to 7x the rack-scale token throughput of Maia 200. Microsoft plans to scale the Azure Maia family to multi-gigawatt deployments starting in 2027.

Maia 300 is built on TSMC's N2P process node, the manufacturing process that sets transistor size and density, and uses a chiplet architecture that separates compute, memory controller and I/O functions. Separate chiplets let each function be built and scaled more independently than in a single large die. Microsoft targets the design at long-context reasoning, latency-sensitive conversational experiences and sustained multi-step agentic tasks. In plain terms, that covers handling very large inputs, keeping chat responses quick, and running tasks that take many model steps in a row.

Maia 300 succeeds Maia 200, which is running production AI workloads for Microsoft Copilot, GitHub Copilot, Microsoft Foundry and Microsoft's own AI models. Maia 200 is an AI accelerator built for inference on TSMC's 3nm process, with native FP8/FP4 tensor cores, as described Microsoft. FP8 and FP4 are lower-precision number formats that reduce memory and compute cost per operation.

The part delivers over 10 petaFLOPS at FP4 and more than 5 petaFLOPS at FP8 Microsoft EMEA. A petaFLOP equals one quadrillion operations per second, so the numbers describe raw calculation rate at those precisions.

That lineage traces to Maia 100, the first generation in the Azure Maia series. Maia 100 integrates 105 billion transistors and was introduced as a custom AI accelerator optimized for AI workloads Microsoft Azure. It is built on 5nm process technology with advanced packaging technology from TSMC.

The October announcement followed August reporting that Microsoft planned to unveil Maia 300 in fall 2026, as early as September, and was negotiating with TSMC for more than 300,000 Maia 300 units by 2027 Yahoo Finance. The Information's reporting on a fall unveil was also carried by Reuters in August Reuters.

The broader context here is what hyperscale silicon is now asked to optimize. Maia 300 is framed around inference economics and serving behavior. Tokens per megawatt is a datacenter operations metric, not a peak FLOPS number. Rack-scale throughput in a 96-accelerator configuration points to the same conclusion. Inference is the volume business. We have seen this pattern before, when cloud providers moved from buying general servers to designing their own hardware once usage grew large enough.

In my view, the workload list is the tell. Long-context reasoning stresses memory capacity and bandwidth. Conversational experiences punish inference latency and tail latency, the delays users notice most. Agentic tasks sustain both over many sequential model calls. A chiplet partition of compute, memory controller and I/O gives Microsoft independent levers on each constraint without respinning monolithic logic. Worth flagging, the 7x rack claim against Maia 200 and the 40 percent efficiency claim against GPU-based systems are system-level comparisons, not isolated die comparisons. They depend on interconnect, power delivery and software as much as transistor density.

For engineers building on Azure, the practical change will be less about N2P or chiplets in isolation and more about what cheaper, lower-latency inference at scale makes feasible to run in production. If inference demand continues to track Copilot, Foundry and first-party model usage, controlling performance per watt and throughput per rack directly controls cost per token and capacity headroom. Custom silicon of this kind tends to start as a hedge against supply and cost, then becomes the platform the software assumes. That is the optimistic arc to watch, because it widens what developers can afford to keep running.