French startup ZML has released a free inference server called LLMD, designed to run open-source large language models across chips from Nvidia, AMD, Google, Apple, and Intel, according to TechCrunch. The product targets enterprises and cloud providers looking to mix hardware vendors rather than lock into a single silicon supplier for LLM serving.

ZML is founded by Steeve Morin. Turing Award winner Yann LeCun has endorsed the company, per the same report. LLMD's product page is live at zml.ai/llmd, and the underlying open-source project remains on GitHub.

The distinction between ZML the framework and LLMD the product matters for anyone evaluating this technically. ZML itself is an ML inference framework, comparable in scope to JAX or PyTorch rather than a turnkey server, as the project's own GitHub issue tracker clarified last year GitHub. LLMD is the production server built atop that framework by the company zml.ai, handling request batching, scheduling, and the operational layer that turns a model-execution runtime into something a cloud operator can actually deploy behind an API endpoint.

ZML is written in Zig, the systems language known for explicit memory control and small runtime footprint, a choice that distinguishes it from the Python-and-CUDA stack most inference tooling is built on NYU Shanghai. ZML v2 shipped on March 24, 2026, and by that release already supported AMD GPUs and AWS Trainium alongside Nvidia and Google TPUs, according to the same source. LLMD extends that hardware span further, adding Apple Metal and Intel Arc to the list Morin cites publicly.

LLMD's stated competitive set includes Baseten, Inferact — the company behind the widely deployed vLLM inference engine — and RadixArk, which commercializes SGLang. Both vLLM and SGLang have become default choices for serving open-weight models in production over the past two years, so a free, hardware-agnostic alternative entering that space is a direct challenge to their commercial layers rather than to the open-source projects themselves.

Morin's pitch extends beyond the major GPU vendors. He named eight European AI chipmakers — Axelera, Fractile, Kalray, OLIX, Q.ANT, SiPearl, SpiNNcloud, and VSORA — as potential beneficiaries of ZML's software, framing LLMD as a portability layer that could let smaller or newer silicon vendors plug into existing model-serving pipelines without each having to build a full software stack from scratch TechCrunch.

That framing points to the real strategic question here: whether hardware abstraction at the inference layer can loosen Nvidia's grip on AI infrastructure economics the way portable compilers once loosened ties to specific CPU architectures. Nvidia's CUDA ecosystem has functioned for a decade as much as a software moat as a hardware advantage, and every serious attempt to write a truly hardware-agnostic inference layer is implicitly a bet against that moat holding. Whether ZML's Zig-based approach achieves the performance parity across chip families that would make switching costs genuinely trivial for a production team is not yet demonstrated at scale — TechCrunch's report covers the release itself, not independent benchmarks.

The economic logic for enterprises is more legible. A mixed-hardware inference layer, if performance holds across vendors, gives buyers negotiating leverage and lets them route workloads to whichever chip is cheapest or most available at a given moment, potentially reducing both cost and energy draw compared to single-vendor lock-in TechCrunch. For the named European chipmakers, several of which have struggled for developer mindshare against Nvidia's dominant tooling, a free and reasonably capable inference server compatible with their silicon could matter more than any single funding round.

It is worth flagging that a free product from a startup still needs a monetization path, and TechCrunch's report does not detail how ZML intends to generate revenue from LLMD itself — whether through managed hosting, enterprise support contracts, or something adjacent. Vendors including Databricks and Red Hat have followed similar open-core patterns with inference tooling before, so the absence of a stated business model here is not unusual, but it is a gap in the public record worth watching as the product matures.

The bigger test for LLMD will be adoption inside production clusters where downtime and throughput regressions carry real cost, not benchmark charts released alongside a launch. If European chipmakers gain even modest traction because a French inference-server startup made switching hardware low-friction, that would be a meaningful data point in a market where software portability has rarely kept pace with the diversity of silicon on offer.