A French startup called ZML has released a free inference server called LLMD, designed to run open-source large language models across hardware from Nvidia, AMD, Google, Apple, and Intel, according to TechCrunch. The product targets enterprises and cloud providers who want flexibility across multiple hardware vendors rather than being locked into a single chip supplier for running AI models.

ZML was founded by Steeve Morin and has backing from Yann LeCun, who won the Turing Award and is a respected AI researcher. The company has published LLMD on its product page, and the underlying code is open-source on GitHub.

What's Actually Here

It helps to know the difference between two things: ZML the framework, and LLMD the product built from it. Think of ZML as the underlying engine—comparable to JAX or PyTorch, frameworks that data scientists use to build AI systems. LLMD, on the other hand, is the production-ready server that sits on top of that engine. It handles the practical plumbing: batching requests together, scheduling them across hardware, and managing the operational layer that lets a company actually run models at scale behind an API. That distinction matters because they solve different problems.

ZML itself is written in Zig, a systems programming language that gives developers fine-grained control over memory and produces lean, efficient code. Most inference software is built with Python paired with Nvidia's CUDA toolkit—a well-established but heavy approach. The Zig choice is both unusual and deliberate, signaling that ZML's builders wanted to optimize for speed and footprint rather than comfort for existing developers.

The hardware support expanded quickly. ZML shipped version 2 in March 2026 with support for AMD GPUs and AWS Trainium chips in addition to Nvidia and Google TPUs. LLMD pushes that envelope further, adding Apple's Metal framework (used for graphics and AI on iPhones and Macs) and Intel Arc GPUs to the mix.

Why This Matters

The competitors here are worth naming. LLMD is positioning itself against Baseten, Inferact (the company behind the widely-deployed vLLM inference engine), and RadixArk. Both vLLM and SGLang—a competing inference tool—have become the standard choice for running open-source models in production over the past two years. A free, hardware-agnostic alternative entering that space is a direct challenge to their commercial business layers.

The broader context here is significant. Morin has publicly identified eight European AI chipmakers—Axelera, Fractile, Kalray, OLIX, Q.ANT, SiPearl, SpiNNcloud, and VSORA—as potential beneficiaries of ZML's software. The idea is that a shared inference layer could let smaller or newer chip makers plug into existing model-serving pipelines without having to build an entire software stack from the ground up. For those companies, that matters because right now they struggle to compete with Nvidia's ecosystem, which is not just powerful but deeply entrenched in every data center that runs AI.

The larger strategic bet here is worth considering. For the past decade, Nvidia's CUDA software has been as much a competitive lock-in as the hardware itself—companies invest in CUDA expertise, write code for it, and switching costs rise over time. Every serious attempt to build a truly hardware-agnostic inference layer is, implicitly, a test of whether that lock-in can be broken the way portable compilers once loosened ties to specific CPU architectures. Whether ZML's Zig-based approach will actually deliver performance parity across different chip families—the kind of consistency that would make switching genuinely painless—has not yet been shown at production scale. TechCrunch's reporting covers the launch itself, not independent benchmarks under real-world loads.

The Business Question

For enterprises, the economic case is straightforward. If LLMD performs consistently across multiple chip vendors, a company gains negotiating leverage. You can route workloads to whichever chip is cheapest or most available at any given moment, potentially lowering costs and power consumption compared to being locked into a single vendor. That's a tangible benefit to a buyer's bottom line.

One thing worth flagging: a free product from a startup still needs to generate revenue somehow, and TechCrunch's report does not explain how ZML plans to make money from LLMD—whether through managed hosting, enterprise support, or something else. Other companies, including Databricks and Red Hat, have followed similar open-source-with-paid-services patterns before, so the absence of a stated business model is not unusual. But it is a gap worth watching as the product matures and the company grows.

What Happens Next

The real test for LLMD will be whether it gets adopted in production clusters where failures cost money and slowdowns matter. Benchmark charts released at launch tell one story; actual uptime and throughput in a live data center tell another.

If European chipmakers gain even modest traction because this French inference server made switching to their silicon low-friction, that would be a meaningful shift in a market where software portability has historically lagged behind the diversity of hardware available. Whether ZML can deliver that outcome is still an open question.