What happened
French AI startup ZML has released ZML/LLMD, a new LLM inference server. According to TechCrunch, the software is designed to run open-source models at peak performance across a wide range of hardware. This includes chips from Nvidia, AMD, Google, Apple, and Intel.
The product is launching as a free, closed-source tool. ZML's stated goal is to break vendor lock-in and allow enterprises to mix and match hardware for better cost and energy efficiency. The Paris-based team of 20 has raised $20 million to support the effort.
How the room's reading it
The release is being framed as a direct challenge to hardware silos and vendor lock-in. Observers see this as another play in the "inference gold rush," where optimising the cost of running models has become more critical than training them. The goal is clear — let builders use a mix of chips from Nvidia, AMD, and others to cut costs and energy consumption.
Practitioners are noting the crowded field, with ZML competing against established players like Inferact (vLLM) and RadixArk (SGLang). However, ZML's ambition to co-design silicon and its backing from figures like Yann LeCun and Solomon Hykes suggest it's being taken seriously. The move is also seen as a potential boost for emerging European chipmakers, giving them a software layer to compete on performance.
Sailfish's take
We've seen this play before — a new software layer promises to abstract away hardware complexity. The pitch is always compelling. The reality is that performance across a truly heterogeneous stack is notoriously difficult to deliver reliably. It's a game of a thousand edge cases.
While a free tool is attractive, we're cautious about building production systems on a closed-source product with an undefined business model. ZML's plan to "measure and then generate revenue" introduces uncertainty for anyone building on top of it. We'd test ZML/LLMD on non-critical workloads to see if the performance claims hold up in a real-world, mixed-chip environment. For now, it’s one to watch, but we wouldn’t move our core inference workloads over just yet.