Breaking

France's ZML launches free LLMD inference server for many AI chips

July 8, 2026 at 04:04 EDT

Infra & Chips
Open Source
Foundation Models

French AI startup ZML, based in Paris, has released ZML/LLMD, a free inference server designed to run open-source LLMs at high speed across a wide range of AI chips. Offered for now as an alpha technical preview, it supports not only NVIDIA but also AMD, Google TPU, Apple Metal and Intel Arc/OneAPI accelerators, TechCrunch reported.

July 8 · ZML

One Server, Every AI Chip: ZML Opens Its Cross-Silicon Inference Engine

Paris startup ZML has released LLMD — a Docker-based server that runs open-source LLMs at high speed across NVIDIA, AMD, Google TPU, Apple Metal and more, offered as a free technical preview aimed at breaking vendor lock-in.

hardware backends supported from one codebase

$20M

raised by the ~20-person team

Free

technical preview — not for production yet

One Container, Six Silicon Backends

Same image zmlai/llmd:latest — mix and match chips, avoid vendor dependence.

NVIDIA

CUDA

AMD

ROCm

Google

TPU

AWS

Trainium / Inferentia

Intel

OneAPI (Arc)

Apple

Metal

How LLMD Runs a Model

Pull weights
Hugging Face · S3 · GCS

→

Run container
single Docker image

→

Serve on any chip
BF16 · gated models

Models: Llama · Qwen 3.5/3.6 · Gemma 3/4 · LFM 2.5 · Mistral 3

The Promise

Praised for container-based deployment and cross-platform GPU support. Yann LeCun called it a hardware-independent inference engine; a hands-on report highlighted its simplicity and performance expectations.

The Caveats

No public tokens-per-second benchmarks yet, and real-world evaluation is limited right after release. Supported models are expanding in stages, and it is not open source — commercialization is planned.

An alternative to NVIDIA concentration — betting that mixing chips beats vendor lock-in in the inference "gold rush."

HERITAGE

Built on the Zig-based, Python-free open-source stack (v2, March 2026)

Tenstorrent support planned; ties with VSORA and Axelera in view

Continue reading

The rest of this article is for AI News Blitz readers. Choose an option below to keep reading.