Breaking

DeepSeek Open-Sources DeepInfer, Its Production Inference Engine, to Help Businesses Cut AI Deployment Costs

October 20, 2026 at 11:10 EDT

Open Source
Infra & Chips
Sales & Business

DeepSeek has released DeepInfer, the inference software infrastructure it uses to serve its AI models at production scale, as an open-source project under the Apache 2.0 license. The move aims to enable enterprises and developers to run DeepSeek models more efficiently on their own hardware, significantly reducing operational expenses.

October 20, 2026 · DeepSeek

DeepSeek Unlocks Production Inference Engine

Open-source DeepInfer software powers efficient AI model serving at scale, enabling enterprises to replicate production-grade efficiency on compatible hardware.

Peak Performance on Hardware

89,000

Peak prefill tokens/sec per GPU

325

Tokens/sec per user at baseline load

Supported Infrastructure

GPU: NVIDIA SM100

NPU: Huawei Ascend 950

Minimum deployment: 64 prefill GPUs + 64 decode GPUs. Requires Linux (GCC 13+), Python 3.12, CUDA 13 or CANN 9.2.

✓

Model Support & Licensing

Initial release: DeepSeek-V4.1-Flash. Open-source under Apache 2.0 license. Model weights available under MIT license. Future models will be supported in the same repository.

Key Optimization Techniques

Speculative decoding: Multi-token drafting with single-pass validation. Custom kernels: FlashMLA (attention), DeepGEMM (matrix ops), DeepEP (expert routing). Batch-invariant inference: Consistent output across load variations. Structured generation: JSON and regex-constrained outputs.

Continue reading

The rest of this article is for AI News Blitz readers. Choose an option below to keep reading.