🚀 RELEASES & LAUNCHES
- DeepSeek open-sources DeepInfer, its production inference framework for DeepSeek models
DeepSeek fully open-sourced DeepInfer, the in-house inference framework powering DeepSeek-V4.1-Flash at scale.- For developers and community engine builders serving DeepSeek models at scale, plus private deployments and public clouds.
- DeepInfer includes optimized kernels (FlashMLA, DeepGEMM, DeepEP), distributed inference with MoE EP + EPLB, speculative decoding (DSpark with confidence-scheduled verification), SWA bounded replay, zero-overhead async scheduling, batch-invariant/deterministic inference, and structured outputs (regex/JSON/EBNF).
- Supports DeepSeek-V4.1-Flash; runs on NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs; requires Python 3.12, CUDA 13 or CANN 9.2; Apache-2.0 licensed; built primarily on vLLM with inspiration from SGLang and TensorRT-LLM.
- Also released deepseek-recipe, the production token encoding/decoding component matching official DeepSeek API behavior.
- Gives the community DeepSeek's production stack to lower compute costs and energy use, and lets custom engines learn from its optimizations.