DeepSeek Open-Sources DeepInfer for Ascend and NVIDIA

DeepSeek has officially open-sourced DeepInfer, a production-grade high-performance inference framework designed to drastically reduce large model deployment costs. The framework natively supports premium data center chips like NVIDIA Blackwell and Huawei Ascend 950. Revealing striking operational metrics, DeepSeek reported that a single cluster processes 155 trillion tokens daily, generating approximately $38 per GPU-hour during off-peak times with an estimated gross margin exceeding 80%.

Technically, DeepInfer is deeply customized from vLLM, integrating cutting-edge kernels such as FlashMLA, MoE EP+EPLB parallelism, and DSpark speculative decoding. Its core breakthrough lies in supporting up to 72 hours of prompt caching—far surpassing industry standards—and SWA Bounded Replay optimization, which significantly boosts throughput and determinism in long-context scenarios. The accompanying open-source component, deepseek-recipe, resolves compatibility issues between self-hosted environments and official APIs regarding prompt concatenation and tool call parsing.

This move marks DeepSeek's critical transition from a "model provider" to an "infrastructure standard setter." By disclosing engineering practices validated at scale along with revenue data, DeepInfer provides a low-cost, high-efficiency reference for enterprise private deployments. It also demonstrates the commercial viability of domestic heterogeneous computing adaptation amidst US-China AI competition, directly challenging the market landscape dominated by closed-source commercial inference services.