Chinese AI lab DeepSeek has open-sourced DeepInfer, its high-performance inference framework, under Apache-2.0 license on October 20, 2026. The toolkit includes custom kernels, distributed parallelism and speculative decoding, and powers the lab’s production-scale deployments, notably DeepSeek-V4.1-Flash. Performance figures show a throughput of 155 trillion tokens per day per cluster, a generation speed of 230 tokens per second and a 96% input cache hit rate. The framework is optimized for NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs. DeepSeek also released deepseek-recipe, a token encoding and decoding tool.
Source: Read the original article
Disclaimer: this content is for information purposes only and is not financial advice. Cryptocurrencies are highly volatile: you may lose all of your capital. Always do your own research. Legal notice
