DeepSeek V4.1 Flash now reaches 848 tokens per second per agent on Nvidia’s GB300 NVL72 system, up from 596 tokens per second roughly a month earlier, a gain of 252 tokens per second without changing the hardware. The improvement comes mainly from adjustments to inference software, kernels and the model itself. The multimodal Mixture-of-Experts model has 552 billion total parameters, with 8 billion active during prefill and 16 billion during decoding. It accepts text and images, supports a context window of one million tokens and compresses the KV cache to about 890 bytes per token. In specialized configurations, aggregate throughput reportedly exceeds one million tokens per second per GPU, at an estimated cost of $0.002 per million tokens under large-hyperscaler pricing.
Source: Read the original article
