SemiAnalysis 发推称 NVIDIA 在 2.8T 参数的 Kimi K3 模型上，通过新的 vLLM 上游优化实现 423 tok/s/user，同硬件上提速 1.68 倍。该工作由 NVIDIA CentML 的 Benjamin 完成，用 Cute DSL 编写面向 decode 的 kernel，通过激进的 Megakernel 融合实现，共 12,000 行代码。
AI 生成摘要 · 以原文为准
BREAKING NEWS🚨: NVIDIA has just achieved 423 tok/s/user 🚀 on the 2.8T Parameter Kimi K3 model via a new upstream vLLM optimization. 1.68x Faster on the Same Hardware!
This was done by NVIDIA CentML's Benjamin, who wrote NVIDIA's new decode-specialized kernels in Cute DSL via aggressive Megakernel fusion. This is a 12,000 LoC Megakernel and is the future of advanced AI-assisted kernel generation. NVIDIA is already working on DeepSeekv4.1 and other models for vLLM too.
Someone should give this NVIDIA 10x engineer a raise for this great new major feature!
来源：SemiAnalysis · x.com