SGLang 合入 megakernel 支持，在 GB300 NVL72 AgentX Harness 上接近每用户 1000 tok/s，MI355X 交互性提升约 1.5 倍，且在 SGLang Docker 镜像中开箱即用。引用的 LMSYS 发布称，SGLang 低延迟模式与 NVIDIA、AMD 合作构建，在 DeepSeek-V4.1-Flash 的 AgentX trace replay 中，NVIDIA GB300 上 P90 每用户 token 速度提升最高 1.38 倍（671 到 928 tok/s），AMD MI355X 上提升 1.56 倍（332 到 516 tok/s）；NVIDIA 端通过 kernel 重叠把解码层时间从 96.8 µs 降至 68.4 µs，AMD 端每层用 2 个 persistent kernel 取代数十个小 kernel，并新增支持外部库自定义 kernel 自动回退的插件系统，GLM-5.3、K3、MiniMax M3 的低延迟 kernel 正在推进。
AI 生成摘要 · 以原文为准
ALERT🚨: SGLANG JUST MERGED MEGAKERNEL SUPPORT, GETTING CLOSE TO 1000 tok/s/user on the GB300 NVL72 AgentX Harness & increasing interactivity by 1.5x on MI355X too! Great work by the SGLang, RadixArk, NVIDIA, and AMD teams!
Best of all, this already works out of the box on SGLang Docker images! Excited to rapidly get this on the official AgentX!
https://x.com/lmsysorg/status/2112342810376712527
Introducing SGLang Low-Latency Mode, built with @NVIDIAAI and @AMD. On DeepSeek-V4.1-Flash with DSpark, it delivers up to 1.38× P90 per-user token speed on NVIDIA GB300 (671 to 928 tok/s) and 1.56× on AMD MI355X (332 to 516 tok/s) in @SemiAnalysis_'s AgentX trace replay. Highlights: - NVIDIA: overlapping kernels cut decoder layer time from 96.8 to 68.4 µs - AMD: 2 persistent kernels per layer replace dozens of small ones, with cross-GPU communication overlapped inside them. - New plugin system: custom kernels plug in from external libraries, with automatic fallback. - Low-latency kernels for GLM-5.3, K3, and MiniMax M3 are in progress and will plug in next.在 X 查看被引用的帖子
来源：SemiAnalysis · x.com