#AMD

Daishin Securities CSPs and AI labs: CSPs and AI labs had requested the development of lower-stack HBM products, such as 4-high and 6-high configurations, for use in low-end inference products. Those plans now appear to have been shelved, with 8-high once again emerging as the minimum stack height. NVIDIA: Rubin’s specifications have changed amid base die supply constraints and requests from customers prioritizing bandwidth over cost efficiency. The original plan called for a mix of 8-high and 12-high HBM, but the 8-high option appears to have been dropped. For Rubin Ultra, scheduled for release in 2027, multiple architectural variants are currently undergoing tape-out. Initial versions are expected to use the highest-specification 8-high HBM4, while subsequent versions are expected to adopt either 8-high or 12-high HBM4E. AMD: AMD is expected to use a mix of 8-high and 12-high HBM4 in its 2027 products. One notable development is the upcoming MI450, which is expected to increase the number of HBM stacks within the CoWoS package to 12, delivering 288GB of HBM4. The increase from eight to 12 HBM stacks per CoWoS package is a positive development for HBM content over the medium to long term. OpenAI and Anthropic: OpenAI is expected to adopt 12-high HBM4, while Anthropic is expected to use 8-high HBM4.

Introducing SGLang Low-Latency Mode, built with @NVIDIAAI and @AMD. On DeepSeek-V4.1-Flash with DSpark, it delivers up to 1.38× P90 per-user token speed on NVIDIA GB300 (671 to 928 tok/s) and 1.56× on AMD MI355X (332 to 516 tok/s) in @SemiAnalysis_'s AgentX trace replay. Highlights: - NVIDIA: overlapping kernels cut decoder layer time from 96.8 to 68.4 µs - AMD: 2 persistent kernels per layer replace dozens of small ones, with cross-GPU communication overlapped inside them. - New plugin system: custom kernels plug in from external libraries, with automatic fallback. - Low-latency kernels for GLM-5.3, K3, and MiniMax M3 are in progress and will plug in next.

Together, AMD EPYC™ CPUs and AMD Instinct™ GPUs provide the compute foundation for next-generation enterprise AI and agentic workflows on OCI. From enterprise applications, databases, and analytics to generative AI, training, and inference, OCI supports a broad range of business and technology priorities. Combined with OCI’s distributed cloud capabilities, organizations can deploy workloads across public cloud, sovereign cloud, dedicated regions, multicloud, and edge environments. Learn more about our work with @AMD: https://social.ora.cl/6012BHyb3o

SHOCKING: Many neocloud executives we spoke with feel that if they have non-NVIDIA networking gear on their cluster, or if their cloud has an AMD GPU or TPU offering, NVIDIA will retaliate. They feel that retaliation includes not giving early allocation or no longer supporting a potential IPO/VC raise.(1/3)🧵