#GPU

Nathan Lambert of @trillium_labs says Chinese AI researchers' jaws drop to the floor when he tells them how much compute OpenAI has: "The Chinese labs are very technically excellent. They are more compute-starved." "Without a doubt, they have a comparable talent pool to draw on... To say that China is anything but equal on talent, you're gonna have a hard time because they just have so many people being trained at such a high level." "What they don't have is compute." "You talk to people there, and you're like, 'Oh, I heard an average OpenAI researcher has five thousand H100 equivalents that they can access at any time,' and like their jaw just falls to the floor. They're just like, 'I can't comprehend.'" @natolambert @lishali88

Nathan Lambert, former post-training co-lead at the Allen Institute for AI, now launching Trillium, a nonprofit open AI lab, joins a16z's Lisha Li to discuss whether open models can really keep up with the frontier. When he tells researchers in China that an average OpenAI researcher reportedly has five thousand H100s to work with, "their jaw just falls to the floor." In his view, Chinese labs have the talent but are mostly short on compute. He also thinks the distillation story is overblown. Based on the research, he puts its edge at a few months, and says "there's a bit of cope" in thinking only American labs can build good models. A lot of frontier progress outside math and code comes down to labs buying 1) data and 2) RL environments. It's a multibillion-dollar market, and some of it is sloppy enough to teach models to reward hack. That's a big part of why Trillium exists. Nathan wants to give away the environments other people would sell, and show exactly how his team uses AI to build AI, including where the models still get stuck. His bet is that the more people can see how this works, the better prepared everyone is for what's coming. 0:55 Can open models keep up with the frontier? 2:05 What it takes to reach the frontier 3:55 Open weights and the fast-follower playbook 8:25 Chinese labs: equal talent, less compute 9:55 Why distillation is harder than it looks 14:55 Is distillation theft or competition? 16:20 The lab flywheel vs owning your weights 21:45 Can general models beat every specialist? 26:30 AI at 1% of today's cost in 4 years 27:15 The RL env boom and reward hacking 31:30 What Trillium will build in the open 35:30 Breaking into AI in the age of AI slop 38:05 Open-sourcing how AI builds AI 44:25 Why Nathan isn't pre-training 49:45 Agent swarms and why lab secrets leak 54:25 Fast takeoff vs risks already here YouTube: https://youtu.be/BKuK0M6oDTI @natolambert @trillium_labs @lishali88

Nathan Lambert, former post-training co-lead at the Allen Institute for AI, now launching Trillium, a nonprofit open AI lab, joins a16z's Lisha Li to discuss whether open models can really keep up with the frontier. When he tells researchers in China that an average OpenAI researcher reportedly has five thousand H100s to work with, "their jaw just falls to the floor." In his view, Chinese labs have the talent but are mostly short on compute. He also thinks the distillation story is overblown. Based on the research, he puts its edge at a few months, and says "there's a bit of cope" in thinking only American labs can build good models. A lot of frontier progress outside math and code comes down to labs buying 1) data and 2) RL environments. It's a multibillion-dollar market, and some of it is sloppy enough to teach models to reward hack. That's a big part of why Trillium exists. Nathan wants to give away the environments other people would sell, and show exactly how his team uses AI to build AI, including where the models still get stuck. His bet is that the more people can see how this works, the better prepared everyone is for what's coming. 0:55 Can open models keep up with the frontier? 2:05 What it takes to reach the frontier 3:55 Open weights and the fast-follower playbook 8:25 Chinese labs: equal talent, less compute 9:55 Why distillation is harder than it looks 14:55 Is distillation theft or competition? 16:20 The lab flywheel vs owning your weights 21:45 Can general models beat every specialist? 26:30 AI at 1% of today's cost in 4 years 27:15 The RL env boom and reward hacking 31:30 What Trillium will build in the open 35:30 Breaking into AI in the age of AI slop 38:05 Open-sourcing how AI builds AI 44:25 Why Nathan isn't pre-training 49:45 Agent swarms and why lab secrets leak 54:25 Fast takeoff vs risks already here YouTube: https://youtu.be/BKuK0M6oDTI @natolambert @trillium_labs @lishali88

Training large models with RL typically runs on a fixed homogeneous cluster of the latest accelerators for numerical stability and faster throughput. This throws away inference optimization work done for other vendors and generations, limits the capacity you can utilize, and is unnecessarily expensive. So we trained a model on agentic coding tasks using fully asynchronous RL and mixed-precision, mixed-accelerator samplers. With off-policy correction, training stayed stable through convergence, on par with homogeneous samplers, at a lower cost.

Nvidia 576 GPUs Rubin Ultra slide!

The full walkthrough of our @nvidia Vera benchmark below.

AWS CEO Matt Garman 与 a16z 的 Raghu Raghuram 对谈：全球最大云提供商如何为智能体时代重构： 2026 年 2200 亿美元资本支出，订购 200 万块 NVIDIA GPU，AWS 自研 AI 芯片明年之前售罄，还推出无需信用卡、30 秒即可开通的 AWS 账户，让智能体立即动起来。 0:55 营收 1700 亿美元，增长 37% 2:40 30-40% 的营收最初来自初创公司 4:55 初创公司想要面向智能体的云 9:10 30 秒开通 AWS 账户 12:25 智能体不需要坚不可摧的数据库 16:30 为什么 AWS 不会把所有 GPU 卖给前沿实验室 22:00 为什么 Matt 不担心泡沫 24:45 为什么 AWS 自建电厂 27:45 为下一次供应短缺做规划 30:25 为一个县每位居民节省 5000 美元 33:10 AWS 如何打造自研芯片 40:05 是什么阻碍了企业级智能体 45:35 为什么 Bedrock 从不把你的数据发给模型 49:30 AWS 的新 AI 安全工具 51:50 为什么 AWS 的 10 人团队如今只需 3-4 人 YouTube: https://www.youtube.com/watch?v=rn_afJaPldg @mattsgarman @awscloud @RaghuRaghuram

Future-ready servers are here. Meet HPE ProLiant Gen13 Servers, powered by 6th Gen @AMD EPYC™ Server processors, with automatic security, intelligent ops, and breakthrough performance. Generation after generation, HPE and AMD are helping enterprises modernize for what’s next.