RoboColosseum 是一个真实世界机器人基准，在 150 多个场景中对数百台真实机器人盲测模型策略，并公开包括失败在内的全部 rollout。测试分 Open 赛道（人类裁判对比两个匿名模型的零样本表现）和 Fine-tune 赛道（同一开源数据微调后每模型 25 次尝试），覆盖来自 OpenAI、Anthropic、NVIDIA、Galaxea、Ai2 等 30 多个策略。
AI 生成摘要 · 以原文为准
Robotics finally has its real-world arena.
RoboColosseum is a real-world benchmark that tests robot policies on hundreds of actual robots across 150+ scenes, not cherry-picked demos. You bring a model, they run it blind on real hardware and publish every rollout (failures included) on Hugging Face.
Two ways models are tested:
- Open track: two unnamed models get the same brand-new instruction in the same scene. A human judge watches both runs and picks the winner, or a tie. Think LMArena, but on real robots. It measures zero-shot performance on novel tasks.
- Fine-tune track: every model is fine-tuned on one fixed task from the same open-source demos, then gets 25 attempts. An apples-to-apples test of how well a model learns a specific skill.
Over 30 policies from OpenAI, Anthropic, NVIDIA, Galaxea, Ai2 and more are tested across Unitree G1, Franka, YAM and SO-101.
Why it matters: across 20,000+ real runs, no model won everywhere. Rankings flipped depending on the task and the robot body. That's the whole point: a single demo or benchmark tells you almost nothing about what a policy can actually do. This is the field finally getting honest, real robots, in the wild, every failure published.
And 10,000 eval hours are free for academic labs.
来源：The Humanoid Hub · x.com