Berkeley 博士生 Woosuk Kwon 2022 年发起的 vLLM 项目已发展为最大的开源推理引擎之一，任何时刻运行在约 50 万块 GPU 上，Google、Amazon 和 Meta 均在生产环境使用。项目从操作系统借来虚拟内存思想按需分配显存；今年 1 月核心维护者创办 Inferact 并融资 1.5 亿美元，为 vLLM 组建全职团队，正在旧金山和新加坡招聘工程。
AI 生成摘要 · 以原文为准
梳理了 vLLM 从 Berkeley 课题到大规模开源推理引擎的脉络，其虚拟内存方法和 1.5 亿美元融资细节可供了解推理基础设施格局。
In 2022, Berkeley PhD student Woosuk Kwon thought he could make Meta's new open-weight model, OPT-175B, run faster in two weeks.
Four years and 3,000 contributors later, that project is @vllm_project, one of the largest open-source inference engines, running on ~500,000 GPUs at any given time.
He and fellow researchers Simon Mo, Kaichao You, and Roger Wang borrowed "virtual memory" from operating systems, handing out memory only as it's needed so each GPU can juggle requests of wildly different sizes. (The "v" in vLLM is for virtual.)
Their first user, Berkeley neighbor @arena, cut its GPU needs in half. Today Google, Amazon, and Meta all run vLLM in production.
In January, the core maintainers founded @inferact and raised $150M to give vLLM a full-time team.
They're hiring across eng, marketing, and devrel in SF and Singapore: https://a16zjobs.substack.com/p/inferact-raised-one-of-the-largest
@simon_mo_ @woosuk_k @KaichaoYou @rogerw0108
来源：a16z · x.com