Open source AI inference project vLLM is courting investors for a US$160 million raise, signalling a major bet on open infrastructure to tame rising AI inference costs.
vLLM, an open source AI inference project spun out of UC Berkeley, is in talks to raise at least US$160 million, despite generating little revenue so far, according to four independent sources speaking to Forbes. The proposed fundraising includes US$60 million in an initial round, followed by at least US$100 million in a second tranche.
The discussions could value vLLM at around US$1 billion, though sources cautioned that the numbers remain fluid. The project originated in the UC Berkeley lab of Ion Stoica, co-founder of Databricks. Simon Mo, co-leader of vLLM, has been pitching investors across Silicon Valley but did not respond to requests for comment.
vLLM currently appears to have no website and no defined commercialisation strategy beyond donations. To date, it has raised just US$300,000, including backing from Sequoia. Yet investor interest is being driven by its technology and adoption rather than revenue.
The project maintains an open source software library on GitHub that improves large language model inference by using GPU memory more efficiently, enabling AI workloads to run on fewer servers. Unlike cloud-only rivals, vLLM allows organisations to optimise inference on their own chips and infrastructure. It has become one of the most starred projects on GitHub.
The interest reflects a broader shift in AI economics. As spending moves from training models to running them, inference costs are rising sharply. OpenAI may be spending over 25 percent of Sora’s revenue on inference, which Sora head Bill Peebles called “completely unsustainable.”
Investors are increasingly backing inference-focused startups, including Fireworks, Baseten and Fal. They also point to open-source success stories such as Red Hat, GitLab and MongoDB. Red Hat was acquired by IBM for US$34 billion in 2019 after generating US$3.4 billion in revenue.
“There’s going to be hundreds of billions of dollars spent on inference,” said Dylan Patel, Founder of Semianalysis. “Redhat makes so much money developing open source software and then building on top of it with services for Linux. So yes, I think it’s possible for vLLM to do the same.”
