TensorZero nabs $7.3M for open source LLM platform.
TensorZero, a Brooklyn-based startup building open-source infrastructure for large language model (LLM) applications, has raised $7.3 million in seed funding to address the challenges enterprises face in deploying production-ready AI systems.
The funding round was led by FirstMark, with participation from Bessemer Venture Partners, Bedrock, DRW, Coalition, and a network of strategic angel investors. The company, founded just 18 months ago, is now among the fastest-rising players in the enterprise AI tooling space.
A push for unified enterprise infrastructure
TensorZero’s approach responds to a growing enterprise frustration: fragmented solutions that require multiple tools stitched together for model access, monitoring, optimisation, and experimentation. Its open-source repository recently surged in popularity, jumping from about 3,000 stars to more than 9,700 on GitHub, making it the top trending project globally for a week.
“Despite all the noise in the industry, companies building LLM applications still lack the right tools to meet complex cognitive and infrastructure needs, and resort to stitching together whatever early solutions are available on the market,” said Matt Turck, General Partner at FirstMark. “TensorZero provides production-grade, enterprise-ready components for building LLM applications that natively work together in a self-reinforcing loop, out of the box.”
From nuclear fusion research to enterprise AI
TensorZero’s philosophy stems from co-founder and CTO Viraj Mehta’s PhD work at Carnegie Mellon, where he focused on reinforcement learning for nuclear fusion reactors. During U.S. Department of Energy projects, he faced extreme constraints.
“That problem leads to a huge amount of concern about where to focus our limited resources,” Mehta said. “We were going to only get to run a handful of trials total, so the question became: what is the marginally most valuable place we can collect data from?”
That experience informed TensorZero’s central idea: maximising the value of every data point to improve systems continuously. Mehta, together with co-founder Gabriel Bianconi, former chief product officer at Ondo Finance, framed enterprise AI applications as reinforcement learning challenges.
“LLM applications in their broader context feel like reinforcement learning problems,” Mehta explained. “You make many calls to a machine learning model with structured inputs, get structured outputs, and eventually receive some form of reward or feedback. This looks to me like a partially observable Markov decision process.”
Performance-driven and open source
Unlike LangChain and LiteLLM, which are widely used for prototyping but often struggle at scale, TensorZero is focused on production-grade deployment. Its Rust-based gateway supports all major LLMs through a unified API, handling more than 10,000 queries per second with sub-millisecond latency.
“LiteLLM (Python) at 100 QPS adds 25-100x+ more P99 latency than our gateway at 10,000 QPS,” the founders noted.
The company describes its core innovation as a “data and learning flywheel” — a feedback loop where production metrics and human feedback continuously optimise models, making them faster and cheaper over time.
“Most companies didn’t go through the hassle of integrating all these different tools, and even the ones that did ended up with fragmented solutions, because those tools weren’t designed to work well with each other,” Bianconi said. “So we realized there was an opportunity to build a product that enables this feedback loop in production.”
The company’s open source approach is also resonating with enterprises that need control over sensitive data. TensorZero has pledged to keep its core platform entirely open source, ruling out paid feature tiers.
“We realized very early on that we needed to make this open source, to give [enterprises] the confidence to do this,” Bianconi said. “In the future, at least a year from now realistically, we’ll come back with a complementary managed service.”
This managed service will target heavier optimisation tasks, such as GPU management, automated experiments, and proactive performance tuning.
What comes next
With fresh funding, TensorZero plans to expand its New York team, accelerate development of its infrastructure, and release research tools for faster experimentation.
“Our ultimate vision is to enable a data and learning flywheel for optimizing LLM applications—a feedback loop that turns production metrics and human feedback into smarter, faster, and cheaper models and agents,” Mehta said. “As AI models grow smarter and take on more complex workflows, you can’t reason about them in a vacuum; you have to do so in the context of their real-world consequences.”
