From Data Labeling to RL Environments
Over the last year, the conversation at frontier AI labs has started shifting from pretraining data to post-training environments.
Researchers are no longer just training models to generate text, but agents that can browse the web, conduct research, use software tools, and complete multi-step workflows autonomously. As agents have started proliferating across enterprises and consumer applications, pushing the frontier of what agents can do has become one of the next major challenges in AI. That progress depends on more compute, better architectures, and better data. Of the three, high-quality data is increasingly the hardest to scale – and for agents, that data is no longer static text but the environments they learn from.
That shift looks very different from the last generation of AI infrastructure. Over the last five years, some of the largest businesses in AI were built around supplying labeled data to frontier labs. Companies like Scale AI, Surge, and Mercor became critical infrastructure providers because training better models required enormous amounts of high-quality human-labeled data.
Why Agent Environments Are Hard to Build
Agents do not learn the same way traditional LLMs do. A static model learns from labeled examples, whereas agents best learn from interacting with dynamic situations, or environments, where they take actions, receive feedback, and continuously adapt their behavior over time. As frontier labs shifted from training chatbots to training autonomous agents capable of completing multi-step tasks, reinforcement learning environments have increasingly emerged as a core part of the modern AI training stack. Reinforcement learning has existed for decades, most well known for its use cases in chess or Atari games where the rules and rewards were clearly defined. Training agents for real-world digital workflows is significantly harder.
Unlike the structured environments of games like chess or Atari, real-world environments are messy and constantly changing. An agent using a browser, conducting research, or navigating enterprise software needs to make decisions across many steps while adapting to changing information along the way. Building environments that simulate these workflows realistically is becoming one of the biggest bottlenecks in training agents. And the bottleneck is not simply creating environments; it is creating environments that models can continuously learn from.
These environments cannot be too easy, otherwise the agents can’t learn from it, but they also cannot be so difficult that agents repeatedly fail without improving. The challenge is finding the “Goldilocks Zone” that consistently pushes models beyond their limits. As Patronus co-founder and CTO Rebecca Qian described to us, building these systems requires creating a kind of “SWAT team” for models that can aggressively stress-test agents while still allowing them to improve.
Most RL environments today are still built manually. Humans design the tasks, define the rewards, and carefully curate the workflows agents train against. That process works at small scale, but it becomes increasingly difficult as models improve. Human curation becomes the bottleneck, environments remain relatively static, and there still remains a struggle to generate enough high-quality scenarios for agents to continue learning from.
Enter Patronus
Patronus is building Digital World Models (“DWM”), AI models trained to simulate how digital environments behave. Instead of manually building every workflow and task by hand, Patronus trains models that can generate training environments across domains like software engineering, computer use, customer support, finance, and research workflows.
What we found compelling is that this approach directly addresses the core scaling problem in RL environments. Traditional environments are largely static, whereas Patronus’s environments are designed to evolve alongside the models themselves. As models improve, the environments can generate new tasks, modify difficulty, and continuously produce fresh training scenarios that remain relevant as frontier capabilities move forward.
The Patronus AI Team
The company has already seen strong traction, growing revenue 15x over the last year as they support a majority of the leading frontier AI labs and hyperscalers. Patronus has quickly become deeply embedded within most of the leading model providers, with engagements rapidly expanding as labs increase investment into reinforcement learning infrastructure and next-generation agent capabilities.
A Research Team Built for This Problem
Patronus’s approach came from a deep background and understanding of the space and where it is headed. The company’s research background stood out to us immediately as well. Co-founders Anand Kannappan and Rebecca Qian previously led early work in responsible AI research and causal inference at Meta, and have since spent their careers at the frontier of AI evaluation — building category-defining systems for LLM benchmarking, hallucination detection, and LLM- and Agent-as-a-Judge evaluation now used by hundreds of thousands of people worldwide. That experience matters because one of the hardest parts of reinforcement learning is not simply generating tasks, but evaluating results and building reward systems that actually improve models without creating shortcuts or reward hacking behaviors.
Co-Founders (Left to Right): CEO Anand Kannappan & CTO Rebecca Qian
Beyond the founders, the broader team also impressed us. Patronus has assembled a highly technical organization of researchers with experience across companies like Meta Superintelligence, Inflection AI, Scale AI, and Amazon AGI. Building scalable RL environments is an unusually interdisciplinary problem, and we believe the team has developed a rare combination of research depth and execution velocity.
Throughout the process, we were consistently impressed by the company’s perspective on where reinforcement learning is headed. They did not describe this as a simple data generation problem or a pure infrastructure problem. Their view was that building high-quality RL environments requires combining simulation infrastructure, evaluation systems, and domain understanding in a way that scales as models improve.
Why We’re Excited
We do not think this market will ultimately be won by the companies that scale contractor labor the fastest. In the short term, many vendors will likely be able to build environments manually for narrow use cases. But over time, we believe the largest outcomes will accrue to the platforms that can consistently generate high-quality environments while scaling at the pace frontier AI labs need.
The broader opportunity here reminds us of earlier infrastructure shifts in AI. Pretraining created the modern data-labeling market, producing a new generation of companies that became deeply embedded within the AI development stack. We believe post-training reinforcement learning is creating the next major infrastructure layer in AI.
That’s why we’re excited to lead Patronus’s recent $50M Series B as they help define this category.
Left to Right: Itay Inbar, Anand Kannappan, Rebecca Qian, Olivia Levine