Each year, we bring together founders, AI researchers, and engineering leaders to discuss the future of AI infrastructure. This year, we were delighted to host the event over breakfast at our Tel Aviv office, exploring how AI infrastructure is evolving and where the next wave of innovation might emerge.
The conversation covered model architecture, compute economics, inference infrastructure, and Israel's role in the AI ecosystem. Here are six key takeaways that stood out.
1. As models improve, much of today's scaffolding and orchestration could become unnecessary.
Agent swarms, orchestration layers, and skills help models search, manage context, and draw on past experience. As models become more capable through developments in long-context processing, sample efficiency, and continual learning, many of these capabilities could become native to the models themselves.
2. Physical AI could drive the next major breakthrough in model architecture.
Physical AI presents a fundamentally different set of challenges from language models. Interacting with the physical world requires understanding spatial relationships, cause and effect, and continuously changing environments, often with limited training data and little room for error. Simply scaling today's LLM architectures may not be enough. Solving these challenges could drive the next major breakthrough in model architecture, opening up an entirely new area of AI innovation.
3. There is significant room to improve compute efficiency.
Training a frontier model requires roughly 10²⁶ operations, while inference can consume several times that amount over the model's lifetime. Much of this computation produces results that are not retained or reused. Rather than taking today's compute requirements as a given, there is an exciting opportunity for companies to rethink how compute is used and develop fundamentally more efficient approaches.
4. The shift toward inference creates new demands for data infrastructure.
Unlike training, inference often requires decisions to be made in real time, sometimes within microseconds. This creates a need for infrastructure that can make relevant data immediately accessible while maintaining security, privacy, and access controls.
5. The shift toward inference lends itself to Israel's AI ecosystem.
Training frontier models requires levels of capital and infrastructure that are difficult for most companies to access. This differs from the capital-efficient innovation that has historically characterized Israel's technology ecosystem. As more innovation shifts toward inference, Israeli founders may be particularly well positioned to build competitive infrastructure businesses.
6. The growing competitiveness of open-weight models creates opportunities for founders.
Open-weight models are closing the gap with frontier proprietary models, allowing enterprises to fine-tune models to their needs while retaining control over their data. For founders, a competitive open-weight ecosystem is critical to reducing dependence on a handful of model providers and preserving the ability to build differentiated products independently.
What This Means for Founders
As AI adoption accelerates, some of the most interesting infrastructure challenges are emerging around compute efficiency, inference, real-time data access, and the ecosystem supporting open-weight models. These are areas where focused teams can develop differentiated solutions without the capital requirements of training frontier models, making them particularly relevant to Israel's technology ecosystem.
If you're building in AI infrastructure, we'd love to hear from you.