Why AI Infrastructure must evolve for Agent Experience — Akshat Bubna, Modal CTO

In a recent interview on the Latent Space podcast, Modal CTO Akshat Bubna argued that current AI infrastructure is not built for the emerging paradigm of agentic workflows. He contrasted the traditional request-response model of cloud APIs with the persistent, stateful, and multi-step interactions that agents require. Bubna explained that agents need infrastructure that can handle long-running tasks, manage intermediate state, and coordinate multiple model calls, which existing serverless and container platforms struggle to support efficiently. He pointed to Modal's own evolution as a case study, where the company redesigned its cloud to prioritize low-latency cold starts, dynamic scaling, and seamless data locality for agent loops. The practical takeaway for builders is that choosing the right infrastructure layer—one that minimizes overhead for agent orchestration—can dramatically simplify development and reduce costs. Rather than bolting agent logic onto general-purpose compute, teams should consider platforms that natively support agentic patterns like checkpointing, retries, and tool integration. This shift, according to Bubna, is necessary to move agents from demos to production.