The Context Problem: What's Really Breaking AI Agents at Scale

DuckLake is simplifying data lakehouse management by replacing complex metadata files with a familiar SQL-based approach. In this episode of The New Stack Podcast, Pedro Holanda, lead DuckLake developer and principal engineer at DuckLake, discusses how the open-source table format reduces complexity, improves performance, and makes large-scale data management more accessible.With the release of DuckLake 1.1 alongside DuckDB 2.0.0 LTS, Holanda explains how storing metadata in a relational database eliminates the need to navigate countless files to locate and query data. By combining Parquet storage with SQL catalogs, DuckLake streamlines data access while addressing common challenges such as excessive small files, inefficient queries, and costly maintenance.The conversation also explores DuckLake’s innovative inlining capabilities, which consolidate smaller datasets to improve efficiency, and its ability to support analytics without massive server infrastructure. As AWS integrates DuckDB into its broader analytics strategy, DuckLake represents an important step toward simplifying data portability, reducing operational overhead, and enabling more efficient analytics for developers and AI agents.Learn more from The New Stack around the latest developments around DuckDB:Why MotherDuck refuses to fork DuckDBDuckDB: Query Processing Is KingJoin our community of newsletter subscribers to stay on top of the news and at the top of your game.   

As AI models become more powerful and affordable, enterprises still face a major obstacle to realizing the full potential of AI agents: the context gap. Without access to accurate, relevant information, even the most advanced models struggle to deliver meaningful results. In this episode of The New Stack podcast, Sri Desikan, an executive at Elastic, explains why providing AI agents with the right context is essential for moving beyond impressive demonstrations to reliable enterprise automation. Simply connecting agents to enterprise data through MCP servers isn't enough. Feeding massive amounts of information into large language models is costly, inefficient, and slow. Elastic's approach centers on building an AI index that organizes enterprise data, metadata, and relationships, enabling agents to retrieve relevant information faster while reducing token consumption. Desikan also explores how precomputed context can help prevent outdated information from influencing AI decisions. The conversation examines the implications for data privacy, agent autonomy, and the future of enterprise AI adoption. Learn more from The New Stack around the latest developments in bridging the enterprise AI context gap: The Real Bottleneck in Enterprise AI Isn’t the Model, It’s Context Why enterprise AI breaks without metrics discipline Why enterprise AI keeps stalling — and how data streaming could unlock it Join our community of newsletter subscribers to stay on top of the news and at the top of your game. 

As AI agents transform software development, developers face a fundamental shift in the skills they need to succeed. In this episode of The New Stack podcast, recorded at AGNTCon and MCPCon in San Jose, Frederic Lardinois speaks with Angie Jones, Vice President of the Agentic AI Foundation, about the evolving developer role and the challenges of adopting AI across engineering organizations.Drawing on her experience leading AI enablement at Block, Jones explains how embedding agent-friendly practices into development workflows helped drive widespread adoption. She emphasizes the growing importance of system design, logical thinking, and building agentic systems rather than simply using AI coding tools.The conversation also explores opportunities for junior developers, the role of open standards like MCP, and how AI-generated contributions are reshaping open source maintenance.As organizations navigate this transition, Jones highlights the importance of cultural change, collaboration, and developing new skills to build secure, reliable AI agents that extend beyond software development. Learn more from The New Stack around the latest developments around the evolving role of the developer:The 3 roles AI agents play in your developer platformAI Will Steal Developer Jobs (But Not How You Think)10 New Jobs for Burned-Out Software DevelopersJoin our community of newsletter subscribers to stay on top of the news and at the top of your game.   

AI adoption is accelerating among cloud-native developers, who are leading the charge in integrating artificial intelligence into software development. According to the Cloud Native Computing Foundation’s (CNCF) Q3 2026 State of Cloud Native Development report, 21.2 million developers—41% of the global developer population—work with cloud-native technologies. Of those, 75% use AI to write and ship code, compared with just 50% of non-cloud-native developers.In this episode of The New Stack podcast, CNCF Executive Director Jonathan Bryce discusses the rapid growth of the cloud-native ecosystem, evolving developer trends, and the expanding role of open source in driving innovation.Bryce highlights CNCF’s project maturity process, from sandbox to incubation and graduation, and explores emerging technologies such as KubeRay and Agent Substrate alongside graduating projects including KubeVirt, Karmada, and Kubeflow.The conversation also examines the Kubernetes AI Conformance program and why understanding open-source technologies is increasingly essential for organizations seeking flexibility, innovation, and competitive advantage in an AI-driven world. Learn more from The New Stack around the latest developments around the State of the Cloud and the evolving cloud native ecosystem.Cloud native application challenges: installing the walking skeletonCloud Realities Are Slowing AI AmbitionsAfter becoming cloud computing’s telemetry standard, OpenTelemetry graduates into the AI infrastructure eraJoin our community of newsletter subscribers to stay on top of the news and at the top of your game. 

As AI agents transform how businesses access and process data, traditional siloed infrastructure is struggling to keep pace. In this episode of The New Stack podcast, recorded at AGNTCon + MCPCon 2026, Frederic Lardinois speaks with Bianca Lewis, Executive Director of the OpenSearch Software Foundation, about why unified, open source data infrastructure is becoming essential for agentic AI.Lewis explains how OpenSearch is evolving beyond traditional search and observability into an AI data infrastructure platform that enables organizations to scale workloads, reduce costs, maintain data sovereignty, and avoid vendor lock-in.The conversation explores how AI agents introduce new observability challenges, requiring organizations to monitor API interactions, trace agent behavior, and capture contextual data to improve accuracy and performance.With production clusters nearly doubling over two years and a growing community of 3,500 contributors, OpenSearch is rapidly expanding its capabilities. Lewis also discusses the foundation's evolving commercial ecosystem, enterprise support programs, and the importance of community-driven innovation in shaping the future of AI infrastructure.Learn more from The New Stack around the latest developments around agentic AI and data infrastructure:Agentic infrastructure operations begin with accurate, reliable infrastructure dataThe rise of agentic AI on Kubernetes: unleashing the new infrastructure layer Join our community of newsletter subscribers to stay on top of the news and at the top of your game. 

AI coding agents are becoming more autonomous, thanks to advances in harness engineering that enable them to identify and correct mistakes before human review. In the latest episode of The New Stack podcast, Vanitha Kumar, Principal Technologist at Thoughtworks, explains how automated tests, feedback loops, and guardrails allow agents to handle longer running tasks with less human intervention while working in parallel.Kumar also explores the rise of AI factories, which provide standardized infrastructure for model selection, workload management, and cost optimization. Much like Kubernetes simplified microservices management, AI factories aim to make deploying and managing AI agents easier at scale.However, greater autonomy introduces security risks as agents gain access to more systems, data, and tools. Kumar emphasizes the importance of guardrails, controlled environments, and human approvals to mitigate these vulnerabilities. She also previews Thoughtworks’ Technology Radar, Volume 35, highlighting emerging AI practices alongside established engineering disciplines, including penetration testing and path-to-production understanding, that are becoming increasingly critical in the AI era.Learn more from The New Stack around the latest developments around engineering disciplines, including penetration testing and path-to-production understanding:Why AI engineering needs old-school disciplineAgents are rewriting the rules of security. Here’s what engineering needs to know.Join our community of newsletter subscribers to stay on top of the news and at the top of your game.

In this episode of The New Stack podcast, recorded at AGNTCon + MCPCon in San Jose, Nick Lucchesi speaks with Mike Chambers, Senior AI Developer Experience Engineer at AWS, about how small decision models could make AI agents faster, more efficient, and cost-effective.Unlike large language models (LLMs) that generate responses through multiple processing steps, decider models quickly evaluate information and return probability-based decisions. Chambers explains how these models can improve agent performance by routing requests to appropriate LLMs, enforcing safety guardrails, and determining when agents should retrieve information from memory.He also shares experiments using AWS Strands Agents and Strands Decider to reduce unnecessary memory lookups and keep agent context relevant. While these models offer significant efficiency advantages, Chambers emphasizes the importance of evaluation, monitoring, and human oversight, particularly for high-risk decisions involving financial transactions.Looking ahead, Chambers envisions a future where developers combine specialized AI models rather than relying exclusively on LLMs, paving the way for more capable, adaptable agents and applications.Learn more from The New Stack around the latest developments around small decision models and its impact on AI:  SLMs vs. LLMs: Why Smaller AI Models Win in BusinessAWS launches a local answer to TypeSafe’s Jev decision modelJoin our community of newsletter subscribers to stay on top of the news and at the top of your game.

In this episode of The New Stack podcast, recorded at AgentCon + MCPCon in San Jose, host Nick Lucchesi speaks with Manik Surtani, co-founder and CTO of the Agentic AI Foundation (AAIF), about the growing importance of open standards, interoperability, and vendor-neutral governance in agentic AI.With 305 member organizations and 10 projects, AAIF is expanding its ecosystem through new additions including Anthropic’s AgentSkills, OpenAI’s OpenResponses, Hugging Face’s OpenM, and Alibaba’s OpenSandbox. Surtani explains how these projects help developers build more portable, trustworthy AI systems while reducing dependence on individual vendors.The conversation explores unresolved challenges in agent memory, identity, authentication, observability, and verifying work performed across multiple agents. Surtani also discusses the foundation’s evolving approach to supporting early-stage open-source projects and why the agentic AI stack may never fully stabilize.Looking ahead, he predicts that AI agents will transform commerce, from purchasing decisions to advertising, making open protocols and industry collaboration increasingly essential.Learn more from The New Stack around the latest developments around open standards and interoperability in agentic AI:  The Agentic Web: How AI Agents Are Shaping the Web’s Future 6 agentic knowledge base patterns emerging in the wild Five AI rivals just backed a shared plugin standard. Here's why it matters for developers.  Join our community of newsletter subscribers to stay on top of the news and at the top of your game. 

Turning a promising AI demo into a reliable, production-ready application requires more than powerful models. At AGNTCon + MCPCon in San Jose, Jeu George, Co-Founder and CEO of Orkes, joins The New Stack’s Nick Lucchesi to discuss the challenges of building mission-critical agentic applications and the infrastructure needed to support them.George explains how Orkes Conductor helps developers orchestrate AI agents, manage failures, enforce security guardrails, and maintain audit trails for accountability and compliance. He also introduces ParticleDB, an agentic database designed to simplify enterprise data access while keeping sensitive information within company cloud environments.Using examples from financial services and e-commerce, George explores when businesses should rely on AI agents, deterministic workflows, or human oversight. He emphasizes the importance of designing systems that anticipate failures, recover automatically, and provide visibility into every agent action.Looking ahead, George predicts cheaper AI models, faster processing, and greater adoption of privately deployed agents will accelerate enterprise AI innovation.Learn more from The New Stack around the latest developments around building mission-critical agentic applications and the infrastructure needed to support them in the AI era:Why Agentic AI Orchestration Is Key To Managing AI ComplexityA practical systems engineering guide: Architecting AI-ready infrastructure for the agentic eraThe rise of agentic AI on Kubernetes: unleashing the new infrastructure layerJoin our community of newsletter subscribers to stay on top of the news and at the top of your game.

At AGNTCon + MCPCon in San Jose, Mazin Gilbert, executive director of the Agentic AI Foundation, joins The New Stack’s Frederic Lardinois to discuss the rapid growth of agentic AI and the importance of open standards in shaping its future.With 4,300 conference registrants and a rapidly expanding membership, the foundation is bringing together technology giants, startups, nonprofits and enterprises to address challenges around scalability, security, interoperability and governance. Gilbert explains how protocols like MCP are fueling a new economy of startups while enabling competing companies, including OpenAI, Anthropic, Google and Microsoft, to collaborate on shared infrastructure.The conversation explores the foundation’s approach to adopting new open source projects, expanding global participation through partnerships with organizations like the Gates Foundation, and encouraging meaningful community contributions. Gilbert also highlights emerging opportunities in agentic commerce, identity, trust, observability and physical AI.Looking ahead, the foundation aims to accelerate standardization and expand its global community, ensuring agentic AI systems can scale securely without costly fragmentation.Learn more from The New Stack around the latest developments around growth of agentic AI and the importance of open standards in shaping its future:The Architect's Guide to Understanding Agentic AI The Agentic Web: How AI Agents Are Shaping the Web's Future Join our community of newsletter subscribers to stay on top of the news and at the top of your game.