
Interested in being a guest? Email us at [email protected]
AI agents are starting to run real customer workflows, and the hardest part isn’t getting a demo to work, it’s knowing the agent is behaving correctly at scale. We talk with Mukul Karnik Vice President of AWS Observability at Amazon Web Services about what “observability for AI agents” actually means when the system can be up, fast, and still wrong. Think guardrails, accuracy, and policy compliance alongside the classic metrics like latency and availability.
We get specific about failure modes that don’t look like normal software bugs: an agent that confidently approves a hazmat return because a user asked the question in an odd way, or an agent that answers something it should refuse. Mukul explains how evals act like tests for agent behavior, how tracing and telemetry help you see tool calls and dependencies end to end, and why a unified view matters when the agent relies on apps, databases, and infrastructure to get the job done. We also dig into what CloudWatch Omni is aiming for with an AI-first experience, including deeper investigations that can be driven by a DevOps agent for root cause analysis.
We also cover how teams can start earlier in the workflow with a free local IDE plugin to visualize agent traces, then bring the same evals and monitoring into production. On top of that, we talk multi-cloud realities, supporting different agent frameworks, and why the line between observability and security keeps getting blurrier as more signals live in the same logs and telemetry. If you’re building agentic systems, this is a practical look at monitoring, debugging, and building trust in autonomous behavior.
Subscribe for more conversations on AI infrastructure and production engineering, share this with a builder on your team, and leave a review with the toughest agent reliability problem you’re trying to solve.
More at https://linktr.ee/EvanKirstel
Flere episoder fra "What's Up with Tech?"



Gå ikke glip af nogen episoder af “What's Up with Tech?” - abonnér på podcasten med gratisapp GetPodcast.








