Dev Interrupted podcast

Who watches the agents? Observability, evals, and guardrails at scale | Grafana Labs’ Devin Cheevers

11/20/2026
0:00
34:51
Rewind 15 seconds
Fast Forward 15 seconds

When an agent takes thousands of actions a day, someone has to keep watch over what it’s doing. Grafana Labs director of product Devin Cheevers saw how much can slip past unnoticed when a one-line timestamp added to an agent’s system prompt wiped out its cache hit rate with the LLM provider. Recorded on site at Grafana’s ObservabilityCON in San Francisco, host Andrew Zigler sits down with Devin to talk through why vibes don’t scale once more teams contribute to an agent, and how evals let domain experts define what good looks like. Devin also shares where guards can stop an agent automatically, and where a human still needs to step in on its work. 

Come bake with us at The Great Business Bake Off

Follow the show:

Follow the hosts:

Follow today's guest:

OFFERS

  • Start Free Trial: Get started with LinearB's AI productivity platform for free.
  • Book a Demo: Learn how you can ship faster, improve DevEx, and lead with confidence in the AI era.

LEARN ABOUT LINEARB

  • AI Code Reviews: Automate reviews to catch bugs, security risks, and performance issues before they hit production.
  • AI & Productivity Insights: Go beyond DORA with AI-powered recommendations and dashboards to measure and improve performance.
  • AI-Powered Workflow Automations: Use AI-generated PR descriptions, smart routing, and other automations to reduce developer toil.
  • MCP Server: Interact with your engineering data using natural language to build custom reports and get answers on the fly.

OFFERS

  • Start Free Trial: Get started with LinearB's AI productivity platform for free.
  • Book a Demo: Learn how you can ship faster, improve DevEx, and lead with confidence in the AI era.

LEARN ABOUT LINEARB

  • AI Code Reviews: Automate reviews to catch bugs, security risks, and performance issues before they hit production.
  • AI & Productivity Insights: Go beyond DORA with AI-powered recommendations and dashboards to measure and improve performance.
  • AI-Powered Workflow Automations: Use AI-generated PR descriptions, smart routing, and other automations to reduce developer toil.
  • MCP Server: Interact with your engineering data using natural language to build custom reports and get answers on the fly.

More episodes from "Dev Interrupted"