The AI Kubernetes Show

Static analysis tools used to be right about as often as a coin flip. Kathleen Goeschel, Principal Product Security Engineer at Red Hat, spent years using machine learning to fix that, and now she's watching large language models change both sides of the fight. How fast vulnerable code gets written, and how fast it gets found.Goeschel walks through how she trained machine learning models on aggregated scanner alerts, software metrics, and prior CVEs to cut false positives in static analysis, and the surprising finding that code churn alone predicted real vulnerabilities better than most of the signals built for that purpose.  Read the blog post and follow us on LinkedIn!

If your AI agent restarts and loses everything it was doing, your platform team is stuck retrofitting recovery logic Kubernetes was never built for. Mark Fussell, CEO of Diagrid and creator of the Dapr project, joins the AI Kubernetes Show to explain why that's the real infrastructure problem in the agentic AI era, not the model.Fussell built the platform running Azure SQL Server at scale during 20 years at Microsoft, then spent the last 9 years building Dapr (Distributed Application Runtime), now a CNCF-graduated project. He and host William Morgan cover durable execution and why he calls it "process reincarnation," why non-determinism and multi-minute latency break assumptions that held for a decade of microservices, and why he tells platform teams to translate what they already know instead of relearning it.  TAKEAWAYS✓ Why durable execution, or "process reincarnation," matters more once language models are in the call path ✓ How to decide when to keep a workflow deterministic instead of routing it through a model ✓ Why non-determinism, not topology, is the real thing that changed since microservices ✓ How Dapr assigns identity at the process level using the SPIFFE standard, and why that matters for agents ✓ Why a log isn't the same thing as tamper-proof attestation ✓ Why Fussell says about 90% of existing platform engineering skills transfer directly to agentic systems⁠Read the blog post⁠!FIND AND FOLLOW US ON LINKEDIN✦ LinkedIn: https://www.linkedin.com/company/the-ai-kubernetes-show/

AI-generated code is 40% more likely to leak a secret. Matt Barker, co-founder of the company that built cert-manager and later led secure workload access at CyberArk, explains why SPIFFE might finally fix workload identity for AI agents.Matt and William Morgan cover why SPIFFE replaces long-lived secrets with short-lived, cryptographic workload identities, what Matt is building at BoltMCP to control what an AI agent can see and do once it's trusted, and why CISOs are being told to enable AI adoption "at all costs."Topics covered:Why SPIFFE replaces secrets with short-lived workload identities (SVIDs)Where SPIFFE's job ends and human-delegated authorization beginsWhy "context window flooding" breaks MCP integrations at scaleWhy Kubernetes stays the default foundation for agentic AI infrastructureRead the blog post: SPIFFE Identity for AI Agents on KubernetesFind us on:YouTubeApple MusicLinkedIn 

Every LLM inference request splits into two phases with completely different hardware profiles: pre-fill and decode. But most Kubernetes clusters still run them on the same pod. Morgan Foster, who works on the llm-d project and the AI Gateway Working Group out of Red Hat's Office of the CTO, joins William Morgan to explain what that means. Foster and Morgan get into why real-time inference workloads fit the primitives Kubernetes already has, while training often runs on Slurm instead. They walk through disaggregated inference: how pre-fill and decode get scheduled on separate pods, how KV cache blocks move between them over RDMA, and what happens today when a pre-fill or decode pod dies mid-request (nothing handles the failover cleanly, and the computed KV blocks just get thrown away). They also cover why faster token generation buys agentic systems more capability, not just quicker responses, and why Kubernetes proxies, built around header-based ingress routing, are struggling to govern AI traffic that depends on streamed JSON request bodies and inference happening outside the cluster. FIND AND FOLLOW US ON: ✦ Spotify: https://open.spotify.com/show/4LLTNjMbd7wNCYhSI9pkQU?nd=1&dlsi=3baba6bcc6b44cae ✦ Apple Music: https://podcasts.apple.com/us/podcast/the-ai-kubernetes-show/id1895650894 TAKEAWAYS✓ Why real-time inference workloads fit Kubernetes' existing primitives, while training often runs on Slurm instead ✓ How disaggregated inference splits the pre-fill and decode phases of an LLM request across separate pods ✓ Why KV cache blocks move between pods over RDMA, and what happens if the pre-fill or decode pod dies mid-transfer ✓ Why more tokens per second buys agentic systems more capability, not just faster responses ✓ Why Kubernetes proxies built for header-based routing struggle with AI traffic that depends on streamed request bodies ✓ What the AI Gateway Working Group is building to fix itRead the blog post at: https://www.buoyant.io/ai-kubernetes-episode/disaggregated-inference-on-kubernetes

There is no agreed upon definition of what cloud native AI means yet, but if you're trying to figure out where Kubernetes AI standards are really headed, check out this episode with someone who's worked on the CNCF's first attempt to define cloud native AI. William Morgan talks with Ron Petty (RXM, CNCF founding contributor, and member of the CNCF's TCG AI) about what's actually settled in cloud native AI and what isn't. They cover the CNCF's shift from working groups to technical community groups (TCGs), Petty's own framing of cloud native AI as a two-way street between building models and using AI to run cloud native systems, and the 15-criteria self-reported conformance program (mostly GPU management) that gets you an "AI certified" stamp. TAKEAWAYS ✓ Why Ron Petty defines cloud native AI as a two-way street: building models on cloud native infrastructure, and using AI to run cloud native systems ✓ What the CNCF's self-reported AI conformance program actually checks, including GPU management and having an AI router on your cluster ✓ Why certificate management and rotation is still unsolved, and how that connects to why Linkerd defaults to mutual TLS ✓ What's actually wrong with Model Context Protocol's design, according to a proxy engineer's read on it ✓ Why long-running AI agents don't need fundamentally different guardrails than short-running ones ✓ How K8sGPT uses existing Kubernetes checks plus an LLM to help junior engineers diagnose cluster issues fasterRead the blog post at: www.buoyant.io/ai-kubernetes-episode/the-state-of-cloud-native-ai-and-whats-still-evolving

Most testing tools bolt onto a CI pipeline and hope for the best. Ole Lensmar, CTO of Testkube, built a company that changes that script. Pull test execution out of CI/CD entirely and run tests as Kubernetes workloads instead.In this episode, Lensmar walks through why Testkube uses Kubernetes as its test execution engine, running Selenium, Playwright, K6, JMeter, and Postman tests as cluster workloads instead of asking teams to adopt new tools. He and podcast host William Morgan dig into who should own testing (developers own functional tests,  while the platform team owns performance, security, and chaos testing). Lensmar's core recommendation is to treat testing and quality as a platform capability, the same way you already treat CI and CD. More AI-generated code means more tests are needed, intelligent test selection can keep pipelines fast as test counts grow, and AI is already useful for triaging failed test logs using Kubernetes and Grafana MCP servers.  TAKEAWAYS: ✓ Where testing ownership splits between developers and the platform team, and where it doesn't ✓ Why "testing as a platform service" should get the same priority as CI and CD ✓ How intelligent test selection works, and why you should never rely on it alone ✓ Where AI already adds value today: triaging failed test logs with Kubernetes and Grafana MCP servers ✓ Why testing the AI components of your application (agents, evals) is becoming its own discipline.Read the blog post at:

Running LLM inference on Kubernetes requires new primitives for routing, autoscaling, and GPU scheduling. Here's what platform engineers need to know.

In this episode of The AI Kubernetes Show, Blake Romano, Staff Software Engineer at Imagine Learning, walks through what it actually looks like to build and run AI agents on Kubernetes at scale. He talks about the architecture choices, the failures, and why the organizational context you bring to the LLM matters more than which Software Development Kit (SDK) you use.Imagine Learning is a K-12 education company building digital platforms for students and educators, and Blake has been driving AI and platform engineering initiatives there.

Learn platform-level security patterns for AI agents on Kubernetes. Close the production gap with LLM guardrails, tool filtering, and short-lived tokens.

Learn how Schonfeld scaled their internal AI platform, SchonAI, using Kubernetes and established guardrails to manage AI agent code volume. Build your AI-native workflow now.