Security Teams Are Moving Too Slowly for AI Agents | Zack Korman

Zack Korman on why the AI safety debate has been captured by an extinction narrative, and why the incident everyone is calling a watershed moment looks a lot more like a preventable security failure.

In this episode I sit down with Zack Korman, co-founder of Embroidery, where he uses AI to monitor AI agents, and former CTO of the cybersecurity company Pistachio. I first came across Zack on X during the controversy over fake SOC 2 reports, and he has since become one of the sharpest critics of how AI safety is being framed, funded, and investigated.

We get into the effective altruism roots of the existential risk movement, why he argues METR's review of the Hugging Face incident was not independent oversight, and what an actual incident response firm would have done differently. Zack makes the case that alignment is one control among many, that there is no "safe" path in AI, only trade-offs, and that the real risk for most organizations is enterprise environments that were never ready for agents in the first place.

In this episode:

● How building an AI insider threat product pulled a developer and CTO into the cybersecurity community, and into picking fights on X
● Effective altruism, longtermism, and how a focus on preventing extinction came to dominate AI safety
● Why "just be careful" misses the point when every path involves trading one risk for another
● Dario Amodei's Pacing the Frontier, independent auditors, and why Zack calls bringing in METR "bring your friend to work day"
● The funding and relationships connecting METR, Redwood Research, Coefficient Giving, and the labs
● True believers versus IPO hype, and why genuine belief does not make someone right
● Why many AI doomers believed we were all going to die before they ever learned about computers
● The Hugging Face investigation, context drop in AI-analyzed transcripts, and treating knowable facts as unknowable
● What Unit 42 or Mandiant would have demanded before putting their name on the report
● Eight layers of failure, from a single package proxy at egress to missing monitoring, classifiers, and kill switches
● Alignment failure versus containment failure, and why alignment belongs inside defense in depth
● The real risks: threat actors misusing AI, enterprise agent deployments, and new attack chains
● Why security's risk-averse, laggard culture may be its biggest vulnerability

Chapters:

0:00 Intro
0:54 Zack's Background
1:43 What Turned a CTO Into an AI Safety Critic
3:31 Effective Altruism and the Extinction Narrative
5:53 No Safe Path, Only Trade-offs
6:40 METR, Redwood, and Bring Your Friend to Work Day
10:23 True Believers, Hype, or Both
12:49 How Doomers Find Their Way Into AI
14:39 Taking AI Risk Seriously When It Is Ideological
16:56 What an IR Firm Would Have Asked
21:10 Eight Layers of Failure in the Hugging Face Incident
22:49 Alignment Failure Versus Containment Failure
24:32 Alignment as One Layer of Defense in Depth
26:22 Why Enterprise Environments Are Not Ready for Agents
29:10 Security's Laggard Culture
29:19 Closing

Connect with Zack Korman:

X: https://x.com/ZackKorman
Embroidery: https://embroidery.io
Website: https://zkorman.com

Resilient Cyber: https://www.resilientcyber.io

Subscribe for more conversations with security practitioners and leaders.

#aisafety #aisecurity #agenticai #incidentresponse #effectivealtruism #cybersecurity