Bounce! Conversations with Larry Weeks podcast

Beyond AI Doom: Zack Korman on the Case for Cautious Acceleration

0:00
50:09
Rewind 15 seconds
Fast Forward 15 seconds

Is AI doom a prophecy or a set of choices we can make right now? What happens if the people building AI start believing it has a soul? And if AI is risky, why might the doomers be wrong about how to respond?

In this episode, Zack Korman explains why AI doom isn't inevitable and why the real risks have less to do with machines plotting against us and more to do with what access we give them and whether anyone is watching.

Zack is the co-founder of Embroidery.io, a cybersecurity company that uses AI to monitor AI agents. He was previously CTO of the cybersecurity company Pistachio. Based in Norway, Zack has become an outspoken voice in the AI safety debate, including media appearances in The Atlantic, Time magazine, and The Wall Street Journal.

In this conversation, we explore how AI agents actually work, what really happened in the Hugging Face incident, the divide between doomers and accelerationists, why believing AI is alive could make it harder to shut down, why Zack treats AI models as extraordinarily valuable malware, how to regulate the way AI is deployed rather than just the labs that build it, how to protect yourself in the age of agents, and how becoming a father changed the way he thinks about the future.

One idea kept resurfacing: whether AI goes well or badly may depend less on what the machines want and more on what we build, what we connect them to, and whether anyone is watching

Some highlights from the episode:

  • Between extraordinary promise and existential fear
  • Doomers, accelerationists, and centrists: The competing philosophies of AI
  • Can we move fast without breaking things?
  • The asymmetry of AI risks and rewards
  • How language models become agents that can take real-world actions
  • Why even a well-aligned AI model can't be perfectly safe
  • The problem of persistence 
  • Kill switches, sandbox escapes, and human oversight
  • AI consciousness, and the effects of anthropomorphism
  • What effective AI regulation might actually look like
  • Interpretability: Understanding a model's choices
  • What AI doomers get right
  • Practical advice for agentic AI risks
  • Potential upsides of AI
  • A new father's perspective on AI risk

Enjoy!

For show notes and more, visit larryweeks.com

 

More episodes from "Bounce! Conversations with Larry Weeks"