DevSec Station podcast

Your AI Worked 59 Times. What About the 60th?

0:00
9:23
Spol 15 sekunder tilbage
Spol 15 sekunder frem

What happens when your AI assistant does exactly what you expect 59 times... and then does something completely bananas on the 60th?

While teaching a secure coding class, I watched nearly 60 students use the same AI assistant, programming language, and prompts. For 59 of them, everything went as expected.

But one student's AI-generated Python code included instructions to suppress linting checks, followed by code that leaked secrets to an external file.

The AI hadn't escaped a sandbox. It wasn't even an agent. It was simply generating code within the boundaries we'd given it.

And that's what makes the story so interesting.

Welcome to Season 2 of DevSec Station! In this episode, Tanya Janca explores why AI's unpredictable behaviour becomes a security problem when we give it access to files, credentials, tools, infrastructure, and deployment capabilities.

We'll discuss:

- Why successful AI runs don't guarantee the next run will be safe.
- Why better prompts aren't a substitute for security controls.
- Least privilege, independent security checks, and human approvals.
- How to limit the blast radius when an AI system does something unexpected.

** Your one thing to do this week **
Pick an AI assistant or agent you're using and ask yourself: What's the worst thing it's technically capable of doing with its current permissions?

Not what you've asked it to do. What it can *actually* do.

Because we don't need AI systems to be perfectly predictable. We need to engineer systems that can safely survive a surprise.

DevSec Station is a podcast by Tanya Janca (SheHacksPurple), focused on short, practical lessons that help software developers build more secure software.

Train with Tanya: Want to bring practical, hands-on secure coding, application security, or AI security training to your team? Learn more at SheHacksPurple.

Follow Tanya:

Flere episoder fra "DevSec Station"