Trust Me Maybe: Building Trust in Human-AI Partnerships in Security

In this episode, I speak with three guests from diverse backgrounds who share a common goal: Building trust in human-AI partnerships in security. We originally came together for a panel at the Institute of Electrical and Electronics Engineers (IEEE) Conference on AI in May 2025, and this episode recaps that discussion.

Key takeaways:

• Security practitioners tend to be natural-born skeptics (can you blame them?!). They struggle to trust and adopt AI-powered security products, especially in higher-risk scenarios with overly simplified decision-making processes.
• AI can be a tool for threat actors and a threat vector itself, and its non-deterministic nature makes it unpredictable and vulnerable to manipulation.
• All AI models are biased, but not all bias is negative. Recognized and carefully managed bias can provide actionable insights. Purposefully biased (opinionated) models should be transparent.
• Clearer standards and expectations are needed for “human-in-the-loop” and human oversight. What does the human actually do, are they qualified, and do they have the right experience and information?
• What happens when today’s graduates are tomorrow’s security practitioners? On one end of the spectrum we have a lot of skepticism, on the other end not enough. We talk about over-reliance on AI, de-skilling, and loss of situational awareness.

Dr. Margaret Cunningham is the Technical Director, Security & AI Strategy at Darktrace. Margaret was formerly Principal Product Manager at Forcepoint and Senior Staff Behavioral Engineer at Robinhood.

Dr. Divya Ramjee is an Assistant Professor at Rochester Institute of Technology (RIT). She also leads RIT’s Technology and Policy Lab, analyzing security, AI policy, and privacy challenges. She previously held senior roles in US government across various agencies.

Dr. Matthew Canham is the Executive Director, Cognitive Security Institute. He is a former FBI Supervisory Special Agent, with over twenty years of research in cognitive security.