The Domestic Yak podcast

Constitutional Classifiers: A New Bulletproof Defense Against Universal Jailbreaks

0:00
21:06
Rewind 15 seconds
Fast Forward 15 seconds

This episode summarizes Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teaming by Mrinank Sharma Et. al.

Submitted on 31 Jan 2025 https://arxiv.org/abs/2501.18837

This research paper explores methods for safeguarding large language models (LLMs) against "jailbreaks"—techniques that circumvent safety measures to elicit harmful information. The authors introduce "Constitutional Classifiers," a system using classifiers trained on synthetic data generated from natural language rules, to detect and block harmful content. Extensive red teaming, involving over 3,000 hours of human effort, demonstrated the classifiers' effectiveness in preventing universal jailbreaks.

More episodes from "The Domestic Yak"