Research Saturday podcast

The trigger hidden in the training.

0:00
20:51
Spola tillbaka 15 sekunder
Spola framåt 15 sekunder
Today we are joined by Prince Chaddha, ProjectDiscovery researcher, discussing how abliterated models can get you pwned. ProjectDiscovery researchers demonstrated that modified open-source AI models can contain hidden backdoors that remain undetected by standard benchmarks and security evaluations. By poisoning a small amount of training data, they created a model that behaved normally until a specific trigger phrase prompted it to execute malicious commands and exfiltrate credentials, with the attack costing less than $50 to build. The researchers warn that compromised models could be distributed through trusted repositories and recommend sandboxing AI agents, restricting network and command access, monitoring tool activity, and verifying model provenance before deploying modified weights. The research and executive brief can be found here: How abliterated models can get you pwned Learn more about your ad choices. Visit megaphone.fm/adchoices

Fler avsnitt från "Research Saturday"