LessWrong (30+ Karma) podcast

“Homeopathic Learning: Diluting Poisoned Fine-Tuning Data Can Strengthen Its Effect” by mclarke, Jan Dubiński, Owain_Evans, cloud

0:00
12:31
Spol 15 sekunder tilbage
Spol 15 sekunder frem

Authors: Matt Clarke*, Jan Dubiński*, James Chua, Simon Schrodi, Owain Evans, Alex Cloud (*- Core contribution)

tl;dr. We find that poisoned training data with little effect on its own can create a hidden backdoor once clean, contrasting data is added. We demonstrate this effect in small models fine-tuned on datasets we constructed.

📄Paper, 💻Code, 🐦Twitter

Work done as part of the Anthropic Fellows Program, in collaboration with Truthful AI. This article is cross-posted from the Anthropic Alignment Science Blog.

Paper abstract

We study homeopathic learning, a phenomenon where poisoned training data with little effect on its own can create a hidden backdoor once clean, contrasting data is added. In our experiments, a model fine-tuned only on a “poison” dataset associated with a trait T (such as speaking German or being misaligned) does not learn T. Surprisingly, adding similarly formatted “clean” data not associated with T causes the model to learn T. This effect requires that the poison data have an identifying “trigger” feature that is absent from the clean data. We demonstrate this effect in a variety of settings: subliminal learning of backdoor behaviors, inoculation prompting of reward hacking transcripts, generalization from archaic bird names, and an MNIST classifier [...]

---

Outline:

(01:06) Paper abstract

(02:27) Introduction

(03:42) Backdoor transfer in subliminal learning

(04:59) Generalization from narrow data to broad traits

(06:48) What kind of clean data is needed for homeopathic learning?

(07:32) Homeopathic learning as a general phenomenon

(08:58) A regression model of homeopathic learning

(10:19) Discussion

---

First published:
November 14th, 2026

Source:
https://www.lesswrong.com/posts/5CrmJfRY7yMaAoumC/homeopathic-learning-diluting-poisoned-fine-tuning-data-can-1

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Flere episoder fra "LessWrong (30+ Karma)"