LessWrong (30+ Karma) podcast

“Evidence that the ‘talker’ has introspective access to the ‘doer’” by Francis Rhys Ward

0:00
26:47
15 Sekunden vorwärts
15 Sekunden vorwärts

TL;DR: We train models to predict their own behaviour in a chat setting. This transfers to making predictions about cheating behaviour in a coding setting. We provide evidence that self-prediction utilises introspective-access to internal states, and that this introspective access is shared across contexts between different parts of the model.

Introduction

Frontier AIs can exhibit different alignment propensities in different contexts. For instance, recent egregious misalignment incidents have typically happened in a coding or cyber context, or on impossible evaluation tasks, wherein the model seems to act in a highly persistent way to either achieve the task or otherwise achieve a high grade on the task. This is a somewhat stark contrast to the typical user experience in the chat setting with the same models, where those models can appear much more aligned and honest.

To a first approximation, we can understand this as the models implementing conditional policies with different alignment propensities on different input distributions, for example, we know that models can display ‘conditional misalignment’. More informally, we might think about the models inhabiting different personas in different contexts. Going further, different parts of the model, i.e., its circuits or activations, might implement (clusters of) different [...]

---

Outline:

(00:32) Introduction

(03:55) Summary of findings

(05:04) Experiment 1: Self-prediction training and transfer

(05:10) Methods

(07:20) Testing transfer to the coding setting

(09:56) Results

(10:19) Self-prediction beats cross-prediction in the chat setting

(10:25) en-US-AvaMultilingualNeural__ Bar graph showing self-prediction accuracy across training conditions, with baseline reference line.

(11:24) Self-prediction transfers to cheating on coding tasks

(12:49) Experiment 2: Self-prediction tracks behavioural change

(12:55) Methods

(15:33) Results

(19:12) Discussion

(19:15) Summary

(19:32) Coding assistant usage

(23:13) Talker vs doer

(25:36) Acknowledgments

(25:47) Appendix

(25:50) Figure 1 result broken down by different self-prediction targets from Binder

(25:56) en-US-AvaMultilingualNeural__ Bar graph "Qwen3.8-27B self-prediction on held-out datasets" showing self-prediction accuracy across datasets.

(26:10) Example: RL thinking chat setting reasoning trace

The original text contained 11 footnotes which were omitted from this narration.

---

First published:
November 12th, 2026

Source:
https://www.lesswrong.com/posts/kTg5FrABJYpYrP8ps/evidence-that-the-talker-has-introspective-access-to-the

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Weitere Episoden von „LessWrong (30+ Karma)“