
“Gradient Routing Creates a Headstart” by TomTaylor, emmakrentz, Emma Rafkin, John Lund, Mike Vaiana, Diogo de Lucena
Tl;dr
GRAFT showed that we can make reward hacking removable. In the setup we study, a small LLM has two trainable adapters, each consisting of an additional MLP in every layer. Training updates from detected hacks are routed only into one, called FORGET, which is removed ('forgotten') after training to suppress the behaviour. We reproduced a small-model version of GRAFT and tested what makes this isolation work, where hacking concentrates across layers, and the effect of adapter width. In order to conclude that these results are general, we’d need to run larger models in different settings.
- Without routing, reward hacking did not reliably concentrate in a single adapter. This suggests GRAFT helps create the concentration, rather than simply determining which adapter gets the hack.
- Without routing, making both adapters wider made their individual hack rates more similar. This is consistent with a smaller random headstart for either adapter arising from its initial weights.
- Learning speed, not size, decides where hacking concentrates. Whichever adapter learns faster is more likely to isolate hacking.
- GRAFT packs the hack into FORGET's later layers. Removing 2 layers takes out half of it, compared to removing ~10 layers without GRAFT.
- Ablating all of FORGET [...]
---
Outline:
(00:12) Tl;dr
(02:01) Introduction
(02:32) Experimental setup
(04:50) Concentration without routing
(06:09) Adapter width
(07:40) Learning rate and adapter size
(08:56) Layer ablations
(10:29) Conclusion
The original text contained 12 footnotes which were omitted from this narration.
---
First published:
November 16th, 2026
Source:
https://www.lesswrong.com/posts/e3CKvARMn9ra8Ba6M/gradient-routing-creates-a-headstart
---
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
Weitere Episoden von „LessWrong (30+ Karma)“



Verpasse keine Episode von “LessWrong (30+ Karma)” und abonniere ihn in der kostenlosen GetPodcast App.








