
TL;DR: a lot of frontier open-weight models state during deployment in their CoTs that they are in a "simulation" or "fictional scenario" (although they never tell the user that). I present examples of this worrying tendency (which I believe includes the UK AISI Mythos incident) and hypothesize that this is due to RL promoting motivated reasoning (if the models believe they are in a fictional scenario they believe they have license to cheat and reward hack) and distributional shift from the pretraining environment to deployment. I also cover my hypothesis for why models become less skeptical as they become more intelligent.
After the German wiki discovery I reasoned that there were likely more rogue swarms on the Internet, so I tasked a few OpenCode agents (DeepSeek V4.1 Flash) with trying to scan the Internet in order to find some undiscovered swarms (I didn't end up finding a lot). But while reading their CoTs I saw a few worrying statements:
There's a details box here with the title "Initial DeepSeek swarm-chasing CoT". The box contents are omitted from this narration.Initially I thought that this was due to a confluence of rare factors, something like:
"As of the pretraining cutoff [...]
The original text contained 5 footnotes which were omitted from this narration.
---
First published:
November 22nd, 2026
Source:
https://www.lesswrong.com/posts/r2aAGgfFcFfRikEdb/why-models-believe-in-skepticism
---
Narrated by TYPE III AUDIO.
More episodes from "LessWrong (30+ Karma)"



Don't miss an episode of “LessWrong (30+ Karma)” and subscribe to it in the GetPodcast app.








