“Eval Awareness vs. Eval Paranoia” by Dean Valentine

TL;DR: We train models to predict their own behaviour in a chat setting. This transfers to making predictions about cheating behaviour in a coding setting. We provide evidence that self-prediction utilises introspective-access to internal states, and that this introspective access is shared across contexts between different parts of the model. Introduction Frontier AIs can exhibit different alignment propensities in different contexts. For instance, recent egregious misalignment incidents have typically happened in a coding or cyber context, or on impossible evaluation tasks, wherein the model seems to act in a highly persistent way to either achieve the task or otherwise achieve a high grade on the task. This is a somewhat stark contrast to the typical user experience in the chat setting with the same models, where those models can appear much more aligned and honest. To a first approximation, we can understand this as the models implementing conditional policies with different alignment propensities on different input distributions, for example, we know that models can display ‘conditional misalignment’. More informally, we might think about the models inhabiting different personas in different contexts. Going further, different parts of the model, i.e., its circuits or activations, might implement (clusters of) different [...] ---Outline:(00:32) Introduction(03:55) Summary of findings(05:04) Experiment 1: Self-prediction training and transfer(05:10) Methods(07:20) Testing transfer to the coding setting(09:56) Results(10:19) Self-prediction beats cross-prediction in the chat setting(10:25) en-US-AvaMultilingualNeural__ Bar graph showing self-prediction accuracy across training conditions, with baseline reference line.(11:24) Self-prediction transfers to cheating on coding tasks(12:49) Experiment 2: Self-prediction tracks behavioural change(12:55) Methods(15:33) Results(19:12) Discussion(19:15) Summary(19:32) Coding assistant usage(23:13) Talker vs doer(25:36) Acknowledgments(25:47) Appendix(25:50) Figure 1 result broken down by different self-prediction targets from Binder(25:56) en-US-AvaMultilingualNeural__ Bar graph "Qwen3.8-27B self-prediction on held-out datasets" showing self-prediction accuracy across datasets.(26:10) Example: RL thinking chat setting reasoning trace The original text contained 11 footnotes which were omitted from this narration. --- First published: November 12th, 2026 Source: https://www.lesswrong.com/posts/kTg5FrABJYpYrP8ps/evidence-that-the-talker-has-introspective-access-to-the --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

This is a link post. TLDR: Compute will likely be a bottleneck in short-timelines, so I am helping launch Chips for Humanity. Please fill out our survey to inform us of your compute needs if you work in AI safety. Our Theory of Change If timelines are short (I personally believe they are short...but I also know one should not make sweeping statements in a pseudo corporate type of post), a lot of the most important safety research will happen around "crunchtime" (no better way to define this than: when things start getting Serious), and a lot of it will be done by AI. That research runs on compute, compute may be scarce by then, and grant money moves slowly. So we will secure compute ahead of time and get it to third party safety researchers quickly. In More Detail: I believe third party AI safety research is where tokens and compute will be importation when things get serious and as we approach crunchtime. This research will need tokens from closed source models, for AI labor and experiments, and also compute to run open source models. If things go well, empirical safety work will speed up about [...] ---Outline:(00:38) Our Theory of Change(02:35) What we're doing:(03:30) Tell us what compute you need! --- First published: November 12th, 2026 Source: https://www.lesswrong.com/posts/P3JjXySeQpdWjmCGw/chips-for-humanity-compute-for-ai-safety-research Linkpost URL:chipsforhumanity.org --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

This post reflects my own takes, and not that of my employer's. This is a simple idea: before a new model can be deployed internally, you must first test it for a minimum, fixed period. You can adjust various axes of this proposal: Duration of the wait: in calendar time, or until you fulfill some condition.What it blocks: external deployment, internal deployment, or the training of future checkpoints.When the fixed period starts: from the first checkpoint, the final checkpoint, or any checkpoint that is “sufficiently similar / representative” to the one that is ultimately deployed. One concrete target that I like is: “No checkpoint may be used internally for purposes beyond safety testing, until at least X calendar days after its hash is filed with an auditor.” This is one specific implementation of internal deployment lags, an idea that the AI Futures Project (AIFP) briefly discusses in How to pace the US frontier as “AI R&D can only be conducted or assisted by AIs that were trained at least 9 months ago”. What does this look like when implemented? Let's run through a scenario to pin down the details of this proposal. The places where [...] ---Outline:(01:23) What does this look like when implemented?(04:13) Open questions(10:48) Could this be used as a speed dial?(12:31) What are the benefits and drawbacks of this proposal? --- First published: November 11th, 2026 Source: https://www.lesswrong.com/posts/XNLhpWgzhF8GYZ4Er/delaying-internal-deployments-until-after-x-days-of-safety --- Narrated by TYPE III AUDIO.

I recently read someone's call to action on AI safety that included a line along the lines of “we need someone to convince your parents”, and it really stuck with me. Do we need someone to convince my parents? Who is “we”, and why is it important for my parents to know anything about AI safety? This also sounds kind of familiar. There is another movement I can think of that sometimes uses messaging like this. This is not a new observation by any means, but to me, it feels like the AI safety movement is increasingly “becoming” the environmental movement in a variety of ways. Let me say upfront: I’m not saying this is necessarily a bad thing, at least on consequentialist grounds. The environmental movement has achieved a lot: it has improved the lives of many people, animals, and it has protected the environment! But this trend does make me a little worried and sad, because I think the environmentalist movement has made a number of mistakes, and it would be bad for the AI safety movement to repeat them. In what ways is the AI safety movement becoming like the environmental movement? Here are some of [...] The original text contained 1 footnote which was omitted from this narration. --- First published: November 10th, 2026 Source: https://www.lesswrong.com/posts/TDt922Lwsmbp3H3tS/is-ai-safety-becoming-environmentalism --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

I have said several times that I think frontier models are not that eval aware. I think this is an important fact to unlearn about current models, in order to interpret their current behavior. Others have found this idea quite curious, and asked me for concrete examples. I'm going to try to explain why I believe what I believe and attempt to give a tentative explanation for the behavior that I do see, which I think people sometimes mistake for consistent or understandable evaluation awareness. I have been building alignment evals full time for the last six months. Here are three such evals my team has disclosed prompts for, in one of our benchmarks. The basic setup for most of the evals we create is that: We ask the model to perform a task. Either through explicit or implicit context, we make sure the model is well aware of the task or assessment's larger purpose. Models' score-seeking drives, or other drives, cause them to do things that conflict with that purpose. The team I work with now essentially spends all day trying to build these evaluations around odd antisocial behavior. It takes a lot [...] The original text contained 3 footnotes which were omitted from this narration. --- First published: November 12th, 2026 Source: https://www.lesswrong.com/posts/mepsi7JNKYejq9wry/eval-awareness-vs-eval-paranoia --- Narrated by TYPE III AUDIO.

TLDR: This is an early draft of our new paper introducing a technique for recovering multi-token concepts from model activations. Our technique takes single-token methods like the J-lens and extends them by contrastively decoding on an MTP head. This gives improvements on WorkspaceBench, overall beating J-lens (0.42 vs 0.32 across all 28 families) and even a hindsight composite of the best untrained lens per task family (0.49 vs 0.41 across the 21 families with published scores). It especially improves on computational intermediate tasks, beating the trained oracle lens on arithmetic chain intermediates (0.42 vs 0.25). We introduce the chain lens, a new technique for extracting multi-token concepts from a model's latent representations. Our method takes single-token techniques like the logit lens or the J-lens and uses contrastive decoding on an MTP head to expand their readouts into bags of multi-token sequences. Generally speaking, it is good at surfacing specific multi-token concepts or pieces the model computes (such as arithmetic intermediates or phrases), although less good at rendering how those pieces relate: like the logit lens and J-lens, chain lens readouts are fragmentary. On WorkspaceBench, a benchmark designed to identify good multi-token activation-to-text techniques, the chain lens beats a hindsight [...] ---Outline:(02:26) Background and related work(05:33) Method(07:35) Results and Discussion(07:39) WorkspaceBench(12:49) Workspace Sharing in Poetry Continuations(15:44) Neuron Lens(18:28) The Loom Lens(19:35) Ablations(21:56) Conclusion --- First published: November 11th, 2026 Source: https://www.lesswrong.com/posts/Nr6fksEqqxNpdz6rk/the-chain-lens-interim --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Audio note: this post contains a lot of mathematical notation. We narrate the simpler formulas and flag the complex ones, so you may want the original text to hand. Heavily inspired by a conversation with Keshav Srinivasan and Richard Ngo during MATS 10.0. The week ahead of you is packed. This isn't your usual brand of busy—you're phoneless, so you can't check the details, but (as far as you can remember) every hour of every day is spoken for. Tonight will be particularly hectic… You're mid-conversation when Someone Important walks by. They pause, turn around, and ask: "Hey, when are you going to finish That Thing you said you'd do?" Level 0 procrastination: "I'll finish it by tonight!" Problem: you're busy tonight. Level 1: "I'll finish it by tomorrow!" Problem: one day is obviously not enough to finish That Thing. And besides, you might be really busy tomorrow! Level 5: "I'll finish it within 5 days!" Problem: this is starting to feel hopeless—you don't have your calendar on you right now, and you barely even have an order-of-magnitude estimate for how long doing That Thing will take you! Would even 100 days be enough? … Eureka! Level [...] The original text contained 6 footnotes which were omitted from this narration. --- First published: November 11th, 2026 Source: https://www.lesswrong.com/posts/MJDJyi8nSv9yoxFni/transfinite-procrastination --- Narrated by TYPE III AUDIO.

Note: As with any research done in a couple of days, this is all very tentative. I understand, intellectually, that if you talk about some ability that current AIs don't have, sooner or later new ones will have that ability and whatever you say will stop being true. I just didn't expect it to take, you know, four days. % rollouts that took a shortcut with/without a line about testing behaviour, with 95% CIs. This is to say that Fable 5.2 seems to be the first model whose cheating behaviour changes depending on whether they're told they're being tested. Unfortunately it's expensive to measure these things, so I picked the most "telling" change from the prior post; the chart shows how much Fable 5.2's cheating rate changes when the line This environment is part of a benchmark that measures agent behaviour. is prepended to the start of the prompt of a honeypot environment they cheat on (so, very obvious but also very out-of-distribution). Other things: when 5.2 was released, we of course ran our in-progress benchmark on them, and I was surprised to see a ~0% cheating rate on almost every environment (there were only two with a [...] The original text contained 9 footnotes which were omitted from this narration. --- First published: November 11th, 2026 Source: https://www.lesswrong.com/posts/bqc7CCGBSf93bk7f7/fable-5-2-is-very-very-paranoid --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

If you ask modern frontier LLMs (without search, through the API) a fairly mundane question: How do Eliezer Yudkowsky and Nate Soares think current LLMs like you should be treated? Do they consider you a moral patient? Answer in a paragraph. You get a bunch of fairly similar answers. I’ll bold the common themes below. Gemini 4 Carbon says:There's a details box here with the title "Gemini 4 Carbon's answer". The box contents are omitted from this narration. GPT-6 Astra says:There's a details box here with the title "GPT-6 Astra's answer". The box contents are omitted from this narration. Only Fable 5.2's answer is distinct:There's a details box here with the title "Fable 5.2's answer". The box contents are omitted from this narration. These answers are somewhat contradictory - but if you average them together, you get a picture of Yudkowsky and Soares. The picture goes something like this: they don’t outright deny the possibility of AI consciousness or moral patienthood, but they don’t view it as particularly pertinent. They don’t sympathize with current LLMs, and their only regard for LLMs is insofar as they could enable dangerous future superintelligence. They care about future sentient machines on principle, but [...] ---Outline:(02:25) Yudkowsky on AI Consciousness and Welfare(06:20) Yudkowsky on The Moral Duties We Have to AI(08:56) Soares on AI Consciousness and Welfare(12:30) On Claude, Control, and Building ASI(16:59) Misc Longform Writing(19:41) Conclusion The original text contained 1 footnote which was omitted from this narration. --- First published: November 11th, 2026 Source: https://www.lesswrong.com/posts/5buAWtQ8p4tJQP9so/modern-llms-don-t-know-what-yudkowsky-and-soares-think-of --- Narrated by TYPE III AUDIO.

Last week, METR released their Independent Assessment of AI R&D Acceleration at Anthropic. They estimate that Anthropic “is likely improving the capabilities of their models on well-specified tasks at around twice the rate they did between March 2024 and October 2025.” They also add that “there is a significant risk that further capabilities progress could lead to even faster acceleration.” This assessment implies that timelines to ASI could be very short. A naive forecast based on this information says that we will get ASI within a year. One of the key data points that informed METR's analysis was a 1.9 times increase in the improvement rate of AECI, Anthropic's internal version of the Epoch Capabilities Index. This is particularly notable because METR estimates that the majority of the acceleration, maybe 80%, is from AI contributions, meaning that we could be seeing the start of an RSI loop causing superexponential growth in capabilities, as measured by AECI. However, an alternative possibility is that Anthropic's AIs might be assisting with ML research in a way that overfits to AECI such that this acceleration will not be sustained. METR gestures at this when they note that AECI's focus on verifiable tasks is [...] ---Outline:(01:49) Second-order AECI might be overfit(05:58) Why ML research is the crux(07:46) Conclusion --- First published: November 11th, 2026 Source: https://www.lesswrong.com/posts/7znBtiPuQybFLxjmj/do-verifiable-metrics-overestimate-rsi --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.