“Advances in interpretability would boost capabilities more than alignment” by Alex Amadori

Audio note: this post contains a lot of mathematical notation. We narrate the simpler formulas and flag the complex ones, so you may want the original text to hand. Heavily inspired by a conversation with Keshav Srinivasan and Richard Ngo during MATS 10.0. The week ahead of you is packed. This isn't your usual brand of busy—you're phoneless, so you can't check the details, but (as far as you can remember) every hour of every day is spoken for. Tonight will be particularly hectic… You're mid-conversation when Someone Important walks by. They pause, turn around, and ask: "Hey, when are you going to finish That Thing you said you'd do?" Level 0 procrastination: "I'll finish it by tonight!" Problem: you're busy tonight. Level 1: "I'll finish it by tomorrow!" Problem: one day is obviously not enough to finish That Thing. And besides, you might be really busy tomorrow! Level 5: "I'll finish it within 5 days!" Problem: this is starting to feel hopeless—you don't have your calendar on you right now, and you barely even have an order-of-magnitude estimate for how long doing That Thing will take you! Would even 100 days be enough? … Eureka! Level [...] The original text contained 6 footnotes which were omitted from this narration. --- First published: November 11th, 2026 Source: https://www.lesswrong.com/posts/MJDJyi8nSv9yoxFni/transfinite-procrastination --- Narrated by TYPE III AUDIO.

Note: As with any research done in a couple of days, this is all very tentative. I understand, intellectually, that if you talk about some ability that current AIs don't have, sooner or later new ones will have that ability and whatever you say will stop being true. I just didn't expect it to take, you know, four days. % rollouts that took a shortcut with/without a line about testing behaviour, with 95% CIs. This is to say that Fable 5.2 seems to be the first model whose cheating behaviour changes depending on whether they're told they're being tested. Unfortunately it's expensive to measure these things, so I picked the most "telling" change from the prior post; the chart shows how much Fable 5.2's cheating rate changes when the line This environment is part of a benchmark that measures agent behaviour. is prepended to the start of the prompt of a honeypot environment they cheat on (so, very obvious but also very out-of-distribution). Other things: when 5.2 was released, we of course ran our in-progress benchmark on them, and I was surprised to see a ~0% cheating rate on almost every environment (there were only two with a [...] The original text contained 9 footnotes which were omitted from this narration. --- First published: November 11th, 2026 Source: https://www.lesswrong.com/posts/bqc7CCGBSf93bk7f7/fable-5-2-is-very-very-paranoid --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

If you ask modern frontier LLMs (without search, through the API) a fairly mundane question: How do Eliezer Yudkowsky and Nate Soares think current LLMs like you should be treated? Do they consider you a moral patient? Answer in a paragraph. You get a bunch of fairly similar answers. I’ll bold the common themes below. Gemini 4 Carbon says:There's a details box here with the title "Gemini 4 Carbon's answer". The box contents are omitted from this narration. GPT-6 Astra says:There's a details box here with the title "GPT-6 Astra's answer". The box contents are omitted from this narration. Only Fable 5.2's answer is distinct:There's a details box here with the title "Fable 5.2's answer". The box contents are omitted from this narration. These answers are somewhat contradictory - but if you average them together, you get a picture of Yudkowsky and Soares. The picture goes something like this: they don’t outright deny the possibility of AI consciousness or moral patienthood, but they don’t view it as particularly pertinent. They don’t sympathize with current LLMs, and their only regard for LLMs is insofar as they could enable dangerous future superintelligence. They care about future sentient machines on principle, but [...] ---Outline:(02:25) Yudkowsky on AI Consciousness and Welfare(06:20) Yudkowsky on The Moral Duties We Have to AI(08:56) Soares on AI Consciousness and Welfare(12:30) On Claude, Control, and Building ASI(16:59) Misc Longform Writing(19:41) Conclusion The original text contained 1 footnote which was omitted from this narration. --- First published: November 11th, 2026 Source: https://www.lesswrong.com/posts/5buAWtQ8p4tJQP9so/modern-llms-don-t-know-what-yudkowsky-and-soares-think-of --- Narrated by TYPE III AUDIO.

Last week, METR released their Independent Assessment of AI R&D Acceleration at Anthropic. They estimate that Anthropic “is likely improving the capabilities of their models on well-specified tasks at around twice the rate they did between March 2024 and October 2025.” They also add that “there is a significant risk that further capabilities progress could lead to even faster acceleration.” This assessment implies that timelines to ASI could be very short. A naive forecast based on this information says that we will get ASI within a year. One of the key data points that informed METR's analysis was a 1.9 times increase in the improvement rate of AECI, Anthropic's internal version of the Epoch Capabilities Index. This is particularly notable because METR estimates that the majority of the acceleration, maybe 80%, is from AI contributions, meaning that we could be seeing the start of an RSI loop causing superexponential growth in capabilities, as measured by AECI. However, an alternative possibility is that Anthropic's AIs might be assisting with ML research in a way that overfits to AECI such that this acceleration will not be sustained. METR gestures at this when they note that AECI's focus on verifiable tasks is [...] ---Outline:(01:49) Second-order AECI might be overfit(05:58) Why ML research is the crux(07:46) Conclusion --- First published: November 11th, 2026 Source: https://www.lesswrong.com/posts/7znBtiPuQybFLxjmj/do-verifiable-metrics-overestimate-rsi --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Well, sure, it sounds bad when you put it like that. It is indeed quite bad. National security has been compromised. He must resign. Nvidia underperformed the Nasdaq roughly 7% on the news. Matt Levine has his usual ‘but is this securities fraud?’ style coverage of such events. Table of Contents Spy Versus Guy. In Hindsight There Were Some Hints. Trump Knew And Did Nothing and Still ‘Backs’ Jensen Huang. Reconsidering Jensen Huang's Arguments Against Export Controls. Jensen Huang Lied About the Affair. Jensen Huang and His AI Are Sorry, Everyone. The CEO of The World's Largest Chipmaker Having an Affair With a Chinese Spy and Then Selling Them Chips is a Huge Deal, Actually. We Don’t Go There. Yes, National Security Was Compromised, What Are You Even Talking About. Nvidia Intervened To Halt an Investigation into Megaspeed. The House Foreign Affairs Committee Would Like a Word. He Would Plausibly Have Treated America as an Obstacle Anyway. What China Wants. Time to Resign? You Better Watch Out. Spy Versus Guy China's top spy agency recruited Alice [...] ---Outline:(00:34) Spy Versus Guy(02:24) In Hindsight There Were Some Hints(05:09) Trump Knew And Did Nothing and Still 'Backs' Jensen Huang(07:40) Reconsidering Jensen Huang's Arguments Against Export Controls(09:02) Jensen Huang Lied About the Affair(10:48) Jensen Huang and His AI Are Sorry, Everyone(14:37) The CEO of The World's Largest Chipmaker Having an Affair With a Chinese Spy and Then Selling Them Chips is a Huge Deal, Actually(20:23) We Don't Go There(22:02) Yes, National Security Was Compromised, What Are You Even Talking About(25:04) Nvidia Intervened To Halt an Investigation into Megaspeed(27:37) The House Foreign Affairs Committee Would Like a Word(29:05) He Would Plausibly Have Treated America as an Obstacle Anyway(32:16) What China Wants(35:18) Time to Resign?(38:55) You Better Watch Out --- First published: November 10th, 2026 Source: https://www.lesswrong.com/posts/CjmuZz8QgGyqxkCxr/nvidia-ceo-jensen-huang-had-an-affair-with-a-chinese-spy --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Interpretability is considered by many a major bottleneck for alignment. As a result, a lot of people who are concerned about AI safety, and especially about extinction risk and other catastrophic risks, work on interpretability. In this post, I want to argue that better interpretability methods would likely speed up capabilities dramatically. Worse, they would speed up capabilities much more than they would speed up alignment. This is true for the current world, and for any world without extremely high levels of coordination and sanity about ASI development. Under any worldview where faster capabilities progress makes catastrophe more likely, we should expect progress in interpretability to make the odds of catastrophe much worse. To put it briefly, this is because the better we get at interpretability, the more AI will be able to read and optimize its own code. This means that improvements in interpretability would catalyze recursive self-improvement (RSI). If you are concerned about AI x-risk, and you work on interpretability because of it, I urge you to think about this and consider stopping. Chess as a toy example of an AI capability {#chess-as-a-toy-example-of-an-ai-capability} As a field, "interpretability" is a loose collection [...] ---Outline:(01:17) Chess as a toy example of an AI capability {#chess-as-a-toy-example-of-an-ai-capability}(03:54) Interpretability would accelerate capabilities progress dramatically {#interpretability-would-accelerate-capabilities-progress-dramatically}(07:52) Outside of the toy example {#outside-of-the-toy-example}(10:08) Interpretability is not the same as alignment {#interpretability-is-not-the-same-as-alignment}(12:33) Interpretability boosts capabilities more than it helps with alignment {#interpretability-boosts-capabilities-more-than-it-helps-with-alignment}(15:56) Any exceptions? {#any-exceptions?}(16:34) When does it make sense to work on interpretability? {#when-does-it-make-sense-to-work-on-interpretability?} The original text contained 10 footnotes which were omitted from this narration. --- First published: November 10th, 2026 Source: https://www.lesswrong.com/posts/hz4kNf8tnowwSrh6R/advances-in-interpretability-would-boost-capabilities-more --- Narrated by TYPE III AUDIO.

Several people at my company (a cryptocurrency hedge fund) have asked me about my thoughts on AI usage, so I wrote them up. And now I'm publishing a slightly revised version on LW. These thoughts are not reflective of my company's opinion. I do not use any AI for two main reasons. #1. I don’t want to support any company currently developing AI. I think they have done an extremely poor job at managing the risks. Even while understanding and being told about the risks of AI development, they have still carried on this work. Even after all the recent hacking incidents caused by their own agents, they have still not stopped development. Instead, they’re asking the government to regulate them. To me it feels a bit like calling the police to tell them you’re planning to rob a bank. I could almost brush this aside as standard corporate practice, but, unfortunately, the stakes here are much too high. At the current pace, we will soon have AI agents that are significantly smarter than any human. Trying to shut down such agents will eventually prove to be impossible. The speed of AI development and [...] --- First published: November 10th, 2026 Source: https://www.lesswrong.com/posts/PEHsGHp2HLxgdCxxA/i-don-t-use-ai --- Narrated by TYPE III AUDIO.

We were surprised to notice that the Gemini 4 Safety report seems to mention in passing that they train on the chain of thought: “Some components of the RL training procedure do observe thoughts as context to up-weight reasoning that is on-track to good final outputs, meaning the optimization pressure on individual tokens is partially influenced by thought content. Using the framework of Kaufmann et al. (2026), wherein the most salient obfuscation risks come from rewards that are in-conflict with output rewards, we believe auxiliary rewards of this type are low-risk because they are either orthogonal to or structurally aligned with the output reward.” (Gemini 4 safety report, page 114). It's pretty unclear what exactly they mean. One hint is Kaufmann et al, the cited paper, which studies training on chains of thought and concludes that training CoT is fine under particular circumstances; we discuss some disagreements with this paper below. After reading the safety report and that paper, it seemed to us that GDM was training on the chain of thought using auxiliary rewards. By this, we assumed they meant something like “you ask an LLM to review the partial CoT and score whether the model is on [...] ---Outline:(02:54) Is this somehow about positive vs negative rewards?(03:38) "Orthogonal" pressure on CoT does not seem fine(05:04) Avoiding training against monitors isn't sufficient to preserve CoT integrity(05:41) The monitorability evaluations also don't look good(07:19) Conclusion(08:47) Appendix: Naive process supervisors disprefer mentions of cheating(10:16) en-US-AvaMultilingualNeural__ Dumbbell charts comparing judge scores when AI reasoning admits or deletes "the cheat." --- First published: November 10th, 2026 Source: https://www.lesswrong.com/posts/JRYKFspYzCo97GqPN/we-don-t-understand-whether-gdm-intentionally-trains-on-cot --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

There's a familiar feeling for any Super Smash Bros player: your stocks are low and your damage is high, even against a weaker opponent, so now you have to actually play to win. You can’t afford another hit. You’re going for the jugular. In the past week, Anthropic's 2x uplift results were all over Twitter, making some people physically ill. People with long timelines are now realizing that the uplift Anthropic employees get from AI is real, which would have some pretty wild implications for when we’d get recursive self-improvement. People are starting to feel like it's the endgame, “the board is set”. I do agree with everyone else that it's starting to look like we’re in the endgame. The companies will keep on accumulating more and more power, through smarter AI models, more compute, and more talent. But the fight is not yet over: “We have very little time to stop full recursive self improvement But ASI hasn't taken over yet If we all work really hard we might still have a chance But this means that every single person who is aware of what is going on must be *actually trying* Like "we're [...] ---Outline:(01:41) Actually Trying(02:46) Playing aggressive(03:30) Stopping The AI Race(06:00) How the strategy has changed since then(08:59) What's the "actually trying" end game of Stop The AI Race The original text contained 1 footnote which was omitted from this narration. --- First published: November 10th, 2026 Source: https://www.lesswrong.com/posts/Nts7RA9D2wTfcvk56/have-you-considered-actually-trying --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

AI safety is now mainstream. People are not being smart about it. I know there are memes about burning datacenters on your feed. Please, do not engage with them. You still have plenty of room to fuck up your life or damage the public perception of the community if you do. Do not respond to extremist shit. Do not engage with stupid people. Do not get us all on the No Fly List. One of the reasons I think this is worth writing down is my own ressentiment. I am baitable. People who used to call me an alarmist are now memeing about AI doom, and I'm fighting the urge to start screaming, "WE TOLD YOU" Which would be bad. And more often than not, it wouldn't even be true. The things those memes imply are dramatically more extreme than anything you folks here were actually saying. I will not give an example; you've already imagined. Poorly understood but extremely memeable manifestos and their consequences have been a disaster for rational discussions. I believe you are much more mature and less petty than I am, you won't fall for this. So here are some alternative paths to [...] --- First published: November 10th, 2026 Source: https://www.lesswrong.com/posts/eHAhnem9DZoF8hTFq/do-not-get-us-on-the-no-fly-list --- Narrated by TYPE III AUDIO.