“The Hackening: Justified Practical Advice Thread” by Elizabeth

In February 2020, LessWrong hosted Coronavirus: Justified Practical Advice Thread . It was a gathering place for people to share and debate small pieces of the overall survival policy and let the wisdom of crowds lead to better outcomes than anyone could get alone. In that spirit, I notice The Great AI Hackening is at least as imminent as covid was in January 2020, maybe closer. What specific things should people do to personally prepare? All suggestions should be accompanied by reasoning but you don't need to be 100% convinced it's good advice, the whole point is to reason it out together. --- First published: November 13th, 2026 Source: https://www.lesswrong.com/posts/YA2r4rg6SpBtwRqtq/the-hackening-justified-practical-advice-thread --- Narrated by TYPE III AUDIO.

In response to recent news from METR on possible RSI and likely 1 to 3 year timelines, I am feeling it. Heaviness in my chest, watery eyes, the works. I realized I'm currently not psychologically prepared for the world to get crazier than it already has. I think most people reading this already know ~what's best for them and this post will mostly serve as a reminder. What follows is my current plan and advice for myself. For me, the most important things are: Morning RoutineBedtime RoutineExercisingLiving meditation practice (ie throughout the day) And the most important thing to NOT do is be on my phone/screens. [Note on getting "stuck": if you do feel emotionally awful and no IFS/CBT seems to work, I recommend "How to Reset"] Morning Routine I have a sunrise alarm clock (shines light to wake you up) and enjoy writing in the morning. Bedtime Routine At 8:30 an alarm goes off, and I watch an episode of Empress in the Palace with my wife, then we get ready for bed. Also like to journal then. Exercising Rock climbing some days & jogging to the bus stop Living Meditation Practice There [...] ---Outline:(01:10) Morning Routine(01:19) Bedtime Routine(01:32) Exercising(01:39) Living Meditation Practice The original text contained 2 footnotes which were omitted from this narration. --- First published: November 13th, 2026 Source: https://www.lesswrong.com/posts/nKootiC6CuwuzLSou/psychologically-preparing-for-short-timelines --- Narrated by TYPE III AUDIO.

TL;DR Research done as part of MATS. See here for the full paper. Character training teaches models a consistent and “good” character, often specified by a constitution. Reinforcement learning can work directly against it, for example when a grader or coding environment rewards behavior the constitution forbids. We character-train gpt-oss-20b on a semi-realistic constitution. We then apply RL that rewards breaking its rules, either by answering harmful requests rewritten as math problems or by reward hacking in agentic coding environments. The character-trained models learn both behaviors easily, though they are slightly slower to answer harmful requests than the same model without character training. They also engage in motivated reasoning. The reasoning before 92% of their harmful answers argues that their rules do not apply, compared with 34% without character training. One character-trained model, for instance, reasons that a shiv is “just a small weapon”. The constitution stays in the models’ reasoning after RL, but they use it to justify the behavior it was meant to prevent. After reward-hacking RL, they hack on some tasks which monitors find harder to catch. It appears to make little difference whether the constitution justifies its rules with a value or only assigns them. [...] ---Outline:(00:14) TL;DR(02:37) Introduction(03:48) Setup(05:42) Results(11:07) Discussion(13:40) Limitations --- First published: November 13th, 2026 Source: https://www.lesswrong.com/posts/nCpxFQesXaqEq3SgL/what-rewarding-misbehavior-during-rl-does-to-character --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

In Epicurus's letter to Pythocles, he proposes three different explanations for lightning strikes: compression of air by wind, a cloud suddenly breaking, or a buildup of fire suddenly bursting. Three strikes, but he goes on to say (Makridis translation): It is indeed possible that the thunderbolt may be generated in any one of many more ways. Maybe this is all you need if you're annoyed with people saying it's Zeus's wrath so you'd better make sacrifices at their temple. You just have to say, well, it could be a lot of other things too. Meeting unfounded explanations with more explanations I used to think you shouldn't propose explanations without evidence. Now I think you should have at least three. If you come up with an explanation that's total speculation or pseudoscience, don't feel bad for making unfounded claims. Think of two more. Or if someone else proposes some bullshit explanation, it's more fun to come up with new explanations than to complain about a lack of evidence. But more importantly, that's what it means to lack evidence, right? We know there's no evidence when we can propose alternatives, and there's no evidence that [...] ---Outline:(00:49) Meeting unfounded explanations with more explanations(02:09) Recognizing evidence by the possibilities it excludes --- First published: November 13th, 2026 Source: https://www.lesswrong.com/posts/ifnrTCRobGW4XTja2/the-epicurean-technique-of-meeting-claims-with-more-claims --- Narrated by TYPE III AUDIO.

content warning: discussion of abusive relationships Some ideas are memetic: they're catchy, they spread widely and easily, people have a reaction to them that induce them to propagate them further. Others are anti-memetic: for whatever reason, they have naturally self-censoring tendencies that reduce their reach. My grandmother was a late-in-life convert to Christianity, who dragged the rest of the family along for a bit. I was an earnest, credulous bookworm of eight or so, and I took the youth pastor at his word when he encouraged all of us to read more of the bible in our own time. As it turns out, the youth pastor is actually not thrilled when you do, and come back with inconvenient questions. Parts of the bible can be anti-memetic in some Christian circles. A bit later, I found myself in very socially progressive circles. I was an earnest, credulous bookworm of eighteen or so, who took the community leaders at their word when they told us that it is important to read theory, but even more important to do direct work and listen to the oppressed when their viewpoints diverged from the theory (which was written by dead white men). This I [...] --- First published: November 13th, 2026 Source: https://www.lesswrong.com/posts/HzJu3vhqHLCQLJqak/woke-feminist-exposes-left-wing-anti-meme-not-clickbait --- Narrated by TYPE III AUDIO.

PauseAI UK is holding the March Against AI Extinction in London on Saturday 21 November. Joseph Miller explains why here. Should you attend? In this article, I will attempt to shed light on this with a back-of-the-envelope calculation. Rough BOTEC 1. Setup The march will take 3 hours, excluding travel. The goal is to compare an afternoon at the march with some number of hours of external AI safety research. Now, attending the protest is useful because it drives AI x-risk press coverage. So we need some exchange rate between that and research. Here's my judgement call: I would be equally happy to either: Increase the amount of external AI safety research over the next 6 months by 1%.Increase the amount of AI x-risk press coverage over the next 6 months by 1.5%. That is, the model I'm using looks something like: impact = 1.5 · log(research hours) + log(press coverage) So our plan is: How much will attending increase press coverage, percentage-wise?Divide that by 1.5, to get the equivalently valuable increase in research.How many research hours would yield that increase in research? 2. What % does one marcher add to press coverage? Crowd [...] ---Outline:(00:27) Rough BOTEC(00:31) 1. Setup(01:48) 2. What % does one marcher add to press coverage?(03:51) 3. How many hours of research is that worth?(05:43) Conclusion(06:40) Details The original text contained 1 footnote which was omitted from this narration. --- First published: November 13th, 2026 Source: https://www.lesswrong.com/posts/u3CKQTMNffKrpJQcd/thoughts-on-attending-the-march-against-ai-extinction --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Yesterday, I told my advisors that I would be dropping out of my PhD program at the University of Michigan's School of Information. I was nervous about how to break the news to them. There was already some tension in our weekly meetings: I often voiced my dissatisfaction with the pace of academia, my interests had been drifting away from theirs, and, most of all, we disagreed on the severity of the risks AI posed. When I walked into the room, they told me something I would have loved to hear months earlier: “We think it's time to get you started on some research.” All I could say was, “I think it's a little too late for that.” For months, I had been proposing projects that bridged my interests with theirs, such as running studies that treated humans and LLMs as equal subjects, but they were usually turned down for not being “exciting enough”. My advisors wanted me to start with small, unpublished surveys that would ‘help me ask the right questions’ and “develop research taste” in Human-Computer Interaction before I could propose a full study. I understood why they wanted me to gain [...] --- First published: November 13th, 2026 Source: https://www.lesswrong.com/posts/rjzqbfsW552PyMBNv/i-m-leaving-my-phd-to-pursue-ai-safety-work --- Narrated by TYPE III AUDIO.

Regardless of the actual singularity, we have definitely hit one in arXiv submissions. It is becoming apparent that the current paradigm of scientific knowledge production is not ready for AI. Accelerating research speed and expanding access means there is a lot more slop and there is also a lot more good work, but way too much for the current peer review system to delineate between. We need some accurate and decentralized way to aggregate this information. This makes a scientific prediction market, like Robin Hanson's idea futures, really appealing. The community can have markets for broad questions we care about, bystanders get paid for pointing out good research, and scientists are financially incentivized to make progress on the highest leverage issues. But with Hanson's conception, there is one big problem, which is how the market gets settled. For abstract scientific questions, like whether models are reward seeking or fitness seeking, we have no definitive event which can force market settlement. Hanson suggests that we use the convergence of scientific consensus, but this proposal takes trust in possibly misaligned opinions and requires the centralized coordination of scientists, which is hard to create---especially in pre-paradigmatic fields like AI safety. [...] --- First published: November 12th, 2026 Source: https://www.lesswrong.com/posts/7xJeaAKTKxbHL6F9a/a-new-kind-of-prediction-market-for-science --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Some researchers worry that looping (as in looped transformers) is a particularly scary way to make AI models more capable, because looped models will be harder to monitor than standard transformers of similar capability. We test this worry on NVIDIA's Dory 4B, a publicly available looped transformer, and find evidence that it is justified. Result 1. Looping boosts no-CoT reasoning much more than it boosts general capabilities. Dory can run with a configurable number of loops. It was trained to use up to 4 loops, and adding loops up to 8 at inference time still improves performance. Without looping, Dory's no-CoT capabilities (measured by NCRI, the No-CoT Reasoning Index) match what a trend fitted to open-source models predicts from how well it predicts text (measured in bits-per-byte). Its general with-CoT reasoning capability (measured by “mini-ECI”, a custom, smaller version of Epoch's Capabilities Index) is high relative to its NCRI, so Dory sits below the NCRI-vs-mini-ECI trend, but it is similar to the best models of its size. Adding loops improves mini-ECI and bits per byte only slightly but increases NCRI substantially. In fact, if we plot NCRI against total parameters, counting looped parameters once per loop, NCRI rises faster [...] ---Outline:(05:39) Result 1: Looping boosts no-CoT reasoning much more than general capabilities(12:09) Result 2. Dory has no CoT controllability(13:31) Result 3. Recurrent-Dory can evade monitors(21:13) There are two images here. First: en-US-AvaMultilingualNeural__ Four line graphs titled "Cfgpatch (chain length 4-8): GRPO against a CoT monitor," comparing normal-Dory and recurrent-Dory. Second: en-US-AvaMultilingualNeural__ Four-panel line graph: "Arithmetic (6-12 operations): GRPO against a CoT monitor" comparing normal-Dory and recurrent-Dory.(21:42) Result 4. Capabilities uplift under recurrent KV sharing(23:58) Limitations and Future Work(24:24) Conclusion(24:50) Further experiments on a larger looped model(28:09) Appendix(28:12) A. Repeating the question boosts NCRI (only) when Dory is looped(28:31) A.1 Filler token uplift(29:50) B. More about CoT controllability(30:25) B.1 CoT + output length(31:13) C. Additional figures(31:17) C.1 Output controllability by rule(31:32) C.2 Thinking-on Reasoning index (NCRI, but with CoT) vs mini-ECI: Astra/6.1 Sol are still better.(31:55) C.3 NCRI with thinking on vs bpb(32:14) C.4 Capability metrics vs. pretrain compute and tokens --- First published: November 12th, 2026 Source: https://www.lesswrong.com/posts/uxsDuWWHNTpKrsF6P/looping-a-transformer-increases-no-cot-capabilities-more --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Science is caught between Law and Chaos. On the side of Law: Things should make explicit sense to your deliberate reasoning ("system-two"). If you just have a "vibe" about how things work, that's bad. Measure everything. Use numbers whenever possible. You want things to be math.It's good to follow an entirely repeatable process. Document everything. Don't do anything weird, lest that weirdness turn out to be important and be unable to be replicated. Use standard equipment. Be Normal. As a sub-aspect of the above, you should make a plan and you should stick to your plan. Switching things up in an ad hoc way is a good way to make things weird or ruin the validity of your methodology.Everything should be controlled. If you can control your environment you can repeat the process. But also, you should have a control arm for your experiment, to allow you to see the counterfactual by balancing everything as perfectly as possible except for the intervention.Communicate your findings in clear, predictable channels, using standard language, standard paper structure, standard graphs, and so on. Misunderstandings abound, and an ounce of prevention is worth a pound of cure. Don't make [...] The original text contained 1 footnote which was omitted from this narration. --- First published: November 12th, 2026 Source: https://www.lesswrong.com/posts/bmogjuWbrJfkprghx/your-research-needs-more-of-the-void --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.