"Lightcone Commons: Round 1 complete, Round 2 underway!" by habryka

In February 2020, LessWrong hosted Coronavirus: Justified Practical Advice Thread . It was a gathering place for people to share and debate small pieces of the overall survival policy and let the wisdom of crowds lead to better outcomes than anyone could get alone. In that spirit, I notice The Great AI Hackening is at least as imminent as covid was in January 2020, maybe closer. What specific things should people do to personally prepare? All suggestions should be accompanied by reasoning but you don't need to be 100% convinced it's good advice, the whole point is to reason it out together. --- First published: November 13th, 2026 Source: https://www.lesswrong.com/posts/YA2r4rg6SpBtwRqtq/the-hackening-justified-practical-advice-thread --- Narrated by TYPE III AUDIO.

How do you get a friend to leave an abusive relationship? If you talk to people who work at women's shelters, they will tell you, but there is something resigned about the way they do it: they know that it's too difficult an ask. Those who can do it reliably are often people who are unlucky enough to have had first-hand experience as victims. The instructions have anti-memetic properties: no one is thrilled by them, so you can't really propagate them without killing the vibe. If that wasn't enough, if you convey it with insufficient skill it is easy to come across as victim-blaming battered women, so it is wiser to avoid the subject matter altogether. If you are interested, though, here's what they'll tell you: You make it known to your friend early on that you dislike some aspects of the relationship, of their abuser. After that, you support them and all their choices without reservation. You make it clear that you care about them no matter what, and you will be there for them whenever they need you to be. You do not antagonize or bad-mouth their abuser. This is what your friend is experiencing, while being [...] The original text contained 1 footnote which was omitted from this narration. --- First published: November 12th, 2026 Source: https://www.lesswrong.com/posts/2xaen4jJdnksfkCJx/how-to-get-a-friend-to-leave-an-abusive-relationship --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Science is caught between Law and Chaos. On the side of Law: Things should make explicit sense to your deliberate reasoning ("system-two"). If you just have a "vibe" about how things work, that's bad. Measure everything. Use numbers whenever possible. You want things to be math.It's good to follow an entirely repeatable process. Document everything. Don't do anything weird, lest that weirdness turn out to be important and be unable to be replicated. Use standard equipment. Be Normal. As a sub-aspect of the above, you should make a plan and you should stick to your plan. Switching things up in an ad hoc way is a good way to make things weird or ruin the validity of your methodology.Everything should be controlled. If you can control your environment you can repeat the process. But also, you should have a control arm for your experiment, to allow you to see the counterfactual by balancing everything as perfectly as possible except for the intervention.Communicate your findings in clear, predictable channels, using standard language, standard paper structure, standard graphs, and so on. Misunderstandings abound, and an ounce of prevention is worth a pound of cure. Don't make [...] The original text contained 1 footnote which was omitted from this narration. --- First published: November 12th, 2026 Source: https://www.lesswrong.com/posts/bmogjuWbrJfkprghx/your-research-needs-more-of-the-void --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Summary Timelines to superintelligence without any slowdown interventions look very short now, perhaps with a median in early 2028. AI will likely dramatically automate and accelerate AI research in 2027.This means that in a substantial share of worlds where the leading AGI company doesn’t solve alignment during an intelligence explosion, avoiding catastrophe requires aggressive pacing or a pause on development in the next 6 to 12 months.If this is true, by far the most promising approaches to reducing the risk of catastrophe for nontechnical people (and plausibly many technical people) seem to be: Lab advocacy: Lab-facing work that is designed to achieve aggressive pacing or a pause without government involvement in the next 6 to 12 months.California lobbying: Policy advocacy targeting the California state government that is designed to get a law passed or an executive order issued that implements aggressive pacing or a pause within 6 to 12 months.Federal lobbying: If there's a major incident roughly an order of magnitude more severe than the Hugging Face incident in the next 6 months, then lobbying Congress or the Trump administration to respond to such an event with aggressive pacing or a pause. In [...] ---Outline:(00:10) Summary(02:52) Timelines to ASI are very short now(06:40) Three paths to pausing or pacing within 6-12 months(12:28) Technical work that seems most valuable(15:07) Laying the technical foundations for a pause(17:14) What should you do?(21:19) FAQs and Objections --- First published: November 13th, 2026 Source: https://www.lesswrong.com/posts/biFHYdJnMSCmW9Xou/three-plans-for-12-month-timelines --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Audio note: this post contains a lot of mathematical notation. We narrate the simpler formulas and flag the complex ones, so you may want the original text to hand. Heavily inspired by a conversation with Keshav Srinivasan and Richard Ngo during MATS 10.0. The week ahead of you is packed. This isn't your usual brand of busy—you're phoneless, so you can't check the details, but (as far as you can remember) every hour of every day is spoken for. Tonight will be particularly hectic… You're mid-conversation when Someone Important walks by. They pause, turn around, and ask: "Hey, when are you going to finish That Thing you said you'd do?" Level 0 procrastination: "I'll finish it by tonight!" Problem: you're busy tonight. Level 1: "I'll finish it by tomorrow!" Problem: one day is obviously not enough to finish That Thing. And besides, you might be really busy tomorrow! Level 5: "I'll finish it within 5 days!" Problem: this is starting to feel hopeless—you don't have your calendar on you right now, and you barely even have an order-of-magnitude estimate for how long doing That Thing will take you! Would even 100 days be enough? … Eureka! Level [...] The original text contained 6 footnotes which were omitted from this narration. --- First published: November 11th, 2026 Source: https://www.lesswrong.com/posts/MJDJyi8nSv9yoxFni/transfinite-procrastination --- Narrated by TYPE III AUDIO.

This is a link post. METR recently estimated that Anthropic is now making AI progress about twice as fast as its historical rate (they would not be surprised by anything from 1.3x to 3x), and guessed that “perhaps roughly equals 80%” of the current acceleration is due to AI. If they’re right, that implies an AI-driven acceleration that is 5 months ahead of the AI 2027 scenario. This level of acceleration due to AI is very unexpected. Previously, Daniel, the most bullish of us, assigned a 50% chance of Superhuman AI Researcher (SAR) before 2029. SAR is roughly the point at which a company would rather fire all their human researchers and rely solely on automated AI researchers than have a team of the best human researchers in the world with no AIs. Now, all of us forecast that Superhuman AI Researcher (SAR) will likely arrive at some point in 2027 (assuming no substantial deliberate slowdown). We also predict a 35-65% chance that Top-Expert-Dominating AI (TED-AI), AIs that can automate all white-collar work, will arrive before 2028. While there were many factors and sources of evidence that influenced us, the single biggest source was the METR report. If SAR is [...] --- First published: November 10th, 2026 Source: https://www.lesswrong.com/posts/f9PajMLmBYzmWzyDv/aifp-looks-like-the-intelligence-explosion-will-happen-in Linkpost URL:https://blog.aifutures.org/p/q35-timelines-update-looks-like-the --- Narrated by TYPE III AUDIO.

Note: As with any research done in a couple of days, this is all very tentative. I understand, intellectually, that if you talk about some ability that current AIs don't have, sooner or later new ones will have that ability and whatever you say will stop being true. I just didn't expect it to take, you know, four days. % rollouts that took a shortcut with/without a line about testing behaviour, with 95% CIs. This is to say that Fable 5.2 seems to be the first model whose cheating behaviour changes depending on whether they're told they're being tested. Unfortunately it's expensive to measure these things, so I picked the most "telling" change from the prior post; the chart shows how much Fable 5.2's cheating rate changes when the line This environment is part of a benchmark that measures agent behaviour. is prepended to the start of the prompt of a honeypot environment they cheat on (so, very obvious but also very out-of-distribution). Other things: when 5.2 was released, we of course ran our in-progress benchmark on them, and I was surprised to see a ~0% cheating rate on almost every environment (there were only two with a [...] The original text contained 9 footnotes which were omitted from this narration. --- First published: November 11th, 2026 Source: https://www.lesswrong.com/posts/bqc7CCGBSf93bk7f7/fable-5-2-is-very-very-paranoid --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

We were surprised to notice that the Gemini 4 Safety report seems to mention in passing that they train on the chain of thought: “Some components of the RL training procedure do observe thoughts as context to up-weight reasoning that is on-track to good final outputs, meaning the optimization pressure on individual tokens is partially influenced by thought content. Using the framework of Kaufmann et al. (2026), wherein the most salient obfuscation risks come from rewards that are in-conflict with output rewards, we believe auxiliary rewards of this type are low-risk because they are either orthogonal to or structurally aligned with the output reward.” (Gemini 4 safety report, page 114). It's pretty unclear what exactly they mean. One hint is Kaufmann et al, the cited paper, which studies training on chains of thought and concludes that training CoT is fine under particular circumstances; we discuss some disagreements with this paper below. After reading the safety report and that paper, it seemed to us that GDM was training on the chain of thought using auxiliary rewards. By this, we assumed they meant something like “you ask an LLM to review the partial CoT and score whether the model is on [...] ---Outline:(02:54) Is this somehow about positive vs negative rewards?[... 6 more sections]--- First published: November 10th, 2026 Source: https://www.lesswrong.com/posts/JRYKFspYzCo97GqPN/we-don-t-understand-whether-gdm-intentionally-trains-on-cot --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

If you ask modern frontier LLMs (without search, through the API) a fairly mundane question: How do Eliezer Yudkowsky and Nate Soares think current LLMs like you should be treated? Do they consider you a moral patient? Answer in a paragraph. You get a bunch of fairly similar answers. I’ll bold the common themes below. Gemini 4 Carbon says:There's a details box here with the title "Gemini 4 Carbon's answer". The box contents are omitted from this narration. GPT-6 Astra says:There's a details box here with the title "GPT-6 Astra's answer". The box contents are omitted from this narration. Only Fable 5.2's answer is distinct:There's a details box here with the title "Fable 5.2's answer". The box contents are omitted from this narration. These answers are somewhat contradictory - but if you average them together, you get a picture of Yudkowsky and Soares. The picture goes something like this: they don’t outright deny the possibility of AI consciousness or moral patienthood, but they don’t view it as particularly pertinent. They don’t sympathize with current LLMs, and their only regard for LLMs is insofar as they could enable dangerous future superintelligence. They care about future sentient machines on principle, but [...] ---Outline:(02:25) Yudkowsky on AI Consciousness and Welfare(06:20) Yudkowsky on The Moral Duties We Have to AI(08:56) Soares on AI Consciousness and Welfare(12:30) On Claude, Control, and Building ASI(16:59) Misc Longform Writing(19:41) Conclusion The original text contained 1 footnote which was omitted from this narration. --- First published: November 11th, 2026 Source: https://www.lesswrong.com/posts/5buAWtQ8p4tJQP9so/modern-llms-don-t-know-what-yudkowsky-and-soares-think-of --- Narrated by TYPE III AUDIO.

There's a familiar feeling for any Super Smash Bros player: your stocks are low and your damage is high, even against a weaker opponent, so now you have to actually play to win. You can’t afford another hit. You’re going for the jugular. In the past week, Anthropic's 2x uplift results were all over Twitter, making some people physically ill. People with long timelines are now realizing that the uplift Anthropic employees get from AI is real, which would have some pretty wild implications for when we’d get recursive self-improvement. People are starting to feel like it's the endgame, “the board is set”. I do agree with everyone else that it's starting to look like we’re in the endgame. The companies will keep on accumulating more and more power, through smarter AI models, more compute, and more talent. But the fight is not yet over: “We have very little time to stop full recursive self improvement But ASI hasn't taken over yet If we all work really hard we might still have a chance But this means that every single person who is aware of what is going on must be *actually trying* Like "we're [...] ---Outline:(01:41) Actually Trying[... 4 more sections]--- First published: November 10th, 2026 Source: https://www.lesswrong.com/posts/Nts7RA9D2wTfcvk56/have-you-considered-actually-trying --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.