"The Case for Low-Competence ASI Failure Scenarios" by Ihor Kendiukhov

Suppose there is a fire in a nearby house. Suppose there are competent firefighters in your town: fast, professional, well-equipped. They are expected to arrive in 2–3 minutes. In that situation, unless something very extraordinary happens, it would indeed be an act of great arrogance and even utter insanity to go into the fire yourself in the hope of "rescuing" someone or something. The most likely outcome would be that you would find yourself among those who need to be rescued. But the calculus changes drastically if the closest fire crew is 3 hours away and consists of drunk, unfit amateurs. Or consider a child living in a big, happy, smart family. Imagine this child suddenly decides that his family may run out of money to the point where they won't have enough to eat. All reassurances from his parents don't work. The child doesn't believe in his parents' ability to reason, he makes his own calculations, and he strongly believes he is right and they are wrong. He is dead set on fixing the situation by doing day trading. What is that if not going nuts? Would those be wrong who ridicule this child and his complete mischaracterization [...] --- First published: March 22nd, 2026 Source: https://www.lesswrong.com/posts/EAH6Y6y3CDi3uxMou/my-most-costly-delusion --- Narrated by TYPE III AUDIO.

I think the community underinvests in the exploration of extremely-low-competence AGI/ASI failure modes and explain why. Humanity's Response to the AGI Threat May Be Extremely Incompetent There is a sufficient level of civilizational insanity overall and a nice empirical track record in the field of AI itself which is eloquent about its safety culure. For example: At OpenAI, a refactoring bug flipped the sign of the reward signal in a model. Because labelers had been instructed to give very low ratings to sexually explicit text, the bug pushed the model into generating maximally explicit content across all prompts. The team noticed only after the training run had completed, because they were asleep. The director of alignment at Meta's Superintelligence Labs connected an OpenClaw agent to her real email, at which point it began deleting messages despite her attempts to stop it, and she ended up running to her computer to manually halt the process. An internal AI agent at Meta posted an answer publicly without approval; another employee acted on the inaccurate advice, triggering a severe security incident that temporarily allowed employees to access sensitive data they were not authorized to view. AWS acknowledged that [...] ---Outline:(00:19) Humanitys Response to the AGI Threat May Be Extremely Incompetent(02:26) Many Existing Scenarios and Case Studies Assume (Relatively) High Competence(04:31) Dumb Ways to Die(07:31) Undignified AGI Disaster Scenarios Deserve More Careful Treatment(10:43) Why This Might Be Useful --- First published: March 19th, 2026 Source: https://www.lesswrong.com/posts/t9LAhjoBnpQBa8Bbw/the-case-for-low-competence-asi-failure-scenarios --- Narrated by TYPE III AUDIO.

A 2022 LessWrong post on orexin and the quest for more waking hours argues that orexin agonists could safely reduce human sleep needs, pointing to short-sleeper gene mutations that increase orexin production and to cavefish that evolved heightened orexin sensitivity alongside an 80% reduction in sleep. Several commenters discussed clinical trials, embryo selection, and the evolutionary puzzle of why short-sleeper genes haven't spread. I thought the whole approach was backwards, and left a comment: Orexin is a signal about energy metabolism. Unless the signaling system itself is broken (e.g. narcolepsy type 1, caused by autoimmune destruction of orexin-producing neurons), it's better to fix the underlying reality the signals point to than to falsify the signals. My sleep got noticeably more efficient when I started supplementing glycine. Most people on modern diets don't get enough; we can make ~3g/day but can use 10g+, because in the ancestral environment we ate much more connective tissue or broth therefrom. Glycine is both important for repair processes and triggers NMDA receptors to drop core temperature, which smooths the path to sleep. While drafting that, I went back to Chris Masterjohn's page on glycine requirements. His estimate for total need [...] ---Outline:(01:49) Glycine helps us sleep by cooling the body(02:26) Glycine cleans our mitochondria as we sleep(04:12) Most people could use more glycine(05:28) Fever is plan B for fighting infection; glycine supports plan A(09:28) Glycines cooling effect via the SCN is unrelated to its immune benefits(10:35) Glycine turns out to be a legitimate antipyretic after all(11:51) Practical considerations --- First published: March 22nd, 2026 Source: https://www.lesswrong.com/posts/87XoatpFkdmCZpvQK/is-fever-a-symptom-of-glycine-deficiency --- Narrated by TYPE III AUDIO.

In this post, I’m trying to put forward a narrow, pedagogical point, one that comes up mainly when I’m arguing in favor of LLMs having limitations that human learning does not. (E.g. here, here, here.) See the bottom of the post for a list of subtexts that you should NOT read into this post, including “…therefore LLMs are dumb”, or “…therefore LLMs can’t possibly scale to superintelligence”. Some intuitions on how to think about “real” continual learning Consider an algorithm for training a Reinforcement Learning (RL) agent, like the Atari-playing Deep Q network (2013) or AlphaZero (2017), or think of within-lifetime learning in the human brain, which (I claim) is in the general class of “model-based reinforcement learning”, broadly construed. These are all real-deal full-fledged learning algorithms: there's an algorithm for choosing the next action right now, and there's one or more update rules for permanently changing some adjustable parameters (a.k.a. weights) in the model such that its actions and/or predictions will be better in the future. And indeed, the longer you run them, the more competent they get. When we think of “continual learning”, I suggest that those are good central examples to keep in mind. Here are [...] ---Outline:(00:35) Some intuitions on how to think about real continual learning(04:57) Why real continual learning cant be copied by an imitation learner(09:53) Some things that are off-topic for this post The original text contained 3 footnotes which were omitted from this narration. --- First published: March 16th, 2026 Source: https://www.lesswrong.com/posts/9rCTjbJpZB4KzqhiQ/you-can-t-imitation-learn-how-to-continual-learn --- Narrated by TYPE III AUDIO.

Independent verification by the Brain Preservation Foundation and the Survival and Flourishing Fund — the results so far Cultivating independent verification Extraordinary claims require extraordinary evidence. In my previous post, "Less Dead", I said that my company, Nectome, has created a new method for whole-body, whole-brain, human end-of-life preservation for the purpose of future revival. Our protocol is capable of preserving every synapse and every cell in the body with enough detail that current neuroscience says long-term memories are preserved. It's compatible with traditional funerals at room temperature and stable for hundreds of years at cold temperatures. In this post, we’ll dive into the evidence for these claims, as well as Nectome's overall approach to cultivating rigorous, independent validation of our methods—a cornerstone of the kind of preservation enterprise I want to be a part of. To get to the current state-of-the-art required two major developmental milestones: Idealized preservation. A method capable of preserving the nanostructure of the brain for small and large animals under idealized laboratory conditions. Specifically, could we preserve animals well if we were allowed to perfectly control the time and conditions of death?   This work (2015-2018) resulted in a brand-new technique—aldehyde-stabilized cryopreservation—which was carefully [...] ---Outline:(00:16) Cultivating independent verification[... 7 more sections]--- First published: March 19th, 2026 Source: https://www.lesswrong.com/posts/NEFNs4vbNxJPJJgYY/nullius-in-verba --- Narrated by TYPE III AUDIO. ---Images from the article:

No-one knows when AI will begin having transformative impacts upon the world. People aren’t sure and shouldn’t be sure: there just isn’t enough evidence to pin it down. But we don’t need to wait for certainty. I want to explore what happens if we take our uncertainty seriously — if we act with epistemic humility. What does wise planning look like in a world of deeply uncertain AI timelines? I’ll conclude that taking the uncertainty seriously has real implications for how one can contribute to making this AI transition go well. And it has even more implications for how we act together — for our portfolio of work aimed towards this end.   AI Timelines By AI timelines, I refer to how long it will be before AI has truly transformative effects on the world. People often think about this using terms such as artificial general intelligence (AGI), human level AI, transformative AI, or superintelligence. Each term is used differently by different people, making it challenging to compare their stated timelines. Indeed even an individual's own definition of their favoured term will be somewhat vague, such that even after their threshold has been crossed, they might have [...] ---Outline:(00:58) AI Timelines[... 7 more sections]--- First published: March 19th, 2026 Source: https://www.lesswrong.com/posts/6pDMLYr7my2QMTz3s/broad-timelines --- Narrated by TYPE III AUDIO. ---Images from the article:

In the last two weeks, social media was set abuzz by claims that scientists had succeeded in uploading a fruit fly. It started with a video released by the startup Eon Systems, a company that wants to create “Brain emulation so humans can flourish in a world with superintelligence.” On the left of the video, a virtual fly walks around in a sandpit looking for pieces of banana to eat, occasionally pausing to groom itself along the way. On the right is a dancing constellation of dots resembling the fruit fly brain, set above the caption ‘simultaneous brain emulation’. At first glance, this appears astounding - a digitally recreated animal living its life inside a computer. And indeed, this impression was seemingly confirmed when, a couple of days after the video's initial release on X by cofounder Alex Wissner-Gross, Eon's CEO Michael Andregg explicitly posted “We’ve uploaded a fruit fly”. Yet “extraordinary claims require extraordinary evidence, not just cool visuals”, as one neuroscientist put it in response to Andregg's post. If Eon had indeed succeeded in uploading a fly - a goal previously thought to be likely decades away according to much of the fly neuroscience community - they’d [...] ---Outline:(03:43) A brief history of fruit fly connectomics[... 3 more sections]--- First published: March 19th, 2026 Source: https://www.lesswrong.com/posts/ybwcxBRrsKavJB9Wz/no-we-haven-t-uploaded-a-fly-yet --- Narrated by TYPE III AUDIO. ---Images from the article:

(Previously: Prologue.) Corrigibility as a term of art in AI alignment was coined as a word to refer to a property of an AI being willing to let its preferences be modified by its creator. Corrigibility in this sense was believed to be a desirable but unnatural property that would require more theoretical progress to specify, let alone implement. Desirable, because if you don't think you specified your AI's preferences correctly the first time, you want to be able to change your mind (by changing its mind). Unnatural, because we expect the AI to resist having its mind changed: rational agents should want to preserve their current preferences, because letting their preferences be modified would result in their current preferences being less fulfilled (in expectation, since the post-modification AI would no longer be trying to fulfill them). Another attractive feature of corrigibility is that it seems like it should in some sense be algorithmically simpler than the entirety of human values. Humans want lots of specific, complicated things out of life (friendship and liberty and justice and sex and sweets, et cetera, ad infinitum) which no one knows how to specify and would seem arbitrary to a [...] ---Outline:(03:21) The Constitutions Definition of Corrigibility Is Muddled(06:24) Claude Take the Wheel(15:10) It Sounds Like the Humans Are Begging The original text contained 1 footnote which was omitted from this narration. --- First published: March 16th, 2026 Source: https://www.lesswrong.com/posts/K2Ae2vmAKwhiwKEo5/terrified-comments-on-corrigibility-in-claude-s-constitution --- Narrated by TYPE III AUDIO.

I see people repeatedly make the mistake of referencing a very low liquidity prediction market and using it to make a nontrivial point. Usually the implication when a market is cited is that it's number should be taken somewhat seriously, that it's giving us a highly informed probability. Sometimes a market is used to analyze some event that recently occurred; reasoning here looks like "the market on outcome O was trading at X%, then event E happened and the market quickly moved to Y%, thus event E made O less/more likely." Who do I see make this mistake? Rationalists, both casually and gasp in blog posts. Scott Alexander and Zvi (and I really appreciate their work, seriously!) are guilty of this. I'll give a recent example from each of them. From Scott's Mantic Monday post on March 2: Having Your Own Government Try To Destroy You Is (At Least Temporarily) Good For Business On Friday, the Pentagon declared AI company Anthropic a “supply chain risk”, a designation never before given to an American firm. This unprecedented move was seen as an attempt to punish, maybe destroy the company. How effective was it? Anthropic isn’t publicly traded, so we [...] --- First published: March 16th, 2026 Source: https://www.lesswrong.com/posts/SrtoF6PcbHpzcT82T/psa-predictions-markets-often-have-very-low-liquidity-be --- Narrated by TYPE III AUDIO. ---Images from the article:

On Thursday, March 26th, a major new AI documentary is coming out: The AI Doc: Or How I Became an Apocaloptimist. Tickets are on sale now. The movie is excellent, and MIRI staff I've spoken with generally believe it belongs in the same tier as If Anyone Builds It, Everyone Dies as an extremely valuable way to alert policymakers and the general public about AI risk, especially if it smashes the box office. When IABIED was coming out, the community did an incredible job of helping the book succeed; without all of your help, we might never have gotten on the New York Times bestseller list. MIRI staff think that the community could potentially play a similarly big role in helping The AI Doc succeed, and thereby help these ideas go mainstream. (Note: Two MIRI staff were interviewed for the film, but we weren’t involved in its production. We just like it.) The most valuable thing most people can do is maximize opening-weekend success. Buy tickets to see the movie now; poke friends and family members to do the same. This will cause more theaters to pick up the movie, ensure it stays in theaters for longer, and broadly [...] --- First published: March 19th, 2026 Source: https://www.lesswrong.com/posts/w9BCbshKra7FKHTzi/the-ai-doc-is-coming-out-march-26 --- Narrated by TYPE III AUDIO.