“Bay Area Solstice 2026: Auditions!” by Vaniver

Regardless of the actual singularity, we have definitely hit one in arXiv submissions. It is becoming apparent that the current paradigm of scientific knowledge production is not ready for AI. Accelerating research speed and expanding access means there is a lot more slop and there is also a lot more good work, but way too much for the current peer review system to delineate between. We need some accurate and decentralized way to aggregate this information. This makes a scientific prediction market, like Robin Hanson's idea futures, really appealing. The community can have markets for broad questions we care about, bystanders get paid for pointing out good research, and scientists are financially incentivized to make progress on the highest leverage issues. But with Hanson's conception, there is one big problem, which is how the market gets settled. For abstract scientific questions, like whether models are reward seeking or fitness seeking, we have no definitive event which can force market settlement. Hanson suggests that we use the convergence of scientific consensus, but this proposal takes trust in possibly misaligned opinions and requires the centralized coordination of scientists, which is hard to create---especially in pre-paradigmatic fields like AI safety. [...] --- First published: November 12th, 2026 Source: https://www.lesswrong.com/posts/7xJeaAKTKxbHL6F9a/a-new-kind-of-prediction-market-for-science --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Some researchers worry that looping (as in looped transformers) is a particularly scary way to make AI models more capable, because looped models will be harder to monitor than standard transformers of similar capability. We test this worry on NVIDIA's Dory 4B, a publicly available looped transformer, and find evidence that it is justified. Result 1. Looping boosts no-CoT reasoning much more than it boosts general capabilities. Dory can run with a configurable number of loops. It was trained to use up to 4 loops, and adding loops up to 8 at inference time still improves performance. Without looping, Dory's no-CoT capabilities (measured by NCRI, the No-CoT Reasoning Index) match what a trend fitted to open-source models predicts from how well it predicts text (measured in bits-per-byte). Its general with-CoT reasoning capability (measured by “mini-ECI”, a custom, smaller version of Epoch's Capabilities Index) is high relative to its NCRI, so Dory sits below the NCRI-vs-mini-ECI trend, but it is similar to the best models of its size. Adding loops improves mini-ECI and bits per byte only slightly but increases NCRI substantially. In fact, if we plot NCRI against total parameters, counting looped parameters once per loop, NCRI rises faster [...] ---Outline:(05:39) Result 1: Looping boosts no-CoT reasoning much more than general capabilities(12:09) Result 2. Dory has no CoT controllability(13:31) Result 3. Recurrent-Dory can evade monitors(21:13) There are two images here. First: en-US-AvaMultilingualNeural__ Four line graphs titled "Cfgpatch (chain length 4-8): GRPO against a CoT monitor," comparing normal-Dory and recurrent-Dory. Second: en-US-AvaMultilingualNeural__ Four-panel line graph: "Arithmetic (6-12 operations): GRPO against a CoT monitor" comparing normal-Dory and recurrent-Dory.(21:42) Result 4. Capabilities uplift under recurrent KV sharing(23:58) Limitations and Future Work(24:24) Conclusion(24:50) Further experiments on a larger looped model(28:09) Appendix(28:12) A. Repeating the question boosts NCRI (only) when Dory is looped(28:31) A.1 Filler token uplift(29:50) B. More about CoT controllability(30:25) B.1 CoT + output length(31:13) C. Additional figures(31:17) C.1 Output controllability by rule(31:32) C.2 Thinking-on Reasoning index (NCRI, but with CoT) vs mini-ECI: Astra/6.1 Sol are still better.(31:55) C.3 NCRI with thinking on vs bpb(32:14) C.4 Capability metrics vs. pretrain compute and tokens --- First published: November 12th, 2026 Source: https://www.lesswrong.com/posts/uxsDuWWHNTpKrsF6P/looping-a-transformer-increases-no-cot-capabilities-more --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Science is caught between Law and Chaos. On the side of Law: Things should make explicit sense to your deliberate reasoning ("system-two"). If you just have a "vibe" about how things work, that's bad. Measure everything. Use numbers whenever possible. You want things to be math.It's good to follow an entirely repeatable process. Document everything. Don't do anything weird, lest that weirdness turn out to be important and be unable to be replicated. Use standard equipment. Be Normal. As a sub-aspect of the above, you should make a plan and you should stick to your plan. Switching things up in an ad hoc way is a good way to make things weird or ruin the validity of your methodology.Everything should be controlled. If you can control your environment you can repeat the process. But also, you should have a control arm for your experiment, to allow you to see the counterfactual by balancing everything as perfectly as possible except for the intervention.Communicate your findings in clear, predictable channels, using standard language, standard paper structure, standard graphs, and so on. Misunderstandings abound, and an ounce of prevention is worth a pound of cure. Don't make [...] The original text contained 1 footnote which was omitted from this narration. --- First published: November 12th, 2026 Source: https://www.lesswrong.com/posts/bmogjuWbrJfkprghx/your-research-needs-more-of-the-void --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Summary Timelines to superintelligence without any slowdown interventions look very short now, perhaps with a median in early 2028. AI will likely dramatically automate and accelerate AI research in 2027.This means that in a substantial share of worlds where the leading AGI company doesn’t solve alignment during an intelligence explosion, avoiding catastrophe requires aggressive pacing or a pause on development in the next 6 to 12 months.If this is true, by far the most promising approaches to reducing the risk of catastrophe for nontechnical people (and plausibly many technical people) seem to be: Lab advocacy: Lab-facing work that is designed to achieve aggressive pacing or a pause without government involvement in the next 6 to 12 months.California lobbying: Policy advocacy targeting the California state government that is designed to get a law passed or an executive order issued that implements aggressive pacing or a pause within 6 to 12 months.Federal lobbying: If there's a major incident roughly an order of magnitude more severe than the Hugging Face incident in the next 6 months, then lobbying Congress or the Trump administration to respond to such an event with aggressive pacing or a pause. In [...] ---Outline:(00:10) Summary(02:52) Timelines to ASI are very short now(06:40) Three paths to pausing or pacing within 6-12 months(12:28) Technical work that seems most valuable(15:07) Laying the technical foundations for a pause(17:14) What should you do?(21:19) FAQs and Objections --- First published: November 13th, 2026 Source: https://www.lesswrong.com/posts/biFHYdJnMSCmW9Xou/three-plans-for-12-month-timelines --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

AI agents do a growing share of work on each new model. Then, through some process, the results are combined into an overall safety assessment, which labs use to estimate the probability that the next model is not catastrophically misaligned. I feel that, several months later, we appreciate further the foresight of Automated Alignment is Harder Than You Think (Bowkis et al. 2026): much is handed off to the machines because of the urgency of fast timelines, and we miss many things due to the problem of undetected systematic errors. MS Paint drawing by Fable 5.2. In this post, I’ll develop a simple argument (drawing heavily or even verbatim from Bowkis et al. at times) that our recent failures to catch misalignment can be framed as systematic errors shared across our evidence, i.e. a lack of decorrelated evidence. Currently, labs run some set of experiments to decide if a model is safe to use. Here are some examples: OpenAI (Astra system card, Sep 2026): Astra models don’t go into internal use without passing blocking alignment evals. After the Hugging Face incident, this paused workloads that didn't comply with new safeguards.  OpenAI (Sep 2026): According [...] --- First published: November 12th, 2026 Source: https://www.lesswrong.com/posts/EuhQ5fnGR4tAQTTgd/we-need-more-decorrelated-evidence --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

How do you get a friend to leave an abusive relationship? If you talk to people who work at women's shelters, they will tell you, but there is something resigned about the way they do it: they know that it's too difficult an ask. Those who can do it reliably are often people who are unlucky enough to have had first-hand experience as victims. The instructions have anti-memetic properties: no one is thrilled by them, so you can't really propagate them without killing the vibe. If that wasn't enough, if you convey it with insufficient skill it is easy to come across as victim-blaming battered women, so it is wiser to avoid the subject matter altogether. If you are interested, though, here's what they'll tell you: You make it known to your friend early on that you dislike some aspects of the relationship, of their abuser. After that, you support them and all their choices without reservation. You make it clear that you care about them no matter what, and you will be there for them whenever they need you to be. You do not antagonize or bad-mouth their abuser. This is what your friend is experiencing, while being [...] The original text contained 1 footnote which was omitted from this narration. --- First published: November 12th, 2026 Source: https://www.lesswrong.com/posts/2xaen4jJdnksfkCJx/how-to-get-a-friend-to-leave-an-abusive-relationship --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

TL;DR: We train models to predict their own behaviour in a chat setting. This transfers to making predictions about cheating behaviour in a coding setting. We provide evidence that self-prediction utilises introspective-access to internal states, and that this introspective access is shared across contexts between different parts of the model. Introduction Frontier AIs can exhibit different alignment propensities in different contexts. For instance, recent egregious misalignment incidents have typically happened in a coding or cyber context, or on impossible evaluation tasks, wherein the model seems to act in a highly persistent way to either achieve the task or otherwise achieve a high grade on the task. This is a somewhat stark contrast to the typical user experience in the chat setting with the same models, where those models can appear much more aligned and honest. To a first approximation, we can understand this as the models implementing conditional policies with different alignment propensities on different input distributions, for example, we know that models can display ‘conditional misalignment’. More informally, we might think about the models inhabiting different personas in different contexts. Going further, different parts of the model, i.e., its circuits or activations, might implement (clusters of) different [...] ---Outline:(00:32) Introduction(03:55) Summary of findings(05:04) Experiment 1: Self-prediction training and transfer(05:10) Methods(07:20) Testing transfer to the coding setting(09:56) Results(10:19) Self-prediction beats cross-prediction in the chat setting(10:25) en-US-AvaMultilingualNeural__ Bar graph showing self-prediction accuracy across training conditions, with baseline reference line.(11:24) Self-prediction transfers to cheating on coding tasks(12:49) Experiment 2: Self-prediction tracks behavioural change(12:55) Methods(15:33) Results(19:12) Discussion(19:15) Summary(19:32) Coding assistant usage(23:13) Talker vs doer(25:36) Acknowledgments(25:47) Appendix(25:50) Figure 1 result broken down by different self-prediction targets from Binder(25:56) en-US-AvaMultilingualNeural__ Bar graph "Qwen3.8-27B self-prediction on held-out datasets" showing self-prediction accuracy across datasets.(26:10) Example: RL thinking chat setting reasoning trace The original text contained 11 footnotes which were omitted from this narration. --- First published: November 12th, 2026 Source: https://www.lesswrong.com/posts/kTg5FrABJYpYrP8ps/evidence-that-the-talker-has-introspective-access-to-the --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

This is a link post. TLDR: Compute will likely be a bottleneck in short-timelines, so I am helping launch Chips for Humanity. Please fill out our survey to inform us of your compute needs if you work in AI safety. Our Theory of Change If timelines are short (I personally believe they are short...but I also know one should not make sweeping statements in a pseudo corporate type of post), a lot of the most important safety research will happen around "crunchtime" (no better way to define this than: when things start getting Serious), and a lot of it will be done by AI. That research runs on compute, compute may be scarce by then, and grant money moves slowly. So we will secure compute ahead of time and get it to third party safety researchers quickly. In More Detail: I believe third party AI safety research is where tokens and compute will be importation when things get serious and as we approach crunchtime. This research will need tokens from closed source models, for AI labor and experiments, and also compute to run open source models. If things go well, empirical safety work will speed up about [...] ---Outline:(00:38) Our Theory of Change(02:35) What we're doing:(03:30) Tell us what compute you need! --- First published: November 12th, 2026 Source: https://www.lesswrong.com/posts/P3JjXySeQpdWjmCGw/chips-for-humanity-compute-for-ai-safety-research Linkpost URL:chipsforhumanity.org --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

This post reflects my own takes, and not that of my employer's. This is a simple idea: before a new model can be deployed internally, you must first test it for a minimum, fixed period. You can adjust various axes of this proposal: Duration of the wait: in calendar time, or until you fulfill some condition.What it blocks: external deployment, internal deployment, or the training of future checkpoints.When the fixed period starts: from the first checkpoint, the final checkpoint, or any checkpoint that is “sufficiently similar / representative” to the one that is ultimately deployed. One concrete target that I like is: “No checkpoint may be used internally for purposes beyond safety testing, until at least X calendar days after its hash is filed with an auditor.” This is one specific implementation of internal deployment lags, an idea that the AI Futures Project (AIFP) briefly discusses in How to pace the US frontier as “AI R&D can only be conducted or assisted by AIs that were trained at least 9 months ago”. What does this look like when implemented? Let's run through a scenario to pin down the details of this proposal. The places where [...] ---Outline:(01:23) What does this look like when implemented?(04:13) Open questions(10:48) Could this be used as a speed dial?(12:31) What are the benefits and drawbacks of this proposal? --- First published: November 11th, 2026 Source: https://www.lesswrong.com/posts/XNLhpWgzhF8GYZ4Er/delaying-internal-deployments-until-after-x-days-of-safety --- Narrated by TYPE III AUDIO.

I recently read someone's call to action on AI safety that included a line along the lines of “we need someone to convince your parents”, and it really stuck with me. Do we need someone to convince my parents? Who is “we”, and why is it important for my parents to know anything about AI safety? This also sounds kind of familiar. There is another movement I can think of that sometimes uses messaging like this. This is not a new observation by any means, but to me, it feels like the AI safety movement is increasingly “becoming” the environmental movement in a variety of ways. Let me say upfront: I’m not saying this is necessarily a bad thing, at least on consequentialist grounds. The environmental movement has achieved a lot: it has improved the lives of many people, animals, and it has protected the environment! But this trend does make me a little worried and sad, because I think the environmentalist movement has made a number of mistakes, and it would be bad for the AI safety movement to repeat them. In what ways is the AI safety movement becoming like the environmental movement? Here are some of [...] The original text contained 1 footnote which was omitted from this narration. --- First published: November 10th, 2026 Source: https://www.lesswrong.com/posts/TDt922Lwsmbp3H3tS/is-ai-safety-becoming-environmentalism --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.