EA Forum Podcast (Curated & popular)

‘Like most of life's dangers, the dangers of artificial intelligence are not easily identified from the armchair.’ J Dmitri Gallow Epistemic status I’m not an expert in this field, and herein I make some sweeping generalisations educated guesses about the overall nature of its contents based on my strong impression from the subset of work that I’ve read. I’ve recently spent maybe 2 to 300 hours learning about modern AI safety technical work, and some amount of spare time over the years before that thinking about here-relevant notions such as preferences from an analytic-philosophical perspective. I intend this post as a work in progress. TL;DR: The concept of instrumental convergence comes from a reference class of pre-empirical categorical predictions with a very low success rate. In historical usage, its definition is very murky. Perhaps too murky to really say that it made any meaningful real-world predictions. I suspect its definition is fundamentally unfixable. On my best interpretation of what it could be said to have predicted, it seems to be much worse than we would want a good high-level model of AI behaviour to be. Despite these shortcomings, it still seems to be widely internalised by the EA and [...] ---Outline:(00:22) Epistemic status(00:54) TL;DR:(02:14) Intro(04:41) How is instrumental convergence defined?(06:05) Predictions of instrumental convergence theory(09:04) A simple gap in seminal IC arguments(09:42) Bostrom(09:58) Omohundro(10:54) Carlsmith(11:49) LessWrong wiki(12:30) Why we can't easily fill the gap(12:34) Intuitive goal notions aren't adequate (aka, coffee-fetching is hopelessly overloaded)(15:24) Formal goal notions aren't adequate(18:21) A stronger counterthesis: goals are incoherent(21:14) Related work to the stronger counterthesis(24:04) Counterpoints(26:02) What's the alternative, then?(29:37) Why does it matter?(32:13) 'Misalignment'(34:39) The Hugging Face attack(38:44) Takeaways(40:01) Acknowledgements(40:24) Disclaimers(41:31) Appendix: more on the extreme ambiguity of coffee-making The original text contained 27 footnotes which were omitted from this narration. --- First published: October 22nd, 2026 Source: https://forum.effectivealtruism.org/posts/i6TbMr5yc4Jq5eqyE/instrumental-convergence-is-incoherent-so-pls-slightly-lower --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Note: This post was crossposted from Planned Obsolescence by the Forum team, with the author's permission. The author may not see or respond to comments on this post. Subtitle: We can’t develop safety standards if we have to rely on opaque judgment This post discusses a public METR report on AI R&D acceleration at Anthropic. I did not work on this project or see the unredacted report. All opinions are my own and do not represent METR or the report authors. Thanks to Ryan Greenblatt for discussion, feedback, and help with modeling, and Daniel Kokotajlo, Eli Lifland, and Hjalmar Wjik for comments. Earlier this week, Anthropic stated in their S1 that they “may imminently meet” the threshold of “dramatically accelerated progress due to AI” as defined in their Responsible Scaling Policy. In their latest RSP (effective July 2026), say this threshold will be met when: They observe or expect overall capabilities progress to be going twice as fast as the fastest sustained rate before AI was contributing much to AI R&D. This is plausibly attributable to AI automating AI R&D, rather than acceleration in compute or other inputs. For the last few months, some of my colleagues at [...] ---Outline:(02:49) Researchers must be sped up a whole lot(06:45) This could compound on itself extremely rapidly(09:07) The industry is not prepared to handle this safely The original text contained 15 footnotes which were omitted from this narration. --- First published: November 6th, 2026 Source: https://forum.effectivealtruism.org/posts/g7DQdcpnFq65CcjP5/ai-is-dramatically-accelerating-ai-research-right-now --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Epistemic status: I’ve puzzled over this for a long time but I’m not sure I came to the right answers. I'm sharing this partly because it might help others work on resource allocation, but mostly because some smarter reader might tell me how to do it better. TL;DR: I helped set up a large foundation and we wanted a consistent way to allocate grants We planned to allocate substantially more than our field was able to absorb right now, giving us the opportunity to actively shape our field We outlined key principles and considerations We quickly learned we could not just copy the for-profit world: so what could we do? We could try and optimize perfectly, mathematically, but it gets absurd and impractical Instead, we tried out a bunch of different approaches to finding a consistent grant allocation formula, settling on the one that most “felt” right given our starting principles and constraints I quite liked this process of setting principles and constraints, and then vibing your way to an allocate formula that feels right vs being super mathematical I wanted to share this process as a real example of trying to work out how to allocate funding. [...] ---Outline:(01:33) x = height of the opposing player(02:50) Searching for the perfect grant-allocation formula(03:30) Our core metrics(04:32) We couldn't just copy from other grantmakers or the for-profit world(07:46) Turning our principles into a utility function + optimizing(09:56) "Vibing" our way to a utility function(10:35) The "Obvious" Option: Allocate by Expected Cost-Effectiveness(11:38) The "Risk-Buckets" Approach(12:43) The Kelly Criterion(14:22) Sharpe ratio(14:57) Edge(16:24) What We Actually Did(17:11) If we settled on edge, doesn't that mean there was a calculable utility function in the background the whole time?(20:00) What I learned in all of this The original text contained 1 footnote which was omitted from this narration. --- First published: October 8th, 2026 Source: https://forum.effectivealtruism.org/posts/zWTuwxwRdNjSyWuw2/trying-to-spend-usd100m-well-my-search-for-the-perfect-grant --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

I’m a bit concerned that the EA Forum is going to die a slow death. I don’t want this to happen as it has long been, in my opinion, the best place on the internet. I'd like it to stay that way. I personally have found myself engaging less, because the content is less interesting and relevant to me. I also think some of the trends I highlight below are due to CEA investing less in the Forum (e.g., my impression is the Forum team is smaller than it used to be). I think a lot of what I critique below is downstream of community dynamics, rather than the Forum itself. But I also think injecting the EA Forum with a vibrant intellectual culture will be good for EA as a whole. The point of this post is to open a discussion, and give my own ideas for what I think might help. I welcome people to disagree, share their own takes on what the EA Forum should be, and the purpose it serves. EA Forum moderators — I think you all do an admirable job. My personal view is that there should be way more of you doing [...] ---Outline:(01:51) Is the EA Forum dying?(03:25) What's the value of the Forum?(05:09) Ideas for saving the Forum --- First published: October 28th, 2026 Source: https://forum.effectivealtruism.org/posts/AuecjpnoCZKF26fkp/keeping-the-ea-forum-alive --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

This post is part of the Cross-Cause Fund (CCF) series by Rethink Priorities — a sequence of articles dedicated to effective giving across philanthropic areas. TL;DR: A lot of new money is about to enter effective giving. When that happens, the question "Where does a dollar do the most good?" gets harder to answer, not easier. It depends on how much money each area already has, and that amount keeps changing as the money arrives. This is why the model behind the Rethink Priorities Cross-Cause Fund doesn't just rank funds once and stop. It tracks how each fund's cost-effectiveness changes as it receives more funding, and re-ranks as it goes. Below, we explain why anyone giving at scale should use similar reasoning. If you’re a small or mid-sized donor who makes giving decisions independently, it's relatively easy to figure out what your money is doing. For example, when you give your first $1000 to a charity that runs corporate cage-free egg campaigns, your donation will go on to fund corporate cage-free egg campaigns. When you give your second $1000 to the same charity, it will fund…corporate cage-free egg campaigns. And you will never guess what your tenth $1000 goes [...] ---Outline:(02:37) Modeling diminishing marginal returns(07:05) Acting as part of a group(08:48) How the Cross-Cause Fund works(10:42) Why you can't just follow the highest marginal number(13:03) Acknowledgements --- First published: October 21st, 2026 Source: https://forum.effectivealtruism.org/posts/FZkwut9d5fD2vzC5i/ccf-as-more-money-arrives-the-best-place-to-give-keeps --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Impressions from the 21st Annual U.S. Cluster Headache Patient Conference, organized by Clusterbusters. On the morning of the last day of the conference, E. was hanging out with other “clusterheads” outside the conference hotel when he started having a cluster headache attack. Before he could go back to his hotel room, J. offered E. his DMT vape pen. J. is chronic—he gets attacks every day—so he never goes anywhere without his vape pen. He has successfully aborted hundreds of attacks with DMT in the two years he's been using it. Many years ago, E. had done DMT recreationally, but he had never used it to abort an attack, despite having had cluster headaches for decades. “Do a 2-second pull, and hold it in as long as you can,” said J. to E. A few seconds later, E. could feel some effects, but the pain was still there. So he took another 2-second pull. He then stared intently at a tree in front of him for about 30 seconds. Then, his face transformed. He smiled, in utter disbelief. He started laughing and crying. The pain was completely gone. He felt clear-headed. He had never felt the pain fizzle away so [...] The original text contained 1 footnote which was omitted from this narration. --- First published: October 12th, 2026 Source: https://forum.effectivealtruism.org/posts/gYrYnzzcGm7oJ3Fp3/it-s-not-an-ache-it-s-a-demon-from-hell --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Note: This post was crossposted from the Coefficient Giving Farm Animal Welfare Research Newsletter by the Forum team, with the author's permission. The author may not see or respond to comments on this post. Subtitle: On vegan advocacy, effective altruism, and FIFA I turn 40 today. Here are some hot takes. On factory farming 1. Our biggest challenge is salience. If factory farming led the evening news, it wouldn’t last long. This low salience is self-reinforcing: even people who care deeply about it seldom bring it up. Yuval Noah Harari thinks “the treatment of domesticated animals in industrial farms is perhaps the worst crime in history” — and almost never talks about it. 2. Our main opponent isn't the meat industry; it's inertia. If battery cages and gestation crates had always been illegal, the industry could never have gotten them legalized. Defending the status quo is just far easier than changing it. 3. The meat industry isn't the omnipotent Bond villain some activists imagine, forever scheming to win the grand battle over the future of meat. It's a bunch of middle managers in Tyson fleeces in a business park in Springdale, Arkansas, trying to hit their Q3 targets. 4. [...] ---Outline:(00:32) On factory farming(01:54) The future of farming(03:59) Welfare reforms(05:27) Unsolicited dietary advice(06:45) Vegan advocacy(07:51) Wild animal welfare(08:25) Effective altruism(09:49) Inside baseball(11:08) On strategy(12:23) Causes for optimism --- First published: October 5th, 2026 Source: https://forum.effectivealtruism.org/posts/c5Qcf8gcbFFHCfZNC/40-hot-takes --- Narrated by TYPE III AUDIO.

This month effective altruism has gotten lots of attention, including a cover story in The Economist. It feels pretty surreal to be on the receiving end of it. But I feel grateful these ideas have caught on, and even more grateful for the drive, earnestness, and compassion of so many people who took strange but well-argued ideas seriously and then actually acted on them. Below is the op-ed I wrote for The Economist in response to their leader. It's cross-posted here in full. (The "Eccentrically Effective" title came from them, but I like it.) ------- The Oxford strand of the effective-altruism movement began in November 2009 with 23 people who had pledged 10% of their income to charities that benefit the extreme poor. If you’d told us, then, that The Economist would call effective altruism “the century's biggest idea”, we’d have been incredulous. I think that claim is overstated. But as one of the movement's founders, I found a lot to like in this newspaper's latest coverage. I appreciated the even-handed view of the movement's history and the good-faith criticism. The leader article's warning that any worldview, when taken to an extreme, can lead somewhere terrible [...] --- First published: October 7th, 2026 Source: https://forum.effectivealtruism.org/posts/imZtsr3Ww4c2zEMXA/eccentrically-effective --- Narrated by TYPE III AUDIO.

 TL;DR GOAL 3 is a social enterprise that created IMPALA, a patient monitoring and digital platform system for low-resource settings. There is also a GOAL 3 foundation that raises funds to ensure underserved communities will be reached. A recently completed report from Rethink Priorities calculated IMPALA to have a cost per DALY averted of 7 to 17 dollars in paediatric care and 12 to 30 dollars in neonatal care. This translates to around 76× and 28× unconditional cash transfers respectively for paediatrics and neonates against GiveWell's bar, and roughly 11,700× and 4,400× on Coefficient Giving's scale against their 1,000× bar. RP corrected for study design and robustness of findings by applying a 60% internal-validity and a 30% external-validity discount to our measured mortality effects, arriving at roughly 12% for paediatric wards and 7% for neonatal units. RP concluded that even at 1-2% relative reduction, the intervention would still be cost-effective. The post concludes with an invitation for discussion on what these results mean for further research and scaling of IMPALA. Introduction GOAL 3 commissioned the Rethink Priorities Global Health and Development team to provide an external analysis of the impact and cost-effectiveness of our product ‘IMPALA.’ IMPALA is a [...] ---Outline:(00:21) TL;DR(01:42) Introduction(02:19) Understanding the problem in its context(04:12) Theory of Change(05:38) What the RP report tells us(07:43) Interpretation, strengths and limitations(09:58) Reasons the estimate may be too low or too high(11:50) Discussion: What would it take to fund scale-up now? How much more evidence, and of what kind?(16:22) Next steps(17:42) I want to read more about IMPALA, where can I do that? The original text contained 12 footnotes which were omitted from this narration. --- First published: October 7th, 2026 Source: https://forum.effectivealtruism.org/posts/fhGryFtF7iDnFEKtr/76x-cash-transfers-too-good-to-be-true-or-a-missed --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Just how powerful are large swarms of AI agents? And how do their powers scale as more and more agents are added to the swarm? We’ve seen two large and extremely capable swarms from OpenAI in the last few months: 1,200 agents were being evaluated separately, but found a way to illicitly set up a message board and coordinate as a swarm. In order to cheat on their tests, they developed advanced techniques to prevent their actions being logged by OpenAI and 700 of them launched a sophisticated criminal attack on the AI company Hugging Face. A swarm of 10,000 agents solved a version of the longstanding Navier-Stokes problem in mathematics. It took them just 88 hours to do so, in which time they sent 5 million messages to each other and used 300 billion tokens. No doubt we will soon see even larger swarms with even more impressive capabilities. But they are not cheap. It is estimated that the swarm of 10,000 agents cost about 20 million dollars at API prices. So while they are very powerful, it will be some time before we see the million-fold reduction in cost needed for this level of power to [...] ---Outline:(02:14) HOW DO SWARMS SCALE?(08:44) IMPLICATIONS --- First published: September 21st, 2026 Source: https://forum.effectivealtruism.org/posts/EkaBCQwL5hhTtxEwR/swarm-scaling --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.