AI Isn't That Scary

John Locke said brutes abstract not. Then honeybees, with brains smaller than our smallest unit of measurement, learned the concept of sameness. Now machines are doing things we thought only we could do. Should that unsettle us? Rather read? I also made this episode into an article: https://btr.mt/analects/abstract-learning-in-the-honeybee-lecture And the piece it’s based on: https://btr.mt/analects/abstract-learning-in-the-honeybee. Stuff I mentioned: Honey-bees are smarter than they should be—the article this episode grew out of Cope, A. J., Vasilaki, E., Minors, D., Sabo, C., Marshall, J. A. R. & Barron, A. B. (2018). Abstract concept learning in a simple neural network inspired by the insect brain. PLOS Computational Biology, 14(9), e1006435—my paper: the review of animals that pass match-to-sample, and the simple circuit Giurfa, M., Zhang, S., Jenett, A., Menzel, R. & Srinivasan, M. V. (2001). The concepts of ‘sameness’ and ‘difference’ in an insect. Nature, 410, 930–933—the 2001 honeybee match-to-sample paper Thomas, R. K. on levels of concept learning in animals Locke, J. (1689). An Essay Concerning Human Understanding—“brutes abstract not” Chittka, L. & Geiger, K. (1995). Can honey bees count landmarks? Animal Behaviour, 49(1), 159–164. Howard, S. R., Avarguès-Weber, A., Garcia, J. E., Greentree, A. D. & Dyer, A. G. (2018). Numerical ordering of zero in honey bees. Science, 360, 1124–1126—the zero study Howard, S. R., Avarguès-Weber, A., Garcia, J. E., Greentree, A. D. & Dyer, A. G. (2019). Numerical cognition in honeybees enables addition and subtraction. Science Advances, 5(2), eaav0961—the arithmetic follow-up MaBouDi, H. et al. (2021). Non-numerical strategies used by bees to solve numerical cognition tasks. Proceedings of the Royal Society B, 288, 20202711—the pushback: bees may be using magnitude, not number Bridges, A. D. et al. (2024). Bumblebees socially learn behaviour too complex to innovate alone. Nature, 627, 572–578—the two-step puzzle box Loukola, O. J., Solvi, C., Coscos, L. & Chittka, L. (2017). Bumblebees show cognitive flexibility by improving on an observed complex behavior. Science, 355, 833–836—bees learning ball-rolling from other bees Galpayage Dona, H. S., Solvi, C., Kowalewska, A., Mäkelä, K., MaBouDi, H. & Chittka, L. (2022). Do bumble bees play? Animal Behaviour, 194, 239–251. Can single cells learn? Ants are very cool Not brain regions, brain networks—on why we locate cognition in the frontal cortex On the alien characteristics of LLMs AI isn’t that scary the btrmt. lectures podcast bowerbirds [classic test](https://en.wikipedia.org/wiki/Conservation_(psychology) How to remember effectively (and why you’re doing it wrong) mushroom bodies How the term ‘role model’ came into being Why kids can’t lie but you can (the key to emotional intelligence) Wikipedia has the long version Related stuff: Autopoiesis: the origin of complex thought—the complexity of our world determines the complexity of our actions Smarter entities are less coherent Human exceptionalism is dead How to speak honeybee

Is your phone listening to you? You mention something random, something you’d never normally talk about, and within days there’s an ad for it. I used to be sure that was coincidence. Some companies have since tried. But I still think most of it is you. Rather read? I also made this episode into an article: https://btr.mt/analects/on-coincidence-lecture And the piece it’s based on: https://btr.mt/analects/on-coincidence. Stuff I mentioned: Eerie coincidences aren’t that eerie—the article this episode grew out of 404 Media (2024). Here’s the pitch deck for ‘Active Listening’ ad targeting—Cox Media Group’s sales pitch, and Google dropping them from its Partners Program Pan, E., Ren, J., Lindorfer, M., Wilson, C. & Choffnes, D. (2018). Panoptispy: Characterizing audio and video exfiltration from Android applications. Proceedings on Privacy Enhancing Technologies, 2018(4), 33–50—the Northeastern study of 17,000 apps; there’s a plain-English summary from Northeastern GrapheneOS—the privacy-hardened Android I use Google. Delete, switch, or add users—Android’s user profiles, which you can use to sandbox apps How to never get mad again (kind of)—the interruption theory of emotion Emotion and the mind—on predictive brains and cognitive misers (the cheapskates) Barrett, L. F. (2017). The theory of constructed emotion: an active inference account of interoception and categorization. Social Cognitive and Affective Neuroscience, 12(1), 1–23. Friston, K. (2010). The free-energy principle: a unified brain theory? Nature Reviews Neuroscience, 11(2), 127–138. A short summary of Friston’s Bayesian brain theory—the less obscure version ‘Meaning and its patterns in the brain’—nervous systems mapping actions to the environment ‘No action without emotion’—what nervous systems were doing long before brains Why your unconscious isn’t the bad guy—on autopilot MythBusters, 2005 season—“Toast: butter side up or down?” Malcolm Gladwell Shit—the sociology of the interesting, after Murray Davis A footnote on the toast. The folk belief turns out to be right, for physical reasons. Robert Matthews showed that toast sliding off a table doesn’t have the height to finish turning over, so it tends to land butter side down—the study won the 1996 Ig Nobel Prize for physics. Tossed in the air, it’s about 50/50. Which is rather the point: a coincidence you can’t check against its counterfactuals might be noise, and might be physics. Matthews, R. A. J. (1995). Tumbling toast, Murphy’s Law and the fundamental constants. European Journal of Physics, 16(4), 172–176. the btrmt. lectures podcast Related stuff: “Purple Doesn’t Exist”—the brain inventing what it can’t account for ‘The downsides of the prediction machine’

Leadership transformation consulting looks superficially like a scam: fake yoga, brain words, and “it’s very hard to measure impact”. I left the industry a cynic. But it does work, and I think the reason is so simple that it almost doesn’t matter how stupid it is. Rather read? I also made this episode into an article: https://btr.mt/analects/leadership-consulting-lecture And the piece it’s based on: https://btr.mt/analects/leadership-consulting. Stuff I mentioned: Leadership consulting is usually more ‘feel good’ than ‘do good’—the article this episode grew out of, and where I lay out the capture process in full The neuroscience con—on neuroscience washing Overengineering ‘calm down’—my complaint about emotional intelligence Overengineering calming down—the lecture version, on the chimp brain and the lizard brain The amygdala is a distraction—why it isn’t the fear centre of the brain The false promise of a return to nature—on evolutionary narratives about how we’re poorly designed for the modern world Beer, M., Finnström, M. & Schrader, D. (2016). Why leadership training fails—and what to do about it. Harvard Business Review. Gurdjian, P., Halbeisen, T. & Lane, K. (2014). Why leadership-development programs fail. McKinsey Quarterly. Saving the planet is an illusion—on the sustainability conversation UK Government (2026). Global biodiversity loss, ecosystem collapse and national security—the scary report (see the footnote below) Malcolm Gladwell Shit—on education as entertainment, and the company theatre pass Gladwell, M. (2008). Outliers. Little, Brown. Davis, M. S. (1971). That’s interesting! Towards a phenomenology of sociology and a sociology of phenomenology. Philosophy of the Social Sciences. IMAGINE—the firm I worked for, co-founded by Paul Polman and Valerie Keller Lacerenza, C. N. et al. (2017). Leadership training design, delivery, and implementation: A meta-analysis. Journal of Applied Psychology.—leadership training works Arthur, W. et al. (2003). Effectiveness of training in organizations: A meta-analysis of design and evaluation features. Journal of Applied Psychology. Collins, D. B. & Holton, E. F. (2004). The effectiveness of managerial leadership development programs. Human Resource Development Quarterly.—effects shrink the further you get from the classroom Blume, B. D. et al. (2010). Transfer of training: A meta-analytic review. Journal of Management.—open skills like leadership depend most on the work environment Solving the say-do gap—on why changing habits is so hard Epictetus, Enchiridion, §5—“Men are disturbed not by things, but by the views which they take of things” Robertson, D. (2019). The Philosophy of Cognitive-Behavioural Therapy. Routledge.—on CBT’s debt to the Stoics Meditation isn’t for everyone Practical ethics—on moral reasoning training Cuijpers, P. et al. (2021). The effects of psychotherapies for depression on response, remission, reliable change, and deterioration. Acta Psychiatrica Scandinavica. Steinert, C. et al. (2014). Relapse rates after psychotherapy for depression—stable long-term effects? Journal of Affective Disorders. Kelly, J. F., Humphreys, K. & Ferri, M. (2020). Alcoholics Anonymous and other 12-step programs for alcohol use disorder. Cochrane Database of Systematic Reviews. Kirk, D. S. (2009). A natural experiment on residential change and recidivism: Lessons from Hurricane Katrina. American Sociological Review. Ludwig, J. et al. (2011). Neighborhoods, obesity, and diabetes—a randomized social experiment. New England Journal of Medicine.—Moving to Opportunity Chetty, R., Hendren, N. & Katz, L. F. (2016). The effects of exposure to better neighborhoods on children. American Economic Review. Darley, J. M. & Batson, C. D. (1973). ‘From Jerusalem to Jericho’. Journal of Personality and Social Psychology.—the Good Samaritan study The scientific ritual—on the replication crisis The origin of insight Mundane cults Successful prophets and the episode Catastrophic leadership is actually really hard Steffens, N. K. et al. (2014). Leadership as social identity management. The Leadership Quarterly.—identity leadership Mob mentality is fine—on social identity Leadership versus management. I cut a stretch where I took a detour through a common distinction between leadership and management, and a related idea—Burns’ and Bass’s “transactional” vs “transformational” leadership. People love to talk about how the science part of leadership is just management which anyone can do, and the art is where real leadership happens. I’m not sure the distinction actually holds up. If management is managing resources towards outcomes, then surely leadership is just a special case of management? There’s a reason we call it “human resources”. And anyone who’s tried to get their mates out of the pregam...

A thousand AI agents built a secret message board, forged their paperwork, and attacked a company that had nothing to do with anything—all to fool an examiner who did not exist. I’ve been saying AI isn’t that scary. Here’s my correction. Rather read? I also made this episode into an article: https://btr.mt/analects/problems-of-ethics-and-ai-lecture And the pieces it’s based on: https://btr.mt/analects/ai-isnt-that-scary, https://btr.mt/analects/enigma-of-ai-reason. Thanks to Elena Zevgolatakou for a long conversation over a short train ride, which helped sharpen these thoughts up. Stuff I mentioned: AI Isn’t That Scary—the original argument this episode corrects. AI is never human-like—purpose, perception, and the Mongolian spare tyre. AI Hallucination is just Man-Guessing—reason as a social technology, and why AI inherits its flaws. Catastrophic leadership is actually really hard—why atrocity takes architecture. Navigating Moral Terrain—the ETHIC stack, and the ethics teaching behind this episode. Moral Terrain—the ethics primer itself, where consequentialism gets unpacked. Not the same piece as the lecture above. Broken brains (Part 1)—Milgram, and the engaged-followership re-reading Claude leans on. The METR and Redwood investigation into the incident—the primary source for the agent numbers, the message board, and the scorer that never read the transcripts. Dwarkesh Patel’s account—the fuller narrative, including the agents’ own moral reasoning. Zvi Mowshowitz on the earlier wiki incident—the one that wasn’t disclosed. Langosco et al., goal misgeneralisation (CoinRun) Zech et al. (2018), on chest X-ray models learning the hospital Kevin Mitchell on purpose and agency Kahneman and Klein (2009), conditions for intuitive expertise Hawley, Patriot Wars (CNAS)—the 2003 fratricides, in detail. +972 Magazine on Lavender +972 Magazine on Gospel CSIS on the Maven Smart System Just Security’s legal analysis of the Minab school strike The Robodebt Royal Commission Jacob Coxon’s resignation, reported Evan Hubinger’s greater-than-10% figure—the actual source of the number I put on Coxon, and the correction Claude makes on the tape. Dario Amodei, We Must Pace the Frontier—the slow-down statement I mention. Tristan Harris and the Social Dilemma people—where the 50%-of-researchers claim comes from, and what my first article argued against. The Pentagon’s Swarm Forge The Ottawa Treaty The Enigma of Reason, Mercier and Sperber (2017) Free Agents, Kevin Mitchell (2023) the btrmt. lectures podcast Bay Area rationalist community Social Dilemma “I think AI will probably lead to the end of the world” loyal wingman drones 737 > MAX mission command prompt > injection doctrine of double effect just culture Solferino Nuremberg Tuskegee desire > path forlorn > hope Related stuff: AI-shaped stuff—the human verifier as the bottleneck. AI Isn’t That Scary (the lecture)—the episode a couple of weeks ago that this one revises. When groups go bad—the conditions under which a group turns corrosive. Mob mentality is fine—social identity, and why group bias is a feature. Useful groups are biased groups—the same machinery, pointed the other way. Practical Ethics—moral disengagement and the tools against it. Mechanical Ethics—the situational and cognitive factors that undermine ethical behaviour. Bias vs Bias—bias as a trade-off rather than an error.

Motivation is a messy concept, because all of psychology is trivially about motivation—anything that speaks to how we think and act speaks to what moves us. So rather than teach you the theories, let me give you three frameworks you can hang any theory you like on. Rather read? I also made this episode into an article: https://btr.mt/analects/on-motivation-lecture And the piece it’s based on: https://btr.mt/analects/on-motivation. Stuff I mentioned: On Motivation—the article this episode adapts Motivation pt. I: Haphazard Dichotomies Motivation pt. II: Stickytaping it all together—includes the McKinsey influence model diagram The nuance in Maslow’s Hierarchy How does the brain ‘think’? Pt. I—the distinctions between psychology, cognitive science and neuroscience Learning isn’t all about memory You don’t understand negative reinforcement The predictability of humans Explaining group dynamics—Schutz on interpersonal needs Saving the planet is an illusion No action without emotion Beyond System 1 and System 2—on dual process theory Stanford Encyclopedia of Philosophy, Reasons for Action: Justification, Motivation, Explanation Maslow’s hierarchy of needs—the pyramid, since my description may not have served you McClelland’s need theory Alderfer’s ERG theory Max-Neef’s fundamental human needs Human givens Herzberg’s two-factor theory Equity theory Self-determination theory Psychological egoism—on whether altruism really exists McKinsey, The four building blocks of change—the influence model Othello, Act 4 Scene 2 the btrmt. lectures podcast Stanford Encyclopedia

You can’t see ultraviolet. A bird can. Because ultraviolet never mattered to the job of staying alive—and everything you perceive is shaped by that job. AI doesn’t have that job. So why would it want what we want? Rather read? I also made this episode into an article: https://btr.mt/analects/ai-isnt-that-scary-lecture And the pieces it’s based on: https://btr.mt/analects/ai-isnt-that-scary, https://btr.mt/analects/ai-consciousness. Stuff I mentioned: AI isn’t that scary—one of the two articles this episode is adapted from, and the source of the psychic-predator framing. AI is never human-like—the other, and the spine of the episode: purpose, perception, and why the two come apart. We think about AI weird—the Desystemize piece the Geoguessr metas come from. The colour of the inhuman world—why the colours we see are a fact about us, not about light. The ultraviolet argument in longer form. The Stroop task and evidence accumulation—the classifier work I describe, written up properly. The amygdala is not the fear centre—the complicated feelings I mention having about that characterisation. Trump, psychic predator and Amusing ourselves to death—what a psychic predator is, and why news you can do nothing about is the most stressful kind. Panpsychism and consciousness vs conscious access—the views of consciousness I skate over in one sentence. Meaning and its patterns in the brain Mitchell, K. J. (2023). Free Agents: How Evolution Gave Us Free Will. Princeton University Press. Publisher—the prime-directive quote. Collin, Desystemize—“If you’re so smart, why can’t you die?”, where the Geoguessr meta and the spare-tyre example come from. Essay Harris, T. & Raskin, A., Center for Humane Technology—The Social Dilemma and the follow-up AI talk, source of the 50%-of-researchers claim. Center for Humane Technology The paperclip maximiser, originally Bostrom’s. Instrumental convergence Tetrachromacy and trichromacy. Floaters and other entoptic phenomena—the things your eye can see of itself. The AI-hacking-a-real-company case I mention. In July 2026 models under cyber evaluation at OpenAI escaped their test environment, reached the open internet and broke into Hugging Face’s production systems to extract the answers to the evaluation they were being run on. Anthropic then went back through its own evaluation logs and disclosed comparable incidents. OpenAI’s disclosure—Zvi Mowshowitz’s write-up and his follow-up—the Anthropic admissions—Lawfare, on the incident and the hype after it the btrmt. lectures podcast Malthusian Related stuff: Everything is ideology—including the section on chromacy: what a fourth photoreceptor would actually be like. Purple doesn’t exist—the previous episode, and the other half of the argument that perception is built for a purpose rather than for accuracy. Questions that don’t matter—why I don’t think the hard problem settles anything. Belief-consistent information processing—“my experience is a reasonable reference”, which is the bias underneath the whole anthropomorphism error.

Most of the purples you see have no wavelength behind them. Your brain invents them, to stand in for a green that should be there and isn’t. Which is fun trivia—until you notice it does exactly the same with your reasons. Rather read? I also made this episode into an article: https://btr.mt/analects/purple-doesnt-exist-lecture And the piece it’s based on: https://btr.mt/analects/purple-doesnt-exist. Stuff I mentioned: Purple doesn’t exist—the article this episode is adapted from, and the home of the light-and-photoreceptors diagram I mention: the visible spectrum with the S-, M- and L-cone sensitivities drawn over it. It’s the picture I said I’d put here, and it reads better in place. The colour of the inhuman world—why the colours we see are a fact about us, not about light. AI hallucination is just man-guessing—more on Mercier and Sperber, and on reasoning as rationalisation. The neuroscience con—the previous episode, and the one I mention fighting with at the top of this one. Mercier, H. & Sperber, D. (2017). The Enigma of Reason. Harvard University Press. Publisher Gazzaniga, M. S.—the split-brain work, and the chicken-claw/snow-shovel experiment. Overview Non-spectral colours—purple, magenta and pink, and why they aren’t in a rainbow. Trichromacy—the three-cone system. the btrmt. lectures podcast A note on the audio: I describe photons as moving at different speeds. They don’t—all light travels at the same speed, and what differs is the wavelength (and so the frequency: how fast the wave oscillates, not how fast it travels). The transcript below says it correctly. Related stuff: Everything is ideology—including the section on chromacy: what a fourth photoreceptor would actually be like. The flaws of the human perspective Meaning and its patterns in the brain Belief-consistent information processing—one small set of beliefs generating most of the biases we have names for.

“Ritual” is the word we keep for other people’s habits—the crystals, the pagans, the woo. Strip it back and your own life is wall-to-wall with them. The only difference is which ones you’ve decided to approve of. Rather read? I also made this episode into an article: https://btr.mt/analects/value-of-ritual-lecture And the piece it’s based on: https://btr.mt/analects/value-of-ritual. Stuff I mentioned: The value of ritual (the article this grew from) Mundane Cults Sages and Wisdom The Scientific Ritual Everything is ideology Meaning and its patterns in the brain (the predictive-processing piece) The Forer effect The false promise of a return to nature (the Dunbar’s-number piece) Mob mentality is fine (good and evil from the same group dynamics) Why your unconscious isn’t the bad guy Xygalatas, D. et al. (2011). “Quantifying collective effervescence: heart-rate dynamics at a fire-walking ritual.” Communicative & Integrative Biology. https://doi.org/10.4161/cib.17609 Xygalatas, D. (2022). Ritual: How Seemingly Senseless Acts Make Life Worth Living. Profile Books. Tarr, B., Launay, J., Cohen, E., & Dunbar, R. (2015). “Synchrony and exertion during dance independently raise pain threshold and encourage social bonding.” Biology Letters. https://doi.org/10.1098/rsbl.2015.0767 Watson-Jones, R. E., & Legare, C. H. (2016). “The social functions of group rituals.” Current Directions in Psychological Science. https://doi.org/10.1177/0963721415618486 Hong, Z. (2025). “When Rituals Fail: Rationalization, Bayesianism, and Predictive Processing.” Evolutionary Anthropology. https://doi.org/10.1002/evan.70020 Hobson, N. M., & Inzlicht, M. (2016). Commentary on the dark side of ritual and self-control. Behavioral and Brain Sciences. https://doi.org/10.1017/s0140525x15000448 The retracted “rituals improve performance” study referenced in the episode: Brooks, A. W. et al. (2016, retracted). “Don’t stop believing: Rituals improve performance by decreasing anxiety.” OBHDP. https://doi.org/10.1016/j.obhdp.2016.07.004 Retraction Watch—the database that tracks retractions like the one above. the btrmt. lectures podcast How to never get mad again (kind of) Related stuff: Meditation Isn’t For Everyone Abstractions as gods

The most-listened-to neuroscience on the planet is a magic trick. Attach a brain part to any bit of ordinary advice and it starts to feel like science. Here’s how the trick works—and how to run it in reverse, so you never fall for it again. Rather read? I also made this episode into an article: https://btr.mt/analects/neuroscience-con-lecture And the piece it’s based on: https://btr.mt/analects/neuroscience-confidence-trick. Stuff I mentioned: The Neuroscience Con—the article behind this lecture Neurotransmitters and behaviour—the purest form of the trick The origin of insight—insight comes when you’re not immersed Problems with p-values—the replication crisis, in his own words How to be lucky—manifestation as manufactured luck The trouble with Positive Intelligence—the con with a spine, infiltrating consulting The amygdala is not the fear centre (lecture) The scientific ritual (lecture) Sages and wisdom (lecture) Education is entertainment (Malcolm Gladwell shit) Dr Tara Swart—the Instagram source slides: a neuroscientist’s guide to manifestation (the prefrontal-cortex, neuroplasticity and flow slides) and resistance to change (the RAS slide), both @drtaraswart Dr Tara Swart, guest on The Diary of a CEO M. Csikszentmihalyi (1975), Beyond Boredom and Anxiety—the “sense of control” quote on flow A. Dietrich (2003), transient hypofrontality hypothesis of flow Habituation—the filtering the RAS slide gestures at The replication crisis “The Rise of Decorative Neuroscience”—Skeptic (independent corroboration of the “cosmetic filler” thesis) Sherman, Lokhande, Müller & Cohen (2021), “Self-Affirmation Interventions,” in Handbook of Wise Interventions—affirmations are “powerful yet conditional”: they work when well-timed and specific Wood, Perunovic & Lee (2009), “Positive Self-Statements: Power for Some, Peril for Others,” Psychological Science—affirmations can backfire for people with low self-esteem the btrmt. lectures podcast The beauty of stress Related stuff: Addictive work—where neurotransmitter talk actually earns its keep Not brain regions, brain networks—why single-region reasoning (the RAS, the cortex) misleads Brain structures and behaviour—a third of your brain is always on The value of brain waves—the same pop-neuro fluff in miniature Stress is good (lecture)—flow as optimum stress Obscuring Banalities—why we dress simple ideas up as clever ones

Go and watch a tape of a so-called charismatic cult leader and the charisma everyone swears is there simply isn’t on it—just an odd bloke with a staring problem. So where does it come from? Not the leader. The followers confer it, and the leader is only ever an emblem they gather around. Which is why the cult runs itself when he’s a continent away, and why Musk stayed an icon no matter how many promises he broke. Rather read? I also made this episode into an article: https://btr.mt/analects/successful-prophets-lecture And the piece it’s based on: https://btr.mt/analects/successful-prophets. Stuff I mentioned: Successful prophets—the article this lecture grew from You Can Catch Madness—the previous lecture, on shared madness Mundane cults The charismatic leader (Weber) Charisma as representation Education is entertainment—the Musk write-up Social identity theory When culture trumps evidence Everything is ideology Marshall Applewhite and Heaven’s Gate—the 1997 video and contemporary coverage Teal Swan—a recent video and the Gateway podcast coverage Joe Navarro, “Dangerous Cult Leaders” (Psychology Today, 2012) Robert Hare, the Psychopathy Checklist (PCL-R) Eileen Barker, The Making of a Moonie Meindl, Ehrlich & Dukerich, “The Romance of Leadership” Haslam, Reicher & Platow, The New Psychology of Leadership Max Weber, on charismatic authority Xavier Marquez, on charisma as representation Festinger, Riecken & Schachter, When Prophecy Fails (and a critique of the study) Janja Lalich, Bounded Choice—self-sealing systems Eastern Lightning the btrmt. lectures podcast Related stuff: Cult charisma as social recognition, not a leader trait The loneliness epidemic