📖 Read
Why AI's "tangled black box" may have been a blurry microscope, and how clean, branching neurons change the story
For ten years, the people who study artificial intelligence have been telling us the same story. Inside every large neural network, they said, there is a knot. Concepts crammed into the same few neurons, jammed together to save space, so snarled that no human could ever read them.
We believed it because it felt true. Of course the machine is a mess. We built something enormous and strange, fed it the entire internet, and asked it to figure things out. Why would it be tidy?
Now one of the researchers who helped tell that story has come back to say: look again.
Reference
Neural Networks Are Already Clean, by Nick Cammarata | Goodfire
Chapters
00:00 Welcome to Heliox and Thanks to PRX Stations
02:03 Today’s Mission: Neural Networks Are Already Clean
02:53 The Superposition Myth Explained
04:46 The Dead Ends of Mechanistic Interpretability
09:46 Sparse Autoencoders and the 2019 Car-in-Dog Puzzle
11:51 The Broken Microscope: Feature Visualization
14:45 The Missing Express Lane: Residual Streams
17:42 The V-Shape Revelation
19:42 The Negative Pole: Round vs. Pointed
21:43 Recursive Generalization: The Tree Metaphor
22:47 The IY Neuron: From Letters to “Do It Yourself”
25:55 The Planetary Geology Neuron
27:49 One Learning Principle: The Branching Rainforest
28:41 Enter the Grassmannian Manifold
33:20 Geodesics and Canonical Angles
35:41 Proving It with Canonical Correlation Analysis
38:27 The Noisy Tail Is Nuance
40:04 The Missing Curves Stress Test
43:16 Do Language Models Hold Up?
44:52 The Colorado Comma Neuron
47:05 Who Stole the Book? Scar and Santa
50:22 The Ecology of Circuits and Gradient Starvation
52:25 The Umbrella-Into-a-Bee Experiment
54:16 Riemannian Optimization and Retractions
57:45 Why Networks Are Already Clean
58:50 The Final Question: Is Human Thought Geometric?
1:00:27 Credits and Outro
This is Heliox: Where Evidence Meets Empathy
Independent, moderated, timely, deep, gentle, clinical, global, and community conversations about things that matter. Breathe Easy, we go deep and lightly surface the big ideas.
Disclosure: This podcast uses AI-generated synthetic voices for a material portion of the audio content, in line with Apple Podcasts guidelines.
We make rigorous science accessible, accurate, and unforgettable.
Produced by Michelle Bruecker and Scott Bleackley, it features reviews of emerging research and ideas from leading thinkers, curated under our creative direction with AI assistance for voice, imagery, and composition. Systemic voices and illustrative images of people are representative tools, not depictions of specific individuals.
We dive deep into peer-reviewed research, pre-prints, and major scientific works—then bring them to life through the stories of the researchers themselves. Complex ideas become clear. Obscure discoveries become conversation starters. And you walk away understanding not just what scientists discovered, but why it matters and how they got there.
Independent, moderated, timely, deep, gentle, clinical, global, and community conversations about things that matter. Breathe Easy, we go deep and lightly surface the big ideas.
Spoken word, short and sweet, with rhythm and a catchy beat.
http://tinyurl.com/stonefolksongs
Mais episódios de "Heliox: Where Evidence Meets Empathy 🇨🇦"
Não percas um episódio de “Heliox: Where Evidence Meets Empathy 🇨🇦” e subscrevê-lo na aplicação GetPodcast.
