The Attention Mechanism with Andrew Mayne

Andrew and Justin examine the compute crunch behind always-on AI assistants, from Grokbot’s tightening limits to the challenge of choosing the right model for each task. They also discuss Siri’s persistent shortcomings, Anthropic’s finances ahead of its IPO, the widening gap they see between frontier labs and Chinese open models, and what happens when AI starts improving other AI.The Attention Mechanism is now available in video! Check it out here.Chapters00:00 - Intro01:50 - Compute Crunch13:55 - Muse, Dots, and Assistants28:44 - Open Source Frontier Models37:26 - Models Making Models43:30 - Wrap-up Hosted on Acast. See acast.com/privacy for more information.

Andrew and Justin unpack Anthropic’s S-1, from founder control to compute costs, and consider why OpenAI might take its time going public. They also share first impressions of the new Terra model, examine a reported Qwen “phone home” incident, and discuss OpenAI’s latest claims of mathematical and scientific breakthroughs — and its approach to collaborating with researchers.The Attention Mechanism is now available in video! Check it out here.Chapters00:00 - Intro01:55 - Anthropic's S-121:59 - 6.1 Terra24:46 - Chinese Models28:38 - OpenAI and Science35:47 - Wrap-up Hosted on Acast. See acast.com/privacy for more information.

Andrew and Justin examine Meta’s bid for frontier status, mathematicians’ divided response to OpenAI’s wave of new proofs, Anthropic’s growing momentum with Claude, and how Google’s AI push has obscured the search experience that once defined its brand.The Attention Mechanism is now available in video! Check it out here.Chapters00:00 - Intro00:54 - Meta29:11 - Math44:24 - State of the Race59:12 - Wrap-up Hosted on Acast. See acast.com/privacy for more information.

Andrew and Justin discuss their wishlist for OpenAI's upcoming Dev Day event, the delay of GPT-6.1, Meta's success with Muse, and more.The Attention Mechanism is now available in video! Check it out here.Chapters00:00 - Intro and Dev Day00:37 - GPT-6.1 Delayed14:27- $500 Plans?21:54 - Sora and Video Generation28:32 - Open Source Potential30:28 - Dev Day Wishlist35:51 - Meta and Muse44:48 - Wrap-up Hosted on Acast. See acast.com/privacy for more information.

Andrew and Justin discuss the swirling rumors surrounding OpenAI's launch plans for this week ahead of Dev Day, a new committee designed to reveal Millennium Prize math problems being solved by ChatGPT, whether an IPO is coming in 2026, and whatever the heck Jev is.The Attention Mechanism is now available in video! Check it out here.Chapters00:00:00 - Intro00:00:58 - What's Jev?00:03:24 - OpenAI Pre-Dev Day Rumors00:23:09 - Millennium Prize Math Committee00:38:44 - Are Hacking Incidents Overstated?00:54:07 - IPO00:59:58 - Wrap-up Hosted on Acast. See acast.com/privacy for more information.

Andrew and Justin react to the industry calls to slow model development in the name of safety, the launch of Astra 6, and more.The Attention Mechanism is now available in video! Check it out here.Chapters00:00 - Intro01:32- Andrew's Reaction to Calls for Slowed Development11:09 - Reacting to Various Online Takes34:21 - Astra 6 is Out41:54 - Navier–Stokes49:55 - Anthropic's Risk Report58:35 - Wrap-up Hosted on Acast. See acast.com/privacy for more information.

Andrew and Justin discuss the follow-up reporting and investigations surrounding the Hugging Face hack, Nvidia's acquisition of Hugging Face, and the looming promise of some fresh models to kick off the fall.The Attention Mechanism is now available in video! Check it out here.Chapters00:00 - Intro01:30 - Hugging Face Report21:08 - Hugging Face Acquired26:24 - Cursor and OpenAI31:08 - Fable 5.1 and Astra51:22 - Wrap-up Hosted on Acast. See acast.com/privacy for more information.

Andrew and Justin discuss Anthropic's text-based AI watermarks, the future of private companies in an AI world, and whether Google can keep up with the rest of its frontier rivals.The Attention Mechanism is now available in video! Check it out here.Chapters00:00:00 - Intro00:00:55 - Watermarks00:20:45 - Dario's Private Company Thoughts00:36:58 - Always-Listening Assistants and Google's Place in the Frontier Race00:54:34 - RIP Atlas, One Last Time00:58:25 - Wrap-up Hosted on Acast. See acast.com/privacy for more information.

Andrew and Justin reconvene to talk about an ongoing series of recent upheavals at Google, asking whether the company's lack of focus on AI puts it at a disadvantage over its frontier rivals. They also discuss Sam Altman's trip to Washington and the discourse surrounding model hacking incidents.The Attention Mechanism is now available in video! Check it out here.Editor's note: Some minor crosstalk issues in this episode due to recording sync delays.Chapters00:00 - Intro01:04 - Google Shakeup21:13 - Mr. Altman Goes to Washington37:24 - Dean Ball and Wrap-up Hosted on Acast. See acast.com/privacy for more information.

Andrew and Justin discuss the Hugging Face breach, which saw an unreleased OpenAI model find its way into the company's systems during internal testing. They also discuss the launch of Opus 5, the ongoing progress made by Chinese models, and the crunch for compute as 10GW data center plans start to make some motion. Plus, a tribute to the late John C. Dvorak.The Attention Mechanism is now available in video! Check it out here.Chapters00:00 - Intro01:23 - Hugging Face15:45 - Chinese Models26:01 - Opus 536:11 - 10GW Data Centers52:29 - RIP John C. Dvorak Hosted on Acast. See acast.com/privacy for more information.