Cortex Compute podcast

Cortex Compute 009

20/11/2026
0:00
22:30
Manda indietro di 15 secondi
Manda avanti di 15 secondi

Episode overview

On October 28, Rust’s team posted a preprint in which 15 large language models caught 53% to 96% of major errors planted in clinical trial reports, against 27% for human reviewers. Weeks earlier, OpenAI had released a batch of AI-produced math results that mathematicians expect to take years to check. Lennart and Vugar ask whether journals should require an AI error check before publication, whether editors prefer AI reviews for their science or their packaging, and how anyone can trust a discovery they can’t follow.

Takeaways

* AI catches errors that reviewers miss. Lennart cites Rust’s preprint, where models caught 53% to 96% of major errors and humans caught 27%, though the team planted the errors and the human scores came from an earlier study.

* Make the error check a prerequisite. Vugar asks whether journals should require authors to run an AI-assisted error check on their articles before they submit.

* Build your own reviewing skill. Lennart had AI interview him about what makes a good paper and used Skill Creator to turn his answers into a “fake Lennart” review skill that catches more than he does but inflates small errors.

* Editors can fall for packaging. In Gary King and Steven Worthington’s study, former editors preferred the AI review in 26 of 30 cases although some AI reviews still had factual errors, and Lennart asks whether science is getting better or reviews only sound smarter.

* AI could narrow the gap between universities. Vugar asks whether AI helps small universities compete with well-funded ones, and Lennart expects the well-funded ones to resist anything that scratches at their monopoly.

* Tinkering is how you learn. Vugar built many websites that never earned him a penny, and Lennart says his week of fiddling with CSS taught him more than the hour Claude and Impeccable needed to put his site live.

* If AI explains AI, who decides? Lennart warns that leaning on AI to explain results like the AI proof against the exponential time hypothesis means “relying on the crutch to get over the crutch,” and Vugar ties the question to the law’s search for truth.

Timestamps

* [00:00:00] Welcome. Lennart and Vugar open Cortex Compute 009 on AI that reviews science and proposes discoveries.

* [00:00:10] Who checks AI’s work? OpenAI releases a batch of math proofs, and mathematicians say checking them will take years.

* [00:01:40] Rust’s error test. 15 models caught 53% to 96% of planted major errors, against 27% for human reviewers.

* [00:02:30] The journals’ double standard. Journals ban AI in peer review but run AI on everything else in your paper.

* [00:02:54] Do we still need the paper? Lennart asks whether a GitHub repository with an interactive overview could replace it.

* [00:03:53] Lennart’s Insight Incubator membership helps you make sense of AI and outlast the hype.

* [00:04:14] AI on both ends. Vugar pictures papers written with AI and read through AI, and asks what human creativity remains.

* [00:05:29] Vugar’s two questions. Should journals require an AI error check, and can AI help small universities compete?

* [00:06:14] Rankings and monopolies. Lennart sees Asian universities pulling ahead in the rankings.

* [00:06:34] A reviewing skill for every journal. Lennart’s “fake Lennart” skill, built with Skill Creator, pre-reviews his students’ papers.

* [00:07:40] It catches more than I do. AI nickels and dimes the small errors but still flags more than Lennart can.

* [00:08:53] Junior to senior, faster. The jump works only for grad students who accept friction and read the papers.

* [00:09:28] AI reviews beat human reviews. Gary King and Steven Worthington compared 124 manuscripts and 134 human reviews.

* [00:10:22] Challenge the draft, not the decision. AI can’t replace the associate editor’s call.

* [00:11:08] Better science or smarter-sounding reviews? People prefer the packaging over the content.

* [00:12:00] Packaging versus process. Lennart drops his CV into Claude, runs Impeccable, and has a website live in an hour instead of a week.

* [00:13:17] Vugar’s lost websites. Most of the sites he built with Wix are gone, but he learned a lot building them.

* [00:15:09] From WordPress to one hour. Vugar’s friend built a professional website in one hour the night before.

* [00:15:59] Why learn the old way? Lennart wants to learn and grow, and compares books with social media and Netflix.

* [00:17:03] Tinkering and the water cooler. Frictionless work removes the joy and the serendipity, like meeting a future investor at a wedding.

* [00:18:25] Catching up with AI. An AI proves the exponential time hypothesis false, and humans need time to check the proof.

* [00:19:42] AI to explain AI. If we need AI to understand AI’s answer, do we still make our own decisions?

* [00:20:27] How do we trust it? The scientific method used to settle this, but some discoveries now outrun it.

* [00:21:11] Truth, fate, and the law. Vugar links the question to the law’s search for truth.

* [00:21:49] Close. Lennart asks where the law stands when we can’t tell truth from fakeness, and invites listeners to subscribe.

00:00 Welcome 00:10 Who checks AI’s work? 01:40 Rust’s error test 02:30 The journals’ double standard 02:54 Do we still need the paper? 03:53 Insight Incubator break 04:14 AI on both ends 05:29 Vugar’s two questions 06:14 Rankings and monopolies 06:34 A reviewing skill for every journal 07:40 It catches more than I do 08:53 Junior to senior, faster 09:28 AI reviews beat human reviews 10:22 Challenge the draft, not the decision 11:08 Better science or smarter-sounding reviews? 12:00 Packaging versus process 13:17 Vugar’s lost websites 15:09 From WordPress to one hour 15:59 Why learn the old way? 17:03 Tinkering and the water cooler 18:25 Catching up with AI 19:42 AI to explain AI 20:27 How do we trust it? 21:11 Truth, fate, and the law 21:49 Close

Links and mentions

People

* Lennart Nacke (host)

* Vugar Ibrahimov (co-host)

* Ruslan Rust, senior author of the error-detection preprint (@rust_ruslan)

* Gary King, director of Harvard’s Institute for Quantitative Social Science

* Steven Worthington, director of data science at Harvard’s Institute for Quantitative Social Science

Books, essays, and papers

* Dominikus Brian, Xavier Isidro Pereira Hernández, Hao Yin, Jeffrey Koury, and Ruslan Rust, “Large language models detect methodological errors that human peer review misses in clinical trials” (medRxiv preprint, October 28, 2026)

* Gary King and Steven Worthington, “AI Article Reviews Dominate Human Peer Reviews at Their Best: Evidence and Policy Choices” (Harvard working paper, 2026)

* OpenAI, “Sharing AI progress in mathematics” (October 6, 2026)

* OpenAI, “Working with academic communities to build scientific understanding” (October 26, 2026), which announces a refutation of the exponential time hypothesis

* Claude (Anthropic), “A 2o(n) Time Algorithm for 3-SAT” (paper and Lean proof on GitHub), Anthropic’s refutation of the exponential time hypothesis

Tools and companies

* Anthropic, Claude, and Skill Creator

* Impeccable, design skill for AI coding tools

* OpenAI

* GitHub

* Wix

* WordPress and Joomla

* Netflix

* X

Mentions

* Want to dive deeper into where AI fits in your own work? Apply for Lennart’s one-on-one coaching.

* Insight Incubator, Lennart’s membership for making sense of AI

* Lennart’s post on X about AI peer review

* The exponential time hypothesis, a conjecture about how fast hard logic problems can be solved

* Earlier Cortex Compute episodes on friction in learning and on serendipity

Over to you

Listen to the full conversation, then subscribe to Cortex Compute on your favourite podcast platform and send this episode to a colleague who reviews for a journal that bans AI. For weekly notes on writing and AI, join the newsletter.



This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit lennartnacke.substack.com/subscribe

Altri episodi di "Cortex Compute"