0:00
46:19
Rewind 15 seconds
Fast Forward 15 seconds

Relatively Human

Season 3, Episode 1: Right for the Wrong Reasons

When a prediction gets the right answer, how do you know why?

Season 3 starts here, with new voices and a new format from the same show creator: each episode takes an open question in science and works through what's known, how sure we are, and what would settle it. We open with an AI that seemed to read chest X-rays but was partly reading the word PORTABLE stamped in the corner. From there, we look at how right answers can be hollow: a Google flu tracker that learned winter as well as flu, research papers where the answer had leaked into the question, and a hospital sepsis alert that was partly predicting what doctors already suspected. Then we ask how you catch it. The answer is to change the conditions and see what breaks, and we meet a cat with elephant skin, a team that rebuilt a famous exam and found they'd been wrong, and the same chatbot entered twice on a leaderboard. Finally, the twist: by the old textbook rule, the biggest AI systems should be the worst offenders, yet they pass these tests better than expected, and the people who build them can't yet fully say why. We end where even they break, with a hurricane stronger than anything the model had ever seen, and on what's still open.

Chapters

00:00 Cold open: what was it looking at?

02:45 Welcome, and the map of the episode

04:37 Part one: a right answer can be hollow

18:03 Part two: how you catch it

30:01 Part three: better than the textbook said

41:28 Same math, or same mechanism?

44:07 What's still open

Nothing in this episode is medical or investment advice.

More episodes from "Relatively Human: Fundamental Laws of Biology and Physics"