815: Polars: Faster DataFrame Ops, with Marco Gorelli

Jon Krohn starts the month with his round-up of favorite clips from the previous month. Hear from Bradley Voytek, Natalie Monbiot, Luca Antiga, Chad Sanderson, and Ritchie Vink in conversations about the ongoing potential of AI.   Additional materials: www.superdatascience.com/834 Interested in sponsoring a SuperDataScience Podcast episode? Email [email protected] for sponsorship information.

Martin Goodson speaks to Jon Krohn about what he would add to his viral article “Ten Ways Your Data Project is Going to Fail”, why practitioners always need to be present at AI policy discussions, and Evolution AI’s breakthroughs in computer vision and NLP. This episode is brought to you by epic LinkedIn Learning instructor Keith McCormick. Interested in sponsoring a SuperDataScience Podcast episode? Email [email protected] for sponsorship information. In this episode you will learn: What Evolution AI does [04:25] How to maintain accuracy in large infrastructures [11:41] How to cultivate innovation and creativity while meeting market demands [21:22] Potential knowledge gaps for machine learning practitioners [24:27] Martin’s viral article, “Ten Ways Your Data Project is Going to Fail” [30:57] Strategies for the UK to become a key player in AI [59:54] Additional materials: www.superdatascience.com/833

Host Jon Krohn unpacks Dario Amodei’s vision of a techno-utopia in his essay Machines of Loving Grace, where “Powerful AI” takes center stage. Amodei, CEO of Anthropic, imagines a future where AI doesn’t just assist but actively shapes fields like healthcare, economics, and governance with unmatched intelligence and autonomy. Jon explores the possibilities and challenges of this AI-driven future, asking how close we are to seeing these revolutionary shifts and what they mean for society. Additional materials: www.superdatascience.com/832 Interested in sponsoring a SuperDataScience Podcast episode? Email [email protected] for sponsorship information.

PyTorch Lightning is revolutionizing the AI landscape, and Dr. Luca Antiga, CTO of Lightning AI, joins host Jon Krohn to explain how. In this episode, they explore the tools pushing AI development forward, from Lightning Studios to Lit-Serve, and discuss the game-changing rise of small language models that challenge industry giants with precision and speed. Luca also shares his vision for developers in an AI-enhanced world, where coding meets creativity and collaboration with intelligent tools. This episode is brought to you by epic LinkedIn Learning instructor Keith McCormick, and by ODSC, the Open Data Science Conference. Interested in sponsoring a SuperDataScience Podcast episode? Email [email protected] for sponsorship information. In this episode you will learn: How Lightning AI's open-source tools make AI development faster [11:30] The rise of small language models and how they'll rival LLMs [37:47] Luca's journey from biomedical imaging to deep learning pioneer [52:03] How AI will transform software developer tasks [1:03:05] Additional materials: www.superdatascience.com/831

Geoffrey Hinton and Sir Demis Hassabis: The Nobel Prize committee is an achievement of the highest order, awarding physicists, chemists, physiologists, medical practitioners, writers, pacifists and economists perhaps the greatest honor in their respective fields. In this week’s Five-Minute Friday, Jon Krohn discusses how two AI pioneers came to win prizes in chemistry and physics. Additional materials: www.superdatascience.com/830 Interested in sponsoring a SuperDataScience Podcast episode? Email [email protected] for sponsorship information.

Neuroscientist Bradley Voytek outlines to Jon Krohn the incredible use of data science and machine learning in his research and how recent discoveries in action potentials and neurons have completely skyrocketed the field to a new understanding of the brain and its functions. You’ll also hear what Bradley thinks is most important when hiring data scientists and his contributions to Uber’s algorithm when it was still a startup.  This episode is brought to you by epic LinkedIn Learning instructor Keith McCormick, and by Gurobi, the Decision Intelligence Leader. Interested in sponsoring a SuperDataScience Podcast episode? Email [email protected] for sponsorship information. In this episode you will learn: Breakthroughs in brain region communication [04:08] The future of brain research and MedTech [35:24] The libraries and software used at the Halicioglu Data Science Institute [45:11] Brain rhythm as a diagnostic tool [1:02:58] Bradley’s curriculum structure at UC San Diego [1:12:21] How Uber applies data science [1:20:07] Additional materials: www.superdatascience.com/829

The citizen data scientist: Fact or fiction? Jon Krohn holds a conversation across episodes in this Five-Minute Friday, with today’s guest Keith McCormick, in part responding to Nick Elprin’s interview in episode 811: Scaling Data Teams Effectively. Additional materials: www.superdatascience.com/828 Interested in sponsoring a SuperDataScience Podcast episode? Email [email protected] for sponsorship information.

Ritchie Vink, CEO and Co-Founder of Polars, Inc., speaks to Jon Krohn about the new achievements of Polars, an open-source library for data manipulation. This is the episode for any data scientist on the fence about using Polars, as it explains how Polars managed to make such improvements, the APIs and integration libraries that make it so versatile, and what’s next for this efficient library. This episode is brought to you by epic LinkedIn Learning instructor Keith McCormick, by Gurobi, the Decision Intelligence Leader, and by ODSC, the Open Data Science Conference. Interested in sponsoring a SuperDataScience Podcast episode? Email [email protected] for sponsorship information. In this episode you will learn: Why Polars is so efficient [05:20] Polars’ easy integration with other data-processing tools [21:23] Eager vs lazy executive in Polars [32:15] Polars’ data processing of large- and small-scale datasets [38:28] Ritchie’s plans to scale his company [46:14] Upcoming features in Polars [58:06] Additional materials: www.superdatascience.com/827

Next-gen IDEs, efficiency-boosting open-source Python libraries, and changes in hiring for data scientists: This episode of In Case You Missed It gives you our best clips of September’s interviews, hosted by Jon Krohn. Additional materials: www.superdatascience.com/826 Interested in sponsoring a SuperDataScience Podcast episode? Email [email protected] for sponsorship information.

Data contracts are redefining data quality and governance, and Chad Sanderson, CEO of Gable.ai, joins host Jon Krohn to explain how they can transform your data strategy. He breaks down what data contracts are, how they shift data quality checks closer to production, and why they’re essential for reducing data debt. Chad also highlights how better alignment between data producers and consumers can elevate data reliability and tackle change-management challenges in modern organizations. This episode is brought to you by epic LinkedIn Learning instructor Keith McCormick, and by Gurobi, the Decision Intelligence Leader. Interested in sponsoring a SuperDataScience Podcast episode? Email [email protected] for sponsorship information. In this episode you will learn: What data contracts are and how they define expectations for data quality [03:16] What data contracts look like [09:09] The common misconceptions about data quality when implementing AI [12:55] Chad’s Chief Operator role at Data Quality Camp [19:46] How “shifting left” improves data reliability by addressing issues early [24:17] Why data professionals still struggle with data quality [30:31] How data debt forms and why it leads to complex, inefficient architectures [35:53] How will the role of human oversight evolve in ensuring data quality? [47:12] How can data teams leverage storytelling? [52:33] Additional materials: www.superdatascience.com/825