#41: What to do with Outliers

In part 2 of this saga on extreme classification, we get into the weeds on how MACH is able to magically handle such massive classification problems. The title says it all -- hash functions are the magical ingredient. We provide a step-by-step view of how one might come up with the MACH algorithm from first principles. 

In this episode, Dr. Derek Feng drops by to chat about a recent paper on a divide-and-conquer approach (Merged-Averaged Classifiers via Hashing) to massive classification problems. In part 1 (of 2 episodes), we describe the general problem solved by and strategy taken by MACH, wherein the original large classification problem is broken down into smaller-sized classification problems. Next week in the second episode, we talk about more technical details of how the division of labor works, and why it works.

We've long heard about the waves that statistics has made in baseball. But what about soccer? In this episode, we summarize a few applications of statistics in European football (or American soccer). 

Domoic acid has plagued shellfish and other wildlife along the Pacific coastline in recent years. Testing for domoic acid concentration in crabs on a regular basis has become important for determining when crabs and their viscera can be safely consumed. Unlike many other common hypothesis tests, the setup used for domoic acid testing is based on the sample maximum rather than the sample mean. In this episode, we critique the testing methodology. 

In this episode, we discuss how two statisticians used data from BoardGameGeek.com to put together their own board game recommendation engine, specifically designed to stay away from mainstream recommendations.

In this episode, Dr. Derek Feng discusses the general issue of data privacy in the age of big data, including topics of differential privacy and federated learning.

We sit down with Dr. Jon Krohn to chat about his work as a Chief Data Scientist at untapt, his newly published bestseller "Deep Learning Illustrated", and his teaching/research.

In this episode, Professor Albert Y. Kim of Smith College describes his post-PhD journey, which included a stint at Google Adwords before academic posts at Reed College, Middlebury College, Amherst College, and Smith College.

Dr. Derek Feng joins us again to discuss the two metrics by which we align all statistical/machine learning methods -- interpretability versus predictive ability. In a world where black box methods reign supreme, what does learning mean?

Guest Dylan O'Connell joins us today to talk about a recent surprising, but legitimate Democratic primary poll result done by Monmouth University. We discuss different perspectives on how to approach a data point that doesn't fit in with the others.