Episode 14: Results from continuous diff-in-diff!

After 16 episodes of working with Claude Code to study the effect of abortion clinic closures on marriage rates using a natural experiment, so to speak, in Texas called "House Bill 2" and continuous difference-in-differences with a treatment of "change in distance", Caitlin and I start our descent. Caitlin shares her own thoughts about returning a poisson estimator, and we continue to wrestle with the implications of differences in econometric estimators and interpretation, as well as what we are willing to live with. Plus a lot of discussion about the implications of AI for our lives as researchers manages to creep in its pretty little head!Scott's Mixtape Substack is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. Get full access to Scott's Mixtape Substack at causalinf.substack.com/subscribe

Caitlin Myers and myself have been studying the effect of House Bill 2 in Texas, which closed half the state's abortion clinics, on marriage rates using "continuous difference-in-differences" estimation for 14 episodes straight. But behind every good diff-in-diff estimate lurks a parallel trends assumption, and so we asked Hannah to look into one confounder -- oil! fracking! energy! Perhaps things related to technologies that enhanced the extraction of oil from the ground, like fracking technology, could have alone shifted marriage rates by bringing men into communities but only for work, temporary residence, not permanence. We talk about a literature where men in transit can alone reshape the marriage markets, which if those events happened disproportionately in our treatment counties, but also our control counties, then parallel trends mechanically *might* be violated. Because remember -- parallel trends is a) about things differentially impacting the treatment and control units on *outcome trends* and b) therefore can impact the treatment group *or* the control group. Hannah comes back with research she did for us on it.Scott's Mixtape Substack is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. Get full access to Scott's Mixtape Substack at causalinf.substack.com/subscribe

This week is a hoot! Caitlin and I finally get some estimates of the different target parameters using continuous treatment diff-in-diff! We discuss in detail how the estimator works and then go through our analysis. Watch to the end to see our reaction when we finally see some output in a beautiful deck!Scott's Mixtape Substack is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. Get full access to Scott's Mixtape Substack at causalinf.substack.com/subscribe

Caitlin and I are back after a one week hiatus as we each ran around traveling in our respective parts of the world. Probably for the best, as it allowed two weeks of twoway fixed effects decompositions to marinate. But now it’s time — can we finally see what a continuous treatment difference-in-differences estimator actually is for goodness sake? And the answer is sort of!In this episode, me and Caitlin wrap up a walk through of what parameters we are identifying with our abortion-marriage paper. I was really puzzled to be honest in the last episode as to what a “dose” even meant in our context. As you may recall, we are studying the effect of House Bill 2 which caused half of Texas’s abortion clinics to close, and in turn made the distance to the nearest abortion clinic to rise. But that led us to wonder:1. Are we studying the effect of distance to the nearest clinic after House Bill 2, or2. Are we studying the effect of the change in distance to the nearest clinic after House Bill 2?So, have fun as you listen to us talk through it out and finally realize at the end that it would appear our dose must be one of those and cannot be the other due to the nature of the design and diff-in-diff itself. Hint: no anticipation places some rails on us. See if you can figure out why.But then we also dive into the continuous treatment diff-in-diff estimator. You’ll learn about splines! You’ll learn about kernels! You’ll learn about polynomials! You’ll learn about b-splines and wavelets and a bunch of other things that draw curvy lines! And you’ll learn about the one situation when you have the permission to interpret that line as a causal effect too!Thanks again for all your support! We hope you enjoy this episode!Scott's Mixtape Substack is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. Get full access to Scott's Mixtape Substack at causalinf.substack.com/subscribe

In one sense, causal inference has two approaches. You can run a regression and then backwards engineer what it means. Think of Imbens and Angrist's 1994 classic Econometrica on the local average treatment effect (LATE) where they show that the Wald estimator (binary treatment, binary instrument) is the average effect for the complier subpopulation. But the other way that causal inference often runs is you start with the parameter of interest, not the regression, and then build the regressions to identify them under minimal but acceptable assumptions. In this episode of the Odd Couple, we switch from estimation to description of the causal parameters introduced in Callaway, Goodman-Bacon and Sant'Anna (2026, AER). These are the well known ATT parameter, but not the ACRT, which is the slope of the dose response curve. We also puzzle over whether our treatment is, in fact, distance measured in levels or is it distance measured as changes. Which is probably one of the values of starting with parameters: it forces you to figure out what your question is!Scott's Mixtape Substack is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. Get full access to Scott's Mixtape Substack at causalinf.substack.com/subscribe

Welcome to the 11th episode of The Mixtape with Scott, season 5, “The Odd Couple” featuring Caitlin Myers! This week we continue the riveting material from last week where we walked through a decomposition of the twoway fixed effects estimator when it’s 2 period, diff-in-diff with a continuous treatment! Yes, you heard me right — be still my beating heart. Scott's Mixtape Substack is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.Me and Caitlin continue to go through this deck that Claude made for us explaining the new Callaway, Goodman-Bacon and Sant’Anna paper, forthcoming at AER, about continuous treatment diff-in-diff. Mainly, though, we are just working our way painstakingly slow through this Frisch-Waugh-Lovell decomposition of the OLS regression to better understand just what OLS is doing.I thought this episode was pretty interesting though your mileage may vary. I mean, if you don’t find two economists trying to help each other understand an econometrics paper, then probably the floor on this episode could be a little low. But that said, I did enjoy it. We both really seemed to help one another better understand the decomposition formula, plus we got to see it with our own eyes. And Claude made some really intuitive graphics that helped both of us. So check it out! As always thanks for tuning in!Scott's Mixtape Substack is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. Get full access to Scott's Mixtape Substack at causalinf.substack.com/subscribe

Today will astonish and amaze because in this one, you will watch me explain the decomposition of the OLS twoway fixed effects estimator for the continuous treatment difference-in-differences! A first for podcast history I would be willing to bet! Thanks again for turning in! (Caitlin said she thought this turned out well, but your mileage may vary).Scott's Mixtape Substack is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. Get full access to Scott's Mixtape Substack at causalinf.substack.com/subscribe

This week's episode of "The Odd Couple" is just Caitlin and Hannah as I had to go to Georgetown to talk about Claude Code at a faculty retreat. But before we get going with a description, Hannah mentioned at the start during the ice breaker about the opening theme song to the podcast, and for those that don't recognize the lyrics, that's Mac Miller's "Small Worlds" sung by my two nephews. So what is this episode about? One of the themes I have been emphasizing in my talks on AI Agents and my substack is that AI Agents have caused a separation between the historic bundling of the production of research and the verification of the results. Since AI Agents are now able to produce so many aspects of the research project autonomously -- that is without much direction from the human researcher -- one of the new tasks of the researcher is to verify them. If you remember from a few weeks ago, Claude Code had nearly instantly worked up the county-level marriage data into a county panel of marriage rates and marriage counts by year. We brought Hannah Sayre, a recent college graduate and current economic consultant, into the project to help us work through the latter task of "human verification". Had Claude done it correctly? How do we verify that it is correct? And if it is not correct, why was it not correct, and how generalizable is that inaccuracy? Hannah was our eyes and ears, our boots on the ground, as she independently investigated the same question, the same task we gave Claude, to on the back end up help us determine whether Claude had indeed found the same irregularities in the original marriage dataset, and if so, what autonomous decisions had he made. And so in this episode, Hannah walks us through it, and she and Caitlin discuss both those findings, as well as begin the work of conceptualizing the process of verification in a world of AI Agents. While not definitive, this is a chance for others to hear more specifically about this. I at least anticipate that all of us will have to wrestle with verification going forward in ways we were not expecting, and maybe even are not prepared for, at least not universally, and definitely not necessarily if in fact AI Agents shrink the size of the project team members due to automation, and how best to respond to that smaller scale, and therefore, fewer people available to do the actual verification itself. Scott's Mixtape Substack is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.Thanks again for tuning in! We hope you are having as much fun with this as we are! Get full access to Scott's Mixtape Substack at causalinf.substack.com/subscribe

This week, Caitlin and Scott and Claude debate what a regression is, and then run some poisson regressions of county level marriages in Texas onto travel distance to the nearest county with an abortion provider! If that doesn’t get your heart beating fast, you should go to the doctor because you may be dead! This is a fun episode and hope you like it! Scott's Mixtape Substack is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. Get full access to Scott's Mixtape Substack at causalinf.substack.com/subscribe

As I’ve said on here, and others have said repeatedly, the bottleneck in research is probably less so production as it is verification now that researchers have access to AI Agents like Claude Code, Gemini and Codex. But how to verify, and what to verify, is largely something all researchers will have to bumble through themselves. We did it the old fashioned way — we asked Hannah to check Claude’s work by independently going through the marriage records we found herself. In today’s episode, Hannah comes back to tell us what she found. And after today, we are now one episode closer to actually running some regressions! Famous last words. Thanks again for your support of the Odd Couple podcast, as well as my substack. Scott's Mixtape Substack is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. Get full access to Scott's Mixtape Substack at causalinf.substack.com/subscribe