What To Do After a Bad Incrementality Test Result

Squarespace Head of Marketing @Josh Groth has been on every side of the table. He’s been platform-side at Meta. He’s been B2B with Square, then DTC with Hims. And across with work with some of the biggest brands in the world, the playbook has evolved but one thing has remained a constant: intellectual curiosity.In this episode, Josh explains why the best question marketers can ask is “Why?” It’s also the question that inspires incrementality testing hypotheses that eventually turn into business-moving decisions. In this episode:What Josh learned platform-side at MetaHow Josh uses conversion lift studiesBig learnings from Josh’s time at SquareHow the playbook shifted jumping to HimsQuantity vs. quality in creativeThe #1 thing Josh screens for when hiring marketersAbout Josh:As Senior Vice President of Marketing, Josh Groth leads the strategies behind Squarespace’s media investment, CRM, and international expansion. Josh brings 18 years of experience scaling technology companies across both B2B and B2C. At Squarespace, his focus is on helping entrepreneurs worldwide turn their ideas into thriving businesses. Prior to joining the company, he served as VP, Growth at Hims and SVP, Global Performance Marketing at Square, where he established a global center of excellence for acquisition and creative production. His earlier career includes growth roles at Meta and Grove Collaborative. Josh is a firm believer that the most impactful growth happens at the intersection of rigorous data and genuine creative curiosity. He received his B.S. in Business Administration from the University of Oregon.About Haus:Haus is the causal marketing measurement platform built for performance-driven brands and enterprises. From GeoLift incrementality testing to Causal MMM, Haus helps marketing and finance leaders understand what's actually driving growth — so they can make smarter business decisions.🔗 Learn more about Haus: https://www.haus.io📅 Book a demo: https://www.haus.io/demo💡Follow Haus on LinkedIn:   / haus-analytics 

Brett Gordon is a Professor of Marketing at Stanford University's Graduate School of Business. He sat down with Haus’ Chief Scientist Joe Wyer, CMO Olivia Kory, and Head of Science Strategy Phil Erickson to discuss his landmark research around advertising measurement, including his “close enough” paper and “predictive incrementality with experimentation” (PIE) paper. (Both are essential reading around Haus — in fact, we hand it out to new hires.)In this episode, they dig in to:Brett’s background (1:04)Brett’s landmark “close enough” paper (5:09)Why marketing data is so bad (15:45)Brett’s “Predicted Incrementality by Experimentation” (PIE) paper (20:09)What marketers can learn from the PIE paper (24:34)Attribution is biased…but can we learn from that bias? (28:30)Generalizing PIE using experiments (31:06)About Brett:Brett Gordon is the Robert A. Magowan Professor of Marketing at Stanford University’s Graduate School of Business. His research focuses on pricing, advertising, promotions, retailing, and experimentation, using methods from causal inference, machine learning, and empirical industrial organization. He partners with companies to measure and enhance marketing effectiveness. His recent work examines digital advertising measurement methods while developing new evaluation approaches.He served as a co-editor at the Journal of Marketing Research from 2023 to 2026 and is the co-creator of the “How I Wrote This” podcast, which goes behind the scenes with authors to understand how great academic marketing papers came to be. Listen on Apple or Spotify.About Haus:Haus is the causal marketing measurement platform built for performance-driven brands and enterprises. From GeoLift incrementality testing to Causal MMM, Haus helps marketing and finance leaders understand what's actually driving growth — so they can make smarter business decisions.🔗 Learn more about Haus: https://www.haus.io📅 Book a demo: https://www.haus.io/demo💡Follow Haus on LinkedIn: https://www.linkedin.com/company/haus-analytics

You see success stories about brands that have leveraged incrementality testing to transform their business. While these are inspiring, they conceal an inescapable reality of executing a high-impact incrementality testing program: Sometimes, test results will bum you out. In our latest episode of Open Haus Playbook, Haus Measurement Strategists Dean Gordon and Ike Armstrong sit down to talk about that difficult moment when you get a test result that “punches you in the chin.” First, they outline ways to frame that poor test result in a way that's productive — not just to your marketing org, but also to the finance team and the C-suite. Then they lay out their playbook for next steps after a less-than-stellar result. They’ll share some tactical testing ideas that can build upon this baseline result and turn it from a disappointment to the start of improved media performance.In this episode:Intros, and defining “bad" results (1:22)The different types of disappointing test results (4:37)Reframing a “bad” test result (7:42)How do you communicate disappointing test results to finance? (13:04)Planning next steps after a bad test result (15:30)Strategy #1: The 1-2-1 approach (21:05)Strategy #2: The Firehose Approach (30:50)About Haus:Haus is the causal marketing measurement platform built for performance-driven brands and enterprises. From GeoLift incrementality testing to Causal MMM, Haus helps marketing and finance leaders understand what's actually driving growth — so they can make smarter business decisions.🔗 Learn more about Haus: https://www.haus.io📅 Book a demo: https://www.haus.io/demo💡Follow Haus on LinkedIn:   / haus-analytics 

Newton CMO Aaron Zagha was early to Amazon when DTC brands were still allergic to it. He was early to influencer marketing when nobody had figured out how to measure it. And he was early to incrementality testing when most CMOs were still trusting last-click attribution and in-platform ROAS.As CMO of Newton Baby, Aaron didn't just build a brand; he helped invent a category, convinced parents that purchasing a baby mattress required research and thought, and scaled the company to an acquisition by Resident (Ashley Home Furniture). Along the way, he quietly became one of the most analytically rigorous growth marketers in DTC.In this episode of Open Haus, Aaron joins Haus CMO Olivia Kory to walk through his full journey: from a finance background that gave him an unusual obsession with measurement, to the scrappy early days of Newton Baby, to their exit — all while taking us blow-by-blow through the experiments and hard-won lessons that made the difference. Aaron also explains how, with Haus, Newton Baby improved marketing efficiency by 30% over a three-year period, significantly boosting efficiency in a tumultuous macroeconomic environment.This is an episode for growth marketers who want to hear what it actually looks like to build a data-driven marketing org from the inside. Aaron’s story lays out just how important it is to see around corners and always remain rigorous (and unemotional) when it comes to figuring out what’s working and what isn’t. In this episode:Aaron's journey from finance to ecommerce (1:26)How Aaron "picked the right horse" with Newton Baby (4:13)The highlights and lowlights from Aaron's time at Newton Baby (8:43)Aaron's early big bet on influencer marketing (12:15)Newton's forward-thinking omnichannel sales strategy (15:56)Inside Newton's acquisition by Resident (22:00)Aaron's measurement journey with Haus (26:16)The efficiency gains Aaron unlocked with incrementality testing (34:50)Balancing optimizing for growth vs. efficiency (38:43)The importance of building a strong customer advisory board (41:49)About Newton:Newton Baby strives to revolutionize the landscape of infant sleep by offering a range of meticulously crafted, healthy, and safe sleep products. At the forefront of their innovation is the creation of the world's first 100% breathable, washable, and recyclable crib mattress. Beyond crib mattresses, their commitment to excellence extends to a diverse array of offerings, including furniture, bedding, and even pet beds. Newton Baby ensures that their products not only meet the highest standards of safety but also cater to the varied needs of families. Operating through direct-to-consumer channels and leveraging the expansive reach of Amazon, Newton Baby brings their transformative sleep solutions to a broad audience, making a positive impact on the well-being of infants and families alike.About Haus:Haus is the causal marketing measurement platform built for performance-driven brands and enterprises. From GeoLift incrementality testing to Causal MMM, Haus helps marketing and finance leaders understand what's actually driving growth — so they can make smarter business decisions.🔗 Learn more about Haus: https://www.haus.io📅 Book a demo: https://www.haus.io/demo💡Follow Haus on LinkedIn:   / haus-analytics 

We brought back Dean Gordon to break down the spend-level tests that help you find the marginal efficiency of your marketing. If you like to roll up your sleeves and get nerdy with test design, this is the episode for you. Table of contents: – What is a spend-level test? (1:45)– What are the different types of spend-level tests? (5:42)– The most scientifically rigorous spend-level test (8:21)– A spend-level test that minimizes account disruption (10:39)– The easiest spend-level test to pull off (15:10)– The least disruptive spend-level test of all (17:12)And if you missed Dean's first episode all about channel diversification, watch it here: https://www.youtube.com/watch?v=4kWmBbEwdBoThis is our inaugural episode of Open Haus Playbook — bite-sized episodes that get tactical about more advanced measurement concepts. If there’s a topic you’d like us to cover, don’t hesitate to let us know.

In our latest episode of Open Haus, CEO Zach Epstein, Chief Strategy Officer Olivia Kory, and VP of Product Greg Dale sit down to talk about the next evolution of the Haus platform. First, Olivia outlines some of the unique challenges marketers face today and how the agentic era presents both opportunities and further complications.Then, Zach and Greg dig into the what Haus has been building to give marketers the systems they need to win the agentic era. A lot of ground is covered — here's the breakdown:- Why marketing is harder than ever (2:09)- It’s a new era at Haus — here’s what that means (7:19)- Introducing Architect (11:00)- How signal, context, and action work together (19:55)- Haus ColdStart (23:43)- Causal MMM expansion (25:23)- Incrementality Index (28:58)- Causal Attribution (31:08)- Haus MCP (36:16)- More detail on Architect (37:55)- Common objections to agentic decisioning (47:15)- Q&A (50:35)

Faith Shin (Sr. Machine Learning Manager, Wayfair) and Joe Meier (VP of Marketing Analytics, SharkNinja) sit down with Olivia to discuss what it takes to build an elite marketing program. They break down:– The good, the bad, and the ugly of building marketing measurement in-house, feat. Haus' Joe Wyer (5:14)– How Wayfair's measurement has evolved in Faith's tenure (13:20)– The moment Wayfair realized they needed a measurement partner (17:16)– How having a partner has helped Wayfair scale (20:14)– SharkNinja's approach to measuring a massive product portfolio (23:05)– The SharkNinja bias for action (26:15)Plus an extensive live Q&A with Faith and Joe.

Olivia is joined by Haus Measurement Strategist Chandler Dutton to react to the Marketing Operators’ recent episode on incrementality. Olivia and Chandler talk: Incrementality in Retention (4:00)Using Haus to fend off vendors asking you for more money (11:52)Articulating a hypothesis (13:47)How to run a diminishing marginal returns test (16:45)Progressive truth and tactical A/B tests (18:47)Upper-funnel Measurement and Long-Term Effects (26:55)Interaction effects between channels (36:30)Creative wins (39:25)If you haven’t heard the Marketing Operators on incrementality, give it a listen here: https://www.youtube.com/watch?v=ckPtmcHacrMComing up: The Haus Growth Lab — a work session for the most forward-thinking growth operators in our community. Hear keynotes from Zach and Olivia on the future of marketing, and sit in on a roundtable with growth leaders from Wayfair, Fox, Equinox, and more. Plenty of time to chat and meet other growth leaders over breakfast and lunch.LA - 5/19 | NY - 5/21RSVP here: https://www.haus.io/growth-lab-lahttps://www.haus.io/growth-lab-ny

Olivia is joined by Haus Measurement Strategist Dean Gordon for a deep dive into channel diversification. Dean explains:Why brands want to diversify (3:54)The important distinction between average and marginal returns (7:15)The three incrementality tests that help you calculate marginal efficiency (11:38)The pros and cons of these different approaches to calculating marginal efficiency (14:47)The consequences of not understanding channel saturation points (18:58)The importance of patiently improving your channels (22:56)How to think about the scalability of secondary channels (23:17)How MMM can supercharge your diversification efforts (28:40)A summary of Dean's diversification framework (33:12)

The gap between having an MMM and actually operationalizing it is one of the biggest challenges in modern marketing measurement. Models produce outputs — but what happens next? How do teams turn those outputs into real budget decisions?In this live Open Haus session, we're bringing together the science and the practice.Phil Erickson, Haus' Head of Science Strategy, will kick things off with a look at the state of MMM in 2026 — the latest science, the emerging methodologies, and why experiments remain the critical missing ingredient for most MMMs.Tyler Horner, Manager of Solutions Consulting at Haus, will break down what it actually looks like to operationalize MMM: how to move past triangulation toward a unified source of truth, and how leading brands are putting model insights into action.Then, marketing leaders from StockX and OluKai will share their real-world experiences operationalizing Causal MMM — what's working, what they've learned, and what they’re planning next.They covered a lot of ground, including:The State of MMM in 2026 — and how it’s evolved (4:00)How we’ve evolved our use of experiments in MMMs (10:17)Adjusting experiments for seasonality so your model isn't biased (20:10)A framework for next steps based on cost curves (22:38)How StockX operationalized MMM and experiments (28:49)How OluKai reduced CAC by 20% overnight by following Causal MMM (37:07)AMA (46:14)