Outperform

Most experimentation programs don't stall because the stats are wrong. They stall in the room when "the data says so" isn't enough to get a marketer to fix tracking, free up engineering time, or talk a CEO out of shipping a redesign with no test plan. This episode is about the part of the job no data science course covers: the soft skills that decide whether your program ever gets off the ground.Shiva Manjunath, host of the experimentation podcast ‘From A to B’ , joins host Ryan Lucht to reframe those skills as capital, the trust and goodwill you build with colleagues, then spend every time you push for a test or challenge a HiPPO. But they don't let soft skills off the hook either, because the same skills that win a room can just as easily sell a bad conclusion, which is why hard skills have to keep them honest. What follows is a candid, practitioner-to-practitioner take on why being right and being effective aren't the same thing.In this episode, we discuss:Why a "culture of experimentation" is really a culture that supports experimentation, and how early wins and internal champions build itHow to spend your career and relationship capital wisely, from picking your battles to knowing when to disagree and commitWhy data teams so often get out-maneuvered by better communicators, and how to close that gap without becoming oneHow to turn a redesign with no test plan into a "we'll test it once we're live" win

What separates experimentation programs that become strategic advantages from those that never make it beyond a handful of A/B tests? According to Lucia van den Brink, it starts with ownership.As founder of The Initial, Lucia helps companies build experimentation programs from the ground up but unlike most agencies, her goal isn't to become a permanent partner. Instead, she works to create internal capabilities, establish experimentation culture, and leave organizations with self-sufficient teams that can continue growing long after her engagement ends.In this episode, Lucia joins host Ryan Lucht to discuss why experimentation is about much more than finding winning tests. She explains how organizations can build momentum through early wins, develop internal champions, avoid getting stuck on tool selection, and create a culture where data—not hierarchy—drives better decisions.In this episode, we discuss:Why experimentation should become an internal capability instead of an outsourced functionHow to build an experimentation culture through early wins, internal champions, and organizational buy-inWhy companies often overthink experimentation tools—and underestimate the importance of simply running experimentsThe biggest mistakes organizations make when launching experimentation programs, including chasing velocity at the expense of learning

What does it really mean to run SEO as an experiment instead of a checklist?In this episode, we’re joined by Will Critchlow and Mat Reiss to explore the evolution of SEO and AEO/GEO A/B testing — from enterprise-level experimentation frameworks to a super simple way for any team to get started: title tags.Will breaks down the fundamentals of SEO and AEO/GEO testing and how it differs from traditional user-based A/B testing. Because Google effectively acts as the “audience,” experiments must split pages rather than users and be implemented server-side so crawlers can see the changes. We discuss why organic traffic is a stronger measure of impact than rankings alone, how counterfactual forecasting helps isolate true lift, and why modern SEO increasingly feels like experimenting against a black-box, AI-driven system.Then I called up Mat to help us get real practical, really quickly — with “title tags”, what he calls the “last mile” of search performance. He explains why small copy changes can meaningfully increase click-through rate without improving rankings, shares a case study that drove a 30% traffic lift from a simple tweak, and outlines how teams of different sizes can begin testing title tags without heavy engineering resources.In this episode:How SEO and AEO/GEO A/B testing differs from traditional experimentationWhy organic traffic is often a more reliable KPI than rankingsThe statistical and structural realities of page-based testingWhy title tags are an underused but high-impact growth leverA real-world example of a 30% traffic lift from a simple title changeWhen SEO testing becomes viable for small and mid-sized e-commerce teams

What’s going to happen with the adoption and development of feature flagging tools in 2026?Just ask Aaron Silverman. As an engineering manager at Datadog, Aaron has spent the past year speaking at conferences, meeting engineers, and talking with teams around the world about how they actually use feature flags and why adoption still looks so uneven heading into 2026.In this episode, we explore the current state of feature flagging and A/B testing, and how teams are evolving their approach as systems scale and risk increases. Aaron shares lessons from working directly with customers, explains why the biggest blockers to adoption are cultural rather than technical, and unpacks the hidden costs of building custom feature flagging solutions. We also discuss what’s next, including deeper integrations with observability tools and designing flags with their full lifecycle in mind to reduce friction and technical debt.In this episode:Where feature flagging adoption stands heading into 2026Why cultural and habit-based barriers slow adoption more than technologyHow feature flags reduce deployment risk and improve customer experienceThe hidden complexity of building DIY feature flagging systemsWhat’s next for feature flagging, from observability integrations to smarter flag design

What happens when data science meets marketing? Just ask Sundar Swaminathan. As the former Head of Brand Marketing Data Science at Uber, he measured and optimized over $1 billion in marketing spend across campaigns. Now he’s taken his career in a slightly new direction as a “marketer with some data science skills” instead of a “data scientist working on marketing” In this episode, we explore the intersection of data science and marketing, and how analytical skills, curiosity, and clear communication shaped Sundar’s career and Uber’s marketing strategy. He shares lessons from leading high-stakes campaigns, applying data-driven thinking to brand and performance marketing, and why culture is just as important as technology. Sundar also talks about Experimental, his podcast and newsletter, that helps marketers become more data-savvy and strategic to drive real results.In this Episode:How data science skills can transform marketing strategyThe role of curiosity, communication, and culture in leadershipInsights from leading Uber’s brand, performance, and lifecycle campaignsLessons for marketers aiming to make data-driven decisionsSundar’s perspective on building a career at the intersection of analytics and marketing

Imagine a team where each stage of the customer funnel has its own growth engineer experimenting fast, guided by data, and powered by AI.Today’s episode dives into how Kameron Tanseli, Lead Growth Engineer at Fyxer AI, blends code, marketing, and curiosity to drive high velocity growth. Tanseli shares how tools like Codex and Greptile turn AI into an accelerator for rapid iteration and why culture, not just technology, keeps that engine running.In this Episode:The evolving role of growth engineers at the intersection of tech and marketingHow AI tools enable faster, smarter experimentationThe culture behind Fyxer AI’s speed—transparency, focus, and decentralized ownershipWhy scaling experimentation starts with outcomes over output