Experimentation Masters

Send us FeedbackBjarn Brunenberg, Experimentation Lead, TomTom shares his strategic framework for experimentation. Bjarn discusses what a strategic experimentation framework is, why it's important to use a strategic approach for experimentation to maximise impact, and, from a practical sense, how a strategic experimentation framework can be used in your organisation.---Bjarn Brunenberg is the Experimentation Lead at TomTom. TomTom is a Dutch multinational developer and creator of location technology and consumer electronics, operating in 29 countries and with revenues of €585M.At TomTom, he is responsible for accelerating growth in the eCommerce team through Conversion Rate Optimisation, Experimentation and strategic GTM plans. He has grown App revenue at TomTom by 100%+ YoY. Key to this growth was developing and implementing a systemic experimentation process, which improved experiment throughput from 20 experiments to over 200 experiments.Prior to this, Bjarn worked as a Growth Marketer at Mollie, Netherlands and also as a Digital Marketing Specialist at Accenture, supporting Google projects.He is a keynote speaker and two-time winner of the Experimentation Culture awards, also creating the Strategic Experimentation Framework, which helps organisations link experimentation to their OKRs and big-picture goals.----Want more? Head to the First Principles website for the episode transcript, resources and reading lists - https://www.firstprinciples.ventures/ep025-bjarn-brunenberg-a-strategic-experimentation-frameworkAdd Beat The Odds newsletter to your inbox for practical tips from world-leading experts in Experimentation, Growth & Product Design. https://www.firstprinciples.ventures/newsletterFollow Gavin on LinkedIn at : https://www.linkedin.com/in/gavinbryant/Follow Gavin on Twitter @signalnotnoise

Send us FeedbackMelanie Kyrklund, Global Head of Experimentation, Specsavers shares her personal journey of establishing a global Centre of Excellence experimentation operating model at Specsavers. Melanie discusses the business triggers for the change, the planning and preparation involved, and the key measures of success for the transition.---Melanie Kyrklund is the Global Head of Experimentation at Specsavers. Specsavers is a multinational optical retail chain, operating in 11 countries with revenues of €4.1B. She has more than 15+ years’ experience gained in global roles across analytics, experimentation, digital marketing, and product management.At Specsavers, her primary responsibility is to facilitate enablement of experimentation across the organisation - building processes, tooling, data, and technical governance to scale experimentation reliably. Prior to this, Melanie has worked as Head of Site Optimisation at Staples Europe, eCommerce Product Owner at Booking.com and Optimisation Consultant at Liberty Global.She is passionate about translating data insights into actionable business strategies and customer solutions through continuous learning via experimentation.----Want more? Head to the First Principles website for the episode transcript, resources and reading lists - https://www.firstprinciples.ventures/ep024-melanie-kyrklund-establishing-a-global-experimentation-centre-of-excellence Add Beat The Odds newsletter to your inbox for practical tips from world-leading experts in Experimentation, Growth & Product Design. https://www.firstprinciples.ventures/newsletterFollow Gavin on LinkedIn at : https://www.linkedin.com/in/gavinbryant/Follow Gavin on Twitter @signalnotnoise

Send us FeedbackTracey Reed, Senior Conversion Optimisation Specialist, Air New Zealand discusses how the airline uses experimentation to create elite customer experiences. Tracey shares how to solve the right customer problems, influencing for impact at multiple levels of your organisation and why experimentation needs to be less confronting for the uninitiated.---Tracey Reed is a Senior CRO Specialist at Air New Zealand, with over twelve years of experience in the digital landscape. Her expertise spans customer-centric sectors, including travel, financial services, and media. By employing data-driven strategies, Tracey enhances customer experiences and drives commercial growth through effective conversion rate optimisation (CRO).At Air New Zealand, she has been instrumental in developing and advancing the company’s CRO initiatives, leading to significant improvements in program maturity. A true advocate for the customer, Tracey utilises a variety of data sources to gain deep insights into challenges, collaborating with cross-functional teams to optimise digital experiences.Her diverse skill set includes performance analytics, digital strategy, behavioural science, and online marketing. With a strong command of various digital media products and technologies, Tracey excels in dynamic, fast-paced environments.----Want more? Head to the First Principles website for the episode transcript, resources and reading lists - https://www.firstprinciples.ventures/ep023-tracey-reed-creating-the-airline-of-the-futureAdd Beat The Odds newsletter to your inbox for practical tips from world-leading experts in Experimentation, Growth & Product Design. https://www.firstprinciples.ventures/newsletterFollow Gavin on LinkedIn at : https://www.linkedin.com/in/gavinbryant/Follow Gavin on Twitter @signalnotnoise

Send us FeedbackIn this episode of the podcast, Adam Furness, Design Manager - Growth, Atlassian discusses Designing Growth Experiments: Experiences Your Customers Will Love & Pay For. Adam shares why Good and Fast can co-exist, Atlassian's Good, Better, Best Framework and how designers build confidence around a new opportunity.--- Adam Furness has spent 10+ years leading world-class product design teams that create value through design excellence. He’s worked as a design lead and manager and has built and scaled high-performing teams across APAC and the US.He’s currently Design Manager on the Growth team at Atlassian, where he’s spent the last 5 years leading full-funnel experimentation teams. He believes that ‘fast’ and ‘good’ can co-exist, values autonomous teams, and a critical, human-centred approach to product and problem-solving.Before Atlassian, Adam was design lead and manager for US sports technology company Hudl.----Want more? Head to the First Principles website for the episode transcript, resources and reading lists - https://www.firstprinciples.ventures/ep022-adam-furness-designing-effective-growth-experimentsAdd Beat The Odds newsletter to your inbox for practical tips from world-leading experts in Experimentation, Growth & Product Design. https://www.firstprinciples.ventures/newsletterFollow Gavin on LinkedIn at : https://www.linkedin.com/in/gavinbryant/Follow Gavin on Twitter @signalnotnoise

Send us FeedbackIn this episode of the podcast, Sunita Verma, Test & Learn Specialist, H&M discusses why mindsets and relationship building matters the most when developing an experimentation culture. Sunita shares insights on managing a global experimentation program, how to start small and make a fast start and gaining stakeholder confidence to impact bigger strategic decisions.---Sunita Verma is currently Test & Learn Specialist (East Asia) at global fashion mega brand H&M. She is an experimentation and growth professional with 5+ years of experience in fostering a culture of experimentation within leading organisations. Her work at Zalora and FairPrice Group has been pivotal in advancing data-driven decision-making and promoting a more agile approach to business operations. At Zalora, an online fashion retailer, Sunita integrated experimentation techniques to enhance user experience and optimise operational processes. Her focus on data-driven strategies and testing methodologies provided valuable insights into user behaviour and preferences. At FairPrice Group, one of Singapore’s largest supermarket chains, Sunita drove the adoption of an agile operational model. Her efforts in data analytics and iterative testing enabled FairPrice Group to make informed decisions more swiftly and respond effectively to evolving customer needs.----Want more? Head to the First Principles website for the episode transcript, resources and reading lists - https://www.firstprinciples.ventures/ep021-sunita-verma-mindset-over-metrics Add Beat The Odds newsletter to your inbox for practical tips from world-leading experts in Experimentation, Growth & Product Design. https://www.firstprinciples.ventures/newsletterFollow Gavin on LinkedIn at : https://www.linkedin.com/in/gavinbryant/Follow Gavin on Twitter @signalnotnoise

Send us FeedbackIn this episode of the podcast, Ken Kutyn, Principal Solutions Engineer, Amplitude discusses how to develop high-velocity experimentation in your organisation. Ken shares what a good experimentation program strategy looks like, laying a solid foundation for experimentation and critical steps for scaled experimentation.---Ken Kutyn is a Principal Solutions Engineer at Amplitude Analytics, with over 8 years of experience at the forefront of Experimentation platforms, including Optimizely and Amplitude. Throughout his career, Ken has partnered with some of the world’s largest brands, guiding them in evaluating, implementing, and scaling their experimentation programs. His journey has taken him across major global tech hubs, including Vancouver, San Francisco, Amsterdam, London, and now Singapore.Ken brings a deep technical expertise and passion for experimentation, focusing on enabling high-velocity testing without compromising user experience. His work helps teams eliminate performance bottlenecks, prevent bugs, and ensure clean data integrity—critical factors for successful, scalable experimentation.After publishing his MBA thesis on the role of experimentation in software development, he has become a sought-after speaker, sharing best practices at conferences, webinars, and podcasts worldwide. His mission is to empower teams with the knowledge and tools to drive impactful experimentation and continuous improvement.----Want more? Head to the First Principles website for the episode transcript, resources and reading lists - https://www.firstprinciples.ventures/ep020-ken-kutyn-developing-high-velocity-experimentation Add Beat The Odds newsletter to your inbox for practical tips from world-leading experts in Experimentation, Growth & Product Design. https://www.firstprinciples.ventures/newsletterFollow Gavin on LinkedIn at : https://www.linkedin.com/in/gavinbryant/Follow Gavin on Twitter @signalnotnoise

Send us FeedbackIn this episode of the podcast, Jonas Alves, CEO ABsmartly, discusses building and scaling one of the world's leading experimentation platforms. Jonas shares why you need to keep experimentation simple, worrying less about experiment interaction effects and future directions for ABsmartly and the experimentation industry.---Jonas Alves is the CEO and co-founder of ABsmartly, a leading experimentation platform designed for trustworthy A/B testing. He has 16+ years-experience in the field of online experimentation working in-house and consulting, with experience including design and build of experimentation platforms, democratisation and scaling of experimentation programs and conducting training with experimentation teams.Prior to ABsmartly, Jonas worked as an experimentation consultant, helping other organisations on their experimentation journey, including Catawiki.com, GetYourGuide, Viagogo, Adidas, Picnic.nl, MinDoktor.se, Match.com (OkCupid, Tinder), Remitly, SpotAHome, ParkMobile and many others.He started with experimentation in 2008 at Booking.com. Here he democratised experimentation across the organisation to many development teams. By 2012, hundreds of people were performing experiments every day.  As Product Owner (Experimentation), Jonas was responsible for experimentation training, new features, reporting and metrics.When leaving Booking.com in 2015, the business was running more than 1000 experiments simultaneously and starting 100+ new experiments every single day.----Want more? Head to the First Principles website for the episode transcript, resources and reading lists - https://www.firstprinciples.ventures/ep019-jonas-alves-building-an-experimentation-platformAdd Beat The Odds newsletter to your inbox for practical tips from world-leading experts in Experimentation & Product Design. https://www.firstprinciples.ventures/newsletterFollow Gavin on LinkedIn at : https://www.linkedin.com/in/gavinbryant/Follow Gavin on Twitter @signalnotnoise

Send us FeedbackIn this episode of the podcast, Aleksander Fabijan, Senior Product Manager Microsoft Experimentation Platform (ExP) team, provides practical strategies and lessons from his paper - It Takes a Flywheel to Fly: Kickstarting & Growing A/B Testing Momentum at Scale. Alex shares how to use the five-step experimentation flywheel to increase experimentation velocity in your organisation.---Aleksander Fabijan is Senior Product Manager at Microsoft. He is part of a 60+ team of Data Scientists, Software Engineers and Program Managers that form the Experimentation Platform (ExP) team. The ExP team is responsible for one of the largest and most cutting-edge online experimentation systems in the industry. The mission of the team is to accelerate innovation at Microsoft through trustworthy experimentation. Major products such as Bing, MS Office, Skype, Xbox, Windows, and MS Teams all use the ExP platform to perform online controlled experiments.Aleksander holds a PhD from Malmo University where he conducted his research at Software Centre, a European collaboration of companies and universities accelerating the adoption of novel software engineering approaches. He has published and co-authored many influential research papers on large-scale online controlled experimentation. Key research interests - running trustworthy experiments, experimentation system architecture, and examination of experiments (analysis).His passions include data-driven decision-making using A/B testing, AI system design, or API design, User Experience (UX) and large-scale experimentation.* The views expressed in this podcast are Alek’s personal perspective and not necessarily reflective of Microsoft.----Want more? Head to the First Principles website for the episode transcript, resources and reading lists - https://www.firstprinciples.ventures/ep016-aleksander-fabijan-the-experimentation-flywheel-kickstarting-and-growing-a/b-testing-momentum-at-scaleAdd Beat The Odds newsletter to your inbox for practical tips from world-leading experts in Experimentation & Product Design. https://www.firstprinciples.ventures/newsletterFollow Gavin on LinkedIn at : https://www.linkedin.com/in/gavinbryant/Follow Gavin on Twitter @signalnotnoise

Send us FeedbackIn this episode of the podcast, Erin Weigel, Senior Group Design Product Manager provides an overview of the journey and challenges of writing her new book Design for Impact: Your Guide to Designing Effective Product Experiments. We discuss some of the key points that stood out when reading Erin's book.---Erin Weigel is currently Senior Group Design Product Manager, responsible for product design management across many business areas, including Partner Advertising, Product Design, Design Operations and Design SystemsErin spent eight years working at Booking.com in Product Owner, Senior Designer and Principal Designer roles. While at Booking.com, Erin designed and coded more than 1,200 A/B tests. She analysed and learned from thousands more. Her unique approach to product design and development combines a service mindset, customer-centric thinking, and A/B testing to deliver highly converting products that customers love. Erin has a Bachelor’s Degree in Fine Art Painting from The Maryland Institute College of Art (MICA), and she has studied at Konstfack University College of Art & Design in Stockholm, Sweden. Nowadays, she shares what she learns about design and experimentation at conferences and workshops around the world. ----Want more? Head to the First Principles website for the episode transcript, resources and reading lists - https://www.firstprinciples.ventures/ep017-erin-weigel-how-to-design-effective-product-experimentsAdd Beat The Odds newsletter to your inbox for practical tips from world-leading experts in Experimentation & Product Design. https://www.firstprinciples.ventures/newsletterFollow Gavin on LinkedIn at : https://www.linkedin.com/in/gavinbryant/

Send us FeedbackIn this episode of the podcast, Matt Mulvey, Senior Experimentation Specialist at British Airways, shares how the UK's flagship airline is undergoing an innovation renaissance with its £7B Transformation Project. Learn how one of the world’s leading airlines plans to transform and improve customer experience.---Matt Mulvey is a Digital Experimentation Specialist at British Airways. With revenues of €3.5B, British Airways is the UK’s flagship airline, with a fleet size of 267 planes, flying to 206 destinations. His expertise includes optimisation of digital products through A/B Testing, Conversion Rate Optimisation, Machine Learning and Digital Analytics.Matt has previously worked as a Senior Conversion Rate Optimisation Specialist and Conversion Rate Optimisation Specialist at Woolworths Supermarkets, New Zealand. In these roles he was responsible for driving experimentation program strategy and execution, championing experimentation company-wide, and increasing experiment velocity and quality through program optimisation. Prior to this, Matt was a Digital Analyst, conducting A/B testing to improve customer experience.Originally from Auckland, New Zealand, Matt also established the Analytics & Optimisation Auckland Meetup Group, New Zealand’s first experimentation focussed meetup group, growing to 100+ members.----Want more? Head to the First Principles website for the episode transcript, resources and reading lists - https://www.firstprinciples.ventures/ep016-matt-mulvey-transforming-customer-experience-at-british-airwaysAdd Beat The Odds newsletter to your inbox for practical tips from world-leading experts in Experimentation & Product Design. https://www.firstprinciples.ventures/newsletterFollow Gavin on LinkedIn at : https://www.linkedin.com/in/gavinbryant/Follow Gavin on Twitter @signalnotnoise