#190 - How to Navigate the Constant Change in UX Research with Learners CEO Alec Levin

Producer's Note:After eight incredible years of conversations, insights, and yes, a few awkward silences, we're bringing the Awkward Silences podcast to a close.We're incredibly grateful to our guests, listeners and hosts who've been part of this journey.But the conversations don't have to end. We invite you to check out Insights Unlocked, a podcast from Auros, where we explore the people, ideas, and innovations shaping the future of AI, research, digital products, and customer experiences.Find Insights Unlocked wherever you listen to podcasts.And from everyone at Awkward Silences, past and present, thank you for listening, learning, and getting curious with us.Episode summary: Can synthetic users really tell us what customers think, or are we putting too much trust in AI-generated answers?In this episode of Awkward Silences, Nathan Isaacs sits down with John Whalen, founder of Brilliant Experience, Lija Hogan, principal of customer experience consulting at UserTesting, and Mike Mace, director of solution marketing at UserTesting, to explore the opportunities, limitations, and ethical questions surrounding synthetic research.The conversation examines how synthetic users, AI-powered personas, and digital twins are changing the way organizations understand their customers. From using real research data to create interactive customer representations to embedding human insights directly into AI workflows, the panel discusses where synthetic research can add value, where it falls short, and why organizations need clear guardrails before adopting these technologies.They also explore the evolving role of UX researchers as AI accelerates product development. Rather than replacing human research, synthetic approaches could help teams explore ideas faster, make more informed decisions, and identify when they need to go back to real people for answers.You'll learn:What synthetic research actually means and how it differs from traditional user researchHow synthetic users, AI personas, and digital twins can help teams explore customer needs and test ideasWhy grounding synthetic research in real human data is essential for accuracy and trustThe ethical considerations and governance practices organizations should establish before using synthetic usersHow research teams can integrate customer insights into AI agents and everyday business workflowsWhy synthetic research could make UX researchers more influential, not less relevantResources & linksJohn Whalen on LinkedInBrilliant ExperienceLija Hogan on LinkedInMike Mace on LinkedInNathan Isaacs on LinkedInUser InterviewsUserTesting 00:00 Intro – The future of synthetic research01:23 What is synthetic research?04:29 How AI is changing customer personas08:24 What's new with synthetic users?11:06 Why synthetic research needs governance14:23 Ethics, consent, and AI research16:04 Why UX researchers still matter19:00 Building AI research skills20:44 Making customer insights interactive23:52 The future of AI research agents26:26 How accurate are synthetic users?29:10 How to get started with synthetic research31:33 Using real customer data with AI32:57 Embedding research into AI workflows36:44 Using synthetic research to validate ideas39:19 Real-time customer insights with AI41:03 Researching underserved audiences43:02 Rethinking traditional UX research46:05 Measuring synthetic research accuracy48:30 Keeping AI research data current50:10 Final advice for UX researchers 

Producer's Note: After eight incredible years of conversations, insights, and yes, a few awkward silences, we're bringing the Awkward Silences podcast to a close after episode 194 (scheduled for Oct. 27th featuring a roundtable discussion on synthetic users).We're incredibly grateful to our guests, listeners and hosts who've been part of this journey.But the conversations don't have to end. We invite you to check out Insights Unlocked, a podcast from Auros, where we explore the people, ideas, and innovations shaping the future of AI, research, digital products, and customer experiences.Find Insights Unlocked wherever you listen to podcasts.And from everyone at Awkward Silences, past and present, thank you for listening, learning, and getting curious with us.Episode summaryIn this episode of Awkward Silences, Auros' Lija Hogan chats with Sal Criscione, senior UX researcher and research leader, about how he's using AI to sharpen his quantitative research skills through hands-on practice, simulated research scenarios, and continuous learning.Sal shares how he turned AI into an interactive practice tool that generates realistic UX research scenarios and datasets, challenges him to select appropriate statistical methods, and provides feedback on his analysis. By separating learning from practicing, he's developed a repeatable approach to building confidence in quantitative research while making the process engaging and fun.The conversation also explores the risks of relying too heavily on AI, including hallucinations, inaccurate calculations, and its tendency to agree with users rather than challenge them. Sal explains why trusted source materials, independent verification, and critical thinking remain essential—and how teaching others can reveal gaps in your own understanding.You'll learn:How to use AI to build quantitative UX research skills through hands-on practiceWhy separating learning from practicing helps develop deeper understandingHow to create AI-generated research scenarios and datasets to practice statistical analysisWays to recognize AI biases, challenge its responses, and verify resultsWhy trusted sources and independent calculations are essential when learning with AIHow teaching others can strengthen your expertise and expose knowledge gapsResources & linksSal Criscione on LinkedIn (https://www.linkedin.com/in/salvatore-criscione/)Lija Hogan on LinkedIn (https://www.linkedin.com/in/lija-hogan-894769/)Nathan Isaacs on LinkedIn (https://www.linkedin.com/in/nathanisaacs/)Quantifying the User Experience (book discussed in the episode; https://www.amazon.com/dp/0128023082)Insights Unlocked podcast (https://www.usertesting.com/resources/podcast)Auros website (https://aurosglobal.com/)

Erin May talks with Jared Forney, Research Operations Principal at Okta, about how the role of research and research operations is shifting as AI accelerates product development. Jared explains why scale and rigor do not have to be opposing forces and why teams need to return to their core goals before adding new tools.Jared dives into how he builds trust across legal, product, and design teams and why that trust becomes the capital you spend when driving big change. He shares why automating more work actually concentrates human attention rather than removing it, how he approaches knowledge management and orchestration across many data sources, and why written communication and technical skills now matter more than ever. Jared also answers audience questions on governance, getting buy in for research ops, working faster under compressed timelines, AI guard rails, and breaking into the field.Highlights02:13 Upskilling for scale and rigor04:33 Balancing speed with research quality14:50 Where humans fit in AI workflows25:00 Rethinking knowledge management and MCP32:00 Best tactic for getting governance buy-in41:35 Enabling faster research under pressureAbout Our GuestJared is a Research Operations Principal based in the San Francisco Bay Area with 8+ years building research infrastructure from the ground up. His work sits at the intersection of governance, tooling, and enablement: building systems that help research teams move faster while keeping data practices sound. He's led repository migrations, developed AI governance frameworks, consolidated tooling ecosystems, and spoken at industry conferences about how operations can amplify — not constrain — great research. Connect with Jared on LinkedIn!ResourcesThe Blueprint for Evaluating AI Across the Research PipelineThe State of Research Operation 2026State of Synthetic Users ReportAI in Research Risk Cascade

This week we're revisiting one of Awkward Silences' most memorable episodes.Much of the conversation around AI in UX involves how it will shape and change researchers' work. This episodes's guest, Jess Holbrook, who leads UX research for Microsoft AI, thinks a lot about how this technology can be improved with design and UX best practices. Jess has spent over a decade working on the UX of AI at companies like Meta and Google. He's also helped develop foundational principles for the ethical use of LLMs.Jess joined Erin and Ben to reframe the AI conversation, focusing on how UXRs can and should adapt, what working on these systems has taught him about the UX discipline, and his philosophy for building teams that are ready to take on the innovative challenges of tomorrow.Jess's recommendations: Matt Webb, Maggie Appleton, and the Pessimists Archive Newsletter.Highlights07:15 Human-Centered AI Evolution17:53 Evolving AI Forecast Strategies24:04 Diverse AI Team Building33:27 Navigating Responsible AI Challenges37:01 Qualitative Analysis with AI Tools40:15 Optimism and Concerns About AIAbout JessSenior Director of Research specializing in human-centered AI. Early stage investor in AI, AI hardware ecosystems, climate, and anything that seems like the world would be better with it in it. Currently leading UX research for Microsoft AI. Previously led the Generative AI team and supported all UX functions for the Responsible AI and data transparency teams at Meta. I also served on the Board of Forging Youth Resilience, a nonprofit that empowers young people to build physical and mental strength for life by providing access to a national network of community-based fitness programs and mentorship.ResourcesThe AI in UX Research Report20+ Tools for Every Research PhaseUsing AI in UX: A Day-in-the-Life

Erin May sits down with Alec Levin, CEO and Co-Founder of Learners, to explore how researchers can navigate the rapid changes reshaping their field. Alec shares his vision for how AI and automation are creating unprecedented opportunities for research teams willing to step up and lead.Alec argues that while AI can handle routine tasks like usability testing and surveys, researchers have a chance to focus on higher-value discovery work that includes prototyping and coding. He emphasizes that this is a critical moment for researchers to define their own future rather than letting executives decide their role. The conversation covers practical examples of AI-powered research workflows, the importance of cross-functional collaboration, and why researchers need to be more proactive in pitching their vision internally.Highlights03:40 Balance sheeting thinking for change06:15 Connecting researchers through community13:11 Discovery research importance 15:02 Vision needed for researcher evolution24:08 Automated research workflows emerging31:16 Human elements AI can't replaceResourcesAI for User Research 101 WorkshopWhat is the New AI in Research Risk Cascade?Assessing the ROI of AI in UX Research WebinarAI Context Engineering for Research Course

This week we're revisiting one of Awkward Silences' most memorable episodes.This episode welcomed Ruby Pryor, founder of Rex, a consulting firm specializing in UX research and strategic design.This episode gets into what it means to demonstrate the "business value" of one's work, specifically how researchers can and should quantify their impact in terms that are tangible to the company. Ruby shared ways to score early wins, build momentum, and overcome communication frictions to find shared value.Episode Highlights03:14 - UX researcher impact: insights, optimization, prioritization, strategy09:14 - Understanding organizational structures and strategy development collaboration20:22 - Challenge of quantifying impact and strategic level21:33 - Measuring strategy impact: challenges and indicators30:04 - Prioritizing investments based on strong market indicators38:51 - Stakeholders prioritize growth, revenue, and cost reductionAbout Our GuestRuby Pryor is the founder of Rex, a service design and UX research consulting firm. Her previous roles include UX research at Grab, strategic design at Boston Consulting Group and management consulting at Nous Group. She has taught courses on increasing the impact of UX to learners from 5 continents and has spoken about design and UX at conferences in Asia and Europe.Resources on UX Research ImpactHow to Track the Impact of Your UX ResearchLeading an Impactful User Research TeamUse Stakeholder Interviews to Drive Impact

Erin May sits down with Sam Ladner, Senior Principal Researcher of Strategy at Workday, to explore the evolving role of AI in qualitative research. Sam brings a refreshingly balanced perspective on where AI can genuinely help researchers and where it fundamentally cannot replace human insight.Sam explains how AI has transformed labor-intensive tasks like transcription and closed coding, freeing researchers to focus on the deeper work of sense-making and understanding outliers. She emphasizes that while AI excels at mathematical correlation hunting, qualitative research is about unriddling complex human experiences that require thinking, feeling, and imagination. The conversation covers practical applications like using MAXQDA for AI-assisted coding, the importance of explaining every outlier in qualitative work, and why emotional storytelling must remain exclusively human territory.Highlights03:09 Why machines can't replace human sense-making08:47 Human moments AI cannot understand12:00 Explaining outliers in qualitative coding16:27 Building effective coding systems for AI21:33 Getting AI to do closed coding properly29:26 When to use AI for low stakes research37:23 AI cannot substitute for qualitative researchAbout SamSam Ladner (she/her) is a sociologist who helps teams innovate, design, and learn. She is the author of Practical Ethnography: A Guide to Doing Ethnography in The Private Sector and Mixed Methods: A Short Guide to Applied Mixed Methods Research. She has worked on dozens of advanced software projects at Microsoft, Amazon, and most recently Workday, where she worked as a Senior Principal Researcher studying the future of work. She is now an independent researcher and consultant, writing her third book, tentatively titled Practical Foresight: Strategic Foresight in Applied Settings. She received her PhD in sociology from York University and lives in Toronto, Ontario, Canada. ResourcesPractical Ethnography: A Guide to Doing Ethnography in The Private SectorMixed Methods: A Short Guide to Applied Mixed Methods ResearchQual vs Quant vs Mixed-Methods Field GuideQualitative Sample Size Calculator

This week we're revisiting one of Awkward Silences' most memorable episodes. This episode welcomed George Whitfield, an expert in applying AI to the analysis of qualitative data. George discussed the intricate challenges of leveraging language models to interpret expansive open-ended data (like interview transcripts), and emphasized the importance of context and not just keyword or topic identification.Episode Highlights03:48 - Innovating consumer insights using AI12:21 - Importance of human involvement in AI tools20:04 - Enhance discussion sections with  AI tools26:50 - AI-inspired insights provide inspiration, not guidance34:12 - Interpretation beyond analyzing transcripts36:46 - Applying engineering rigor to the process of building a businessAbout Our GuestGeorge Whitfield is an Entrepreneur-in-Residence at the Martin Trust Center for MIT Entrepreneurship, Lecturer at the MIT Sloan School of Management, and Software Engineer at Google. George holds 4 patents and has 3 degrees from MIT including a Bachelors in Electrical Engineering and Computer Science and a Masters and Ph.D. in Materials Science and Engineering.Resources on Qual Data and AIOur AI in UX Report shares findings from a survey of over 1,000 researchersWhat does it mean to "code" qualitative data? This breakdown explains it all.Interested in trying an AI tool for your analysis? Here are 20 worth considering.

Erin May sits down with Aneta Kmiecik, founder of Be Your Own Design Team, to tackle one of the biggest questions facing designers today: how to position AI skills in portfolios and resumes. With 15 years in creative industries and a community of 200,000 followers, Aneta brings unique insights into navigating career strategy during this AI transformation.Aneta reveals her Past Present Future framework for career positioning and explains why most companies aren't explicitly requiring AI skills yet, but smart candidates should still showcase them strategically. She discusses the three designer archetypes emerging at AI native companies and shares practical advice on standing out when AI can generate portfolios and case studies for everyone.CHAPTERS00:00 Intro04:06 AI Skills in Job Descriptions Reality Check06:52 Research Strategy for AI Portfolio Positioning14:23 Audience Context Determines AI Skill Positioning17:49 Past Present Future Portfolio Framework27:10 Common Portfolio Mistakes in AI Era35:46 Future of AI Skills in Applications38:39 Favorite Research Interview TechniquesRESOURCESLenny’s Podcast featuring Jenny Wen, Design Lead for Claude - https://www.youtube.com/watch?v=eh8bcBIAAFo 2024 AI in User Research Report - https://www.userinterviews.com/ai-in-ux-research-report CONNECT WITH ANETA KMIECIK:- LinkedIn: Aneta Kmiecik- Instagram: @ux.aneta- Newsletter: Be Your Own Design Team- Portfolio Course: uxportfolio.co

Ben Wiedmaier sits down with user research consultant Nikki Anderson to explore how researchers can stay relevant when budgets are tight and stakeholders are stressed. Nikki runs Drop in Research where she helps teams conduct research, facilitate better meetings, and build research practices from the ground up. She shares how companies are increasingly coming to her with validation requests rather than discovery work, driven by pressure to ship fast and hit numbers. The conversation reveals practical strategies for mixed methods research, using facilitation to drive action from insights, and leveraging AI tools without compromising research quality.Nikki explains why she creates forcing functions to slow stakeholders down when they want to rush into AI features without understanding the problem. She advocates for parallel convergent research design where surveys, interviews, and secondary research happen simultaneously to deliver insights faster. The discussion covers why synthesis remains a bridge too far for AI, how to turn research presentations into collaborative workshops, and specific ways researchers can demonstrate value through facilitation skills.Highlights00:00 Intro05:20 Mixed Methods and Data Triangulation10:07 Finding Small Tests When Teams Won't Wait14:46 Stakeholder Stress Driving Validation Requests23:30 Facilitation as Core Research Skill40:03 Using AI for Research Planning45:39 Demonstrating AI Skills to StakeholdersResources- AI for User Research 101 Course- 30+ Tools for every phase of UX Research- AI Context Engineering for Research Course