56. AI for UX Analysis: How Accurate Is It? (feat. Christian Holst & Jamie Holst, Baymard Institute)

As AI-powered tools flood the market promising quick evaluations and audits, the conversation zeroes in on a critical issue: the reliability of these tools. In this episode, co-hosts Therese Fessenden and NN/G VP Kate Moran sit down with Jamie Holst and Christian Holst, the co-founders of Baymard Institute, to examine the growing role of AI in UX analysis. The group discusses the risks of over-relying on AI—particularly for junior UX practitioners—and the deeper implications for professionalism in the field.About the speakers:Christian Holst (Co-founder & Research Director, Baymard Institute)Jamie Holst (Co-founder & CTO, Baymard Institute)Kate Moran (VP, NN/G)Links Referenced in the Episode:Try UX Ray for Free: https://baymard.com/product/ux-rayRead their article on AI accuracy: https://baymard.com/blog/ai-heuristic-evaluationsRelated NN/G Courses:Accelerating Research with AIAI for Design WorkflowsDesigning AI ExperiencesAI Product StrategyCheck out ALL of our courses at nngroup.com/learningRelated NN/G Articles:AI Hallucinations: What Designers Need to KnowAI-Powered Tools for UX Research: Issues and LimitationsAI as a UX AssistantAI Chatbots Discourage Error Checking

The interest in “customer insights” is higher than ever, and more people are interested in both research insights and carrying out research on their own. But, who’s making sure research is done well? Therese sits down with Ned Dwyer, CEO and co-founder of Great Question, to talk about the balance between access and rigor in UX research. They discuss the need for democratization charters, how AI is reshaping research roles, and why enabling good research at scale is as much a cultural challenge as it is a technical one.Connect with Ned Dwyer: LinkedInLearn more about Great QuestionSubscribe to Great Question’s NewsletterLearn about UX: nngroup.com/learningRelated NN/g Courses & Articles:ResearchOps: Scaling User Research (UX Certification course)Democratize Research in 5 Steps (free article)Democratization of UX (13-min video)

In this episode, Therese interviews Jay Dutta—Global Head of UX at Deutsche Bank about how designing at scale isn’t just about reaching millions, but about preserving depth: of empathy, culture, and context. With a career spanning Adobe, Flipkart, and MakeMyTrip, Jay reflects on why he’s drawn to complex, broken systems and how those environments offer the richest opportunities for impact. He discusses the importance of designing for the individual within vast systems, and how storytelling, risk-taking, and honoring human experiences can lead to more meaningful and scalable design solutions.About the speaker: JayDutta.com | LinkedInDesignUp Conference (Bangalore, India - Nov 3-5, 2025 ): https://designup.io/Related NN/G Course: Global UX: Designing and Leading Across Cultures (full-day & half-day courses)Related Free NN/G Articles:‘Our Users Are Everyone’: Designing Mass-Market Products for Large User Audiences (free article)Modify Your Design for Global Audiences: Crosscultural UX Design (free article)Imagery Helps International Shoppers Navigate Ecommerce Sites (free article)

What does it really take to get hired in design today? In this episode, design recruiter and former product designer Hang Xu shares hard truths about what’s changed in hiring, why portfolios miss the mark, and how candidates can better align with what employers actually want. He also talks about his bold new project, OnlyFAANGs.design, where he documents his journey back into design from the recruiter’s seat.About Hang Xu: LinkedIn | OnlyFAANGs.designLevel Up Your UX Career with UX Certification: nngroup.com/learningRelated Free NN/G Resources:Templates and Tools for UX Job Seekers (article)How to Apply & Secure a UX Job (article)The Return of the UX Generalist (article)How to Get UX Experience Without a Job (video)User Experience Careers Report (free report)More articles & videos about UX careers

Who do we really mean when we say “the user”? And who gets left out?In this episode, Therese Fessenden chats with cultural anthropologists Mike Youngblood and Ben Chesluk, who are challenging overly narrow definitions of users in UX. They explore how systems thinking and anthropology can deepen our understanding of complex user ecosystems — and why rethinking “the user” is more important than ever.About the speakers:Mike Youngblood (LinkedIn)Ben Chesluk (LinkedIn)About Rethinking Users, including some helpful resources and a link to purchase the book:https://www.rethinkingusers.comArticle about User Ecosystem Thinking:https://www.epicpeople.org/toward-ethnography-of-friction-and-ease-in-complex-systemsMike & Ben’s other books:The Routledge Companion to Practicing Anthropology and Design, which includes Mike & Ben’s chapter on systems thinking in design anthropology, as well as many other chapters/contributors exploring the role of anthropology in design:Ben’s book Money Jungle is an anthropological study of the redevelopment of New York City’s Times Square.Mike’s book Cultivating Community is an ethnographic study of a massive, grassroots social movement in rural India.Related NN/g Articles & Videos:Personas vs. Archetypes (article)Why Personas Fail (article)Ethnography in UX (3-min video)Contextual Inquiry: Inspire Design by Observing and Interviewing Users in Their Context (article)

What does it mean to design experiences in a world where pixels are no longer the focus? In this episode, service design thought-leader and author Erika Flowers shares her thoughts on how designers must evolve as AI transforms the tools, processes, and expectations of our work. Erika shares lessons from her journey—spanning early web design, pioneering service blueprinting at Intuit, and orchestrating experiences at NASA—and offers a forward-looking take on why skills like facilitation, systems thinking, and storytelling will define the future of design.About Erika Flowers: helloerikaflowers.com | LinkedInRelated NN/g CoursesService BlueprintingAccelerating Research with AIAI for Design WorkflowsRelated Free NN/g Articles:How Service Design Will Evolve with AI AgentsThe Future-Proof DesignerWhy I’m Not Worried About My UX Job in the Era of AIMore Free Articles on AI

AI can do more than it’s ever done… but there’s a lot of unfounded hype, especially when it comes to user research. When should you delegate tasks to AI? And when should you insist on keeping the human in the loop? In this episode, Therese Fessenden sits down with Alexander Knoll, co-founder of Condens, to discuss the strengths and limitations of AI tools for research, and the evolving role of the user researcher.About Alexander & Condens: LinkedIn | Condens.ioResearch Repository Guide: https://condens.io/guides/research-repository-guide/On-Demand Recording of Condens Event: Making an Impact with User Research: How to Drive Change and Get NoticedAlex’s Article with NN/g: Common-Sense AI Integration: Lessons from the Cofounder of CondensOther Related NN/g Articles & Courses:Free Articles about AI & UXCourse: UX Basic TrainingCourse: Accelerating Research with AICourse: AI for Design WorkflowsCourse: Designing AI Experiences

In this episode, Peter Merholz—co-founder of Adaptive Path and co-author of Org Design for Design Orgs—joins us to talk about what it really takes to build and scale effective design teams. We explore the systems, structures, and leadership mindsets that shape successful organizations, the relationships between product and design teams, and how design leaders can foster clarity, consistency, and impact as their teams grow. Whether you’re managing a team or aspiring to, this conversation is packed with insights on navigating the evolving role of design in today’s organizations.Learn More About Peter:Website: https://petermerholz.com/Newsletter: https://petermerholz.com/newsletter/ LinkedIn: https://linkedin.com/in/petermerholzPodcast: https://findingourway.design/Book: https://orgdesignfordesignorgs.com/Full-Day Courses (also available in half-day formats):DesignOps: Scaling UX Design: https://www.nngroup.com/courses/scaling-ux-design-and-user-research/?utm_source=podcast&utm_medium=social&utm_campaign=ep50Product and UX: Building Partnerships for Better Outcomes: https://www.nngroup.com/courses/product-and-ux/?utm_source=podcast&utm_medium=social&utm_campaign=ep50Leading Highly Effective UX Teams: https://www.nngroup.com/courses/leading-ux-teams/?lm=pm-ux-different-views-of-responsibilities&pt=article&utm_source=podcast&utm_medium=social&utm_campaign=ep50UX Leader: Essential Skills for Any UX Practitioner: https://www.nngroup.com/courses/ux-leadership-skills-for-all/?utm_source=podcast&utm_medium=social&utm_campaign=ep50New UX Managers: https://www.nngroup.com/courses/new-manager/?utm_source=podcast&utm_medium=social&utm_campaign=ep50Related Free NN/g Articles:DesignOps: Study Guide: https://www.nngroup.com/articles/design-ops-study-guide/?utm_source=podcast&utm_medium=social&utm_campaign=ep50Product & UX: Study Guide: https://www.nngroup.com/articles/product-and-ux-study-guide/?utm_source=podcast&utm_medium=social&utm_campaign=ep50PM and UX Have Markedly Different Views of Their Job Responsibilities: https://www.nngroup.com/articles/pm-ux-different-views-of-responsibilities/?utm_source=podcast&utm_medium=social&utm_campaign=ep50

UX practitioners often struggle with shifting requirements and increasing product complexity, making it difficult to design intuitive experiences. Object-Oriented UX (OOUX) offers a structured approach to simplify digital products by focusing on core objects rather than pages or screens. Sophia Prater explains how this method improves usability, reduces inconsistencies, and streamlines collaboration across teams.About Sophia Prater: Bio (via RewiredUX.com) | LinkedIn | InstagramLearn more about Object-Oriented UX:Upcoming Free UX Level-Up TrainingSign up for the newsletterOOUX PodcastOOUX for AI & AI for OOUX (course)Related NN/g Courses & Resources:UX Basic Training (full-day or 2 half-day course)What is Object-Oriented UX? (3-min video)

As AI continues to evolve, designers are increasingly finding themselves tasked with creating AI-powered experiences—often without prior experience in the space. In this episode, NN/g UX Specialist Caleb Sponheim discusses the current state of AI tools, what’s working (and what’s not), and how designers can position themselves to navigate this shift. We also explore what it really means to develop AI safely and how designers can play a role in ensuring ethical, user-friendly AI experiences.About the speaker: Bio | LinkedInFull-Day Course: Designing AI Experiences(also available in half-day formats)Related NN/g Articles:Artificial Intelligence GlossaryHow Do Generative AI Systems Work?Redefine Your Design Skills to Prepare for AIAI Features Must Solve Real ProblemsMore Free Articles on AI