Lenny's Podcast: Product | Career | Growth

Chip Huyen is a core developer on Nvidia’s Nemo platform, a former AI researcher at Netflix, and taught machine learning at Stanford. She’s a two-time founder and the author of two widely read books on AI, including AI Engineering, which has been the most-read book on the O’Reilly platform since its launch. Unlike many AI commentators, Chip has built multiple successful AI products and platforms and works directly with enterprises on their AI strategies, giving her unique visibility into what’s actually happening inside companies building AI products.We discuss:1. What people think makes AI apps better vs. what actually makes AI apps better2. What pre-training vs. post-training is, and why fine-tuning should be your last resort3. How RLHF (reinforcement learning from human feedback) actually works4. Why data quality matters more than which vector database you choose5. Why high performers are seeing the most gains from AI coding tools6. Why most AI problems are actually UX issues—Brought to you by:Dscout—The UX platform to capture insights at every stage: from ideation to production: https://www.dscout.com/Justworks—The all-in-one HR solution for managing your small business with confidence: https://ad.doubleclick.net/ddm/trackclk/N9515.5688857LENNYSPODCAST/B33689522.423713855;dc_trk_aid=616485030;dc_trk_cid=237010502;dc_lat=;dc_rdid=;tag_for_child_directed_treatment=;tfua=;gdpr=$Persona—A global leader in digital identity verification: https://withpersona.com/lenny—Where to find Chip Huyen:• X: https://x.com/chipro• LinkedIn: https://www.linkedin.com/in/chiphuyen/• Website: https://huyenchip.com/—Where to find Lenny:• Newsletter: https://www.lennysnewsletter.com• X: https://twitter.com/lennysan• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/—In this episode, we cover:(00:00) Introduction to Chip Huyen(04:28) Chip’s viral LinkedIn post(07:05) Understanding AI training: pre-training vs. post-training(08:50) Language modeling explained(13:55) The importance of post-training(15:20) Reinforcement learning and human feedback(22:23) The importance of evals in AI development(31:55) Retrieval augmented generation (RAG) explained(38:50) Challenges in AI tool adoption(43:19) Challenges in measuring productivity(45:20) The three-bucket test(49:10) The future of engineering roles(55:31) ML Engineers vs. AI engineers(57:12) Looking forward: the impact of AI(01:05:48) Model capabilities vs. perceived performance(01:08:23) Lightning round and final thoughts—Referenced:• Chip’s LinkedIn post on what actually improves AI apps: https://www.linkedin.com/posts/chiphuyen_aiapplications-aiengineering-activity-7358971409227792384-y0mf/• Prediction and Entropy of Printed English: https://www.princeton.edu/~wbialek/rome/refs/shannon_51.pdf• Why experts writing AI evals is creating the fastest-growing companies in history | Brendan Foody (CEO of Mercor): https://www.lennysnewsletter.com/p/experts-writing-ai-evals-brendan-foody•Inside the expert network training every frontier AI model | Garrett Lord (Handshake CEO): https://www.lennysnewsletter.com/p/inside-handshake-garrett-lord• First interview with Scale AI’s CEO: $14B Meta deal, what’s working in enterprise AI, and what frontier labs are building next | Jason Droege: https://www.lennysnewsletter.com/p/first-interview-with-scale-ais-ceo-jason-droege• Anthropic’s CPO on what comes next | Mike Krieger (co-founder of Instagram): https://www.lennysnewsletter.com/p/anthropics-cpo-heres-what-comes-next• Why AI evals are the hottest new skill for product builders | Hamel Husain & Shreya Shankar (creators of the #1 eval course): https://www.lennysnewsletter.com/p/why-ai-evals-are-the-hottest-new-skill• The rise of Cursor: The $300M ARR AI tool that engineers can’t stop using | Michael Truell (co-founder and CEO): https://www.lennysnewsletter.com/p/the-rise-of-cursor-michael-truell• Stanford webinar—How AI Is Changing Coding and Education, Andrew Ng & Mehran Sahami: https://www.youtube.com/watch?v=J91_npj0Nfw• He saved OpenAI, invented the “Like” button, and built Google Maps: Bret Taylor on the future of careers, coding, agents, and more: https://www.lennysnewsletter.com/p/he-saved-openai-bret-taylor• Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann: https://www.lennysnewsletter.com/p/anthropic-co-founder-benjamin-mann• Lenny’s vibe-coded app made on Lovable: https://gdoc-images-grab.lovable.app/• Story of Yanxi Palace: https://www.imdb.com/title/tt8865016/• Steve Jobs’s quote: https://www.goodreads.com/quotes/427317-remembering-that-i-ll-be-dead-soon-is-the-most-important—Recommended books:• The Complete Sherlock Holmes: https://www.amazon.com/Complete-Sherlock-Holmes-Volumes/dp/0553328255• AI Engineering: Building Applications with Foundation Models: https://www.amazon.com/AI-Engineering-Building-Applications-Foundation/dp/1098166302• The Selfish Gene: https://www.amazon.com/Selfish-Gene-Anniversary-Introduction/dp/0199291152• From Third World to First: The Singapore Story: 1965-2000: https://www.amazon.com/Third-World-First-Singapore-1965-2000/dp/0060197765—Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email [email protected] may be an investor in the companies discussed. To hear more, visit www.lennysnewsletter.com

Nicole Forsgren created the most widely used frameworks for measuring developer productivity—DORA and SPACE. She wrote the foundational book Accelerate and is about to release her newest book, Frictionless, a practical guide for helping teams move faster in the AI era. She’s currently Senior Director of Developer Intelligence at Google.We discuss:1. Why most productivity metrics are a lie2. Signs that your engineering team could be moving much faster3. Why AI accelerates coding but developers aren’t speeding up as much as you think4. AI’s impact on engineers getting into “flow”5. Her framework for building and scaling a developer experience team6. The three components of developer experience: flow state, cognitive load, and feedback loops—Brought to you by:Mercury—The art of simplified finances: https://mercury.com/WorkOS—Modern identity platform for B2B SaaS, free up to 1 million MAUs: https://workos.com/lennyCoda—The all-in-one collaborative workspace: https://coda.io/lenny—Where to find Nicole Forsgren:• Twitter: https://twitter.com/nicolefv• LinkedIn: https://www.linkedin.com/in/nicolefv/• Website: https://nicolefv.com/—Where to find Lenny:• Newsletter: https://www.lennysnewsletter.com• X: https://twitter.com/lennysan• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/—In this episode, we cover:(00:00) Introduction to Nicole Forsgren(05:09) The concept of developer experience (DevEx)(08:33) Flow state and cognitive load in the age of AI(12:02) Challenges in measuring productivity with AI(21:19) The importance of developer experience for business value(22:20) Common issues and solutions in developer experience(26:49) Signs your eng team is moving too slow(29:52) How AI is improving productivity(33:32) Real examples of productivity improvements(36:35) Introducing her new book, Frictionless(43:40) How to get started building a DevEx team(45:15) The impact of forming developer experience teams(46:15)  How to measure the impact of DevEx teams(48:53) Measuring the impact of AI tools on productivity(55:16) Survey design for developer experience(57:59) Popular AI tools for developers(59:08) Bringing a product mindset to DevEx improvements(01:00:40) AI corner(01:02:33) Lightning round and final thoughts—Referenced:• How to measure and improve developer productivity | Nicole Forsgren (Microsoft Research, GitHub, Google): https://www.lennysnewsletter.com/p/how-to-measure-and-improve-developer• DORA: https://dora.dev/• The SPACE framework: A comprehensive guide to developer productivity: https://getdx.com/blog/space-metrics/• Measuring developer productivity with the DX Core 4: https://getdx.com/research/measuring-developer-productivity-with-the-dx-core-4/• Gloria Mark’s website: https://gloriamark.com/• Taking Flight with Copilot: https://dl.acm.org/doi/10.1145/3589996• DevEx in Action: https://spawn-queue.acm.org/doi/10.1145/3639443• CodeX: https://openai.com/codex/• Devin: https://devin.ai/• Abi Noda on LinkedIn: https://www.linkedin.com/in/abinoda/• DX is joining Atlassian: https://getdx.com/blog/dx-is-joining-atlassian/• GitHub Copilot: https://github.com/features/copilot• Cursor: https://cursor.com/• The rise of Cursor: The $300M ARR AI tool that engineers can’t stop using | Michael Truell (co-founder and CEO): https://www.lennysnewsletter.com/p/the-rise-of-cursor-michael-truell• Gemini Code Assist: https://codeassist.google/• Claude Code: https://www.claude.com/product/claude-code• The AI-native startup: 5 products, 7-figure revenue, 100% AI-written code | Dan Shipper (co-founder/CEO of Every): https://www.lennysnewsletter.com/p/inside-every-dan-shipper• Love Is Blind on Netflix: https://www.netflix.com/title/80996601• Shrinking on AppleTV+: https://tv.apple.com/us/show/shrinking/umc.cmc.apzybj6eqf6pzccd97kev7bs• Ninja Creami: https://www.amazon.com/Ninja-NC301-CREAMi-Containers-Bundle/dp/B0BLGR5JPV/• Jura coffee maker: https://www.amazon.com/Jura-Nordic-Automatic-Coffee-Machine/dp/B0CF65BFZ1/—Recommended books:• Frictionless: https://developerexperiencebook.com/• DevEx Workbook: https://developerexperiencebook.com/#workbook• Outlive: The Science and Art of Longevity: https://www.amazon.com/Outlive-Longevity-Peter-Attia-MD/dp/0593236599• Back Mechanic: https://www.amazon.com/Back-Mechanic-Stuart-McGill-2015-09-30/dp/B01FKSGJYC• How Big Things Get Done: The Surprising Factors That Determine the Fate of Every Project, from Home Renovations to Space Exploration and Everything in Between: https://www.amazon.com/How-Big-Things-Get-Done/dp/0593239512/• The Undoing Project: A Friendship That Changed Our Minds: https://www.amazon.com/dp/B01KBM82M4/—Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email [email protected] may be an investor in the companies discussed. To hear more, visit www.lennysnewsletter.com

Dylan Field is co-founder and CEO of Figma, a beloved tool used by every modern product team. Founded in 2012, Figma has expanded from a single design tool to a comprehensive platform including FigJam, Slides, Dev Mode, and, most recently, Figma Make. After a $20 billion acquisition by Adobe fell through due to regulatory pushback, Dylan led the company to a successful IPO in 2025.What you’ll learn:• How Dylan kept internal morale up after the Adobe acquisition fell through• His approach to maintaining pace and a sense of urgency 13 years in• How to systematically develop taste• How Figma decides which product lines to add• Why Dylan obsesses over “time to value”• How AI is making design more valuable—Brought to you by:Stripe—Helping companies of all sizes grow revenue—Transcript: ⁠https://www.lennysnewsletter.com/p/why-ai-makes-design-craft-and-quality-the-new-moat⁠—My biggest takeaways (for paid newsletter subscribers): ⁠https://www.lennysnewsletter.com/i/175569466/my-biggest-takeaways-from-this-conversation⁠—Where to find Dylan Field:• X: https://x.com/zoink• LinkedIn: https://www.linkedin.com/in/dylanfield/—Where to find Lenny:• Newsletter: https://www.lennysnewsletter.com• X: https://twitter.com/lennysan• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/—In this episode, we cover:(00:00) Introduction to Dylan Field(03:58) The Adobe deal fallout(05:50) Maintaining team morale post-deal(09:13) Strategies for sustaining high performance(13:37) Maintaining Figma’s unique company culture(16:22) Dylan’s leadership evolution(21:03) How to improve clarity as a leader(24:40) The controversy behind FigJam(31:06) Lessons from expanding Figma’s core product line(39:32) Time-to-value(45:14) Introduction to Figma Make(48:26) AI app prototyping and the future of Figma Make(53:38) Lessons from Figma’s AI product launch(57:47) The importance of craft(59:54) Developing good taste(01:05:35) The future of product development(01:10:32) Why AI won’t steal your job(01:14:37) AI corner(01:18:32) Lightning round and final thoughts—Referenced:• Dylan Field live at Config: Intuition, simplicity, and the future of design: https://www.lennysnewsletter.com/p/dylan-field-live-at-config• Figma: https://www.figma.com/• Adobe: https://www.adobe.com/• Vision, conviction, and hype: How to build 0 to 1 inside a company | Mihika Kapoor (Product at Figma): https://www.lennysnewsletter.com/p/vision-conviction-hype-mihika-kapoor• Notion’s lost years, its near collapse during Covid, staying small to move fast, the joy and suffering of building horizontal, more | Ivan Zhao (CEO and co-founder): https://www.lennysnewsletter.com/p/inside-notion-ivan-zhao• $46B of hard truths from Ben Horowitz: Why founders fail and why you need to run toward fear (a16z co-founder): https://www.lennysnewsletter.com/p/46b-of-hard-truths-from-ben-horowitz• FigJam: https://www.figma.com/figjam/• Cursor chat: https://help.figma.com/hc/en-us/articles/4403130802199-Use-cursor-chat-in-Figma-Design• Figma Slides: https://www.figma.com/slides/• Figma Sites: https://www.figma.com/sites/• Figma Buzz: https://www.figma.com/buzz/• Figma Draw: https://www.figma.com/draw/• Figma Design: https://www.figma.com/design/• Dev Mode: https://www.figma.com/dev-mode/• Figma Make: https://www.figma.com/make/• Zach Lloyd on X: https://x.com/zachlloydtweets• Warp: https://www.warp.dev/• Dylan’s post on X about Figma on an AI product leaderboard: https://x.com/zoink/status/1968588014935801884• Kurt Cobain: https://en.wikipedia.org/wiki/Kurt_Cobain• Damien Correll on LinkedIn: https://www.linkedin.com/in/damiencorrell/• Marcin Wichary on LinkedIn: https://www.linkedin.com/in/mwichary/• Loredana Crisan on LinkedIn: https://www.linkedin.com/in/loredanacrisan/• Amber Bravo on LinkedIn: https://www.linkedin.com/in/amberbravo/• Figma’s 2025 AI report: Perspectives from designers and developers: https://www.figma.com/blog/figma-2025-ai-report-perspectives/• Jevons paradox: https://en.wikipedia.org/wiki/Jevons_paradox#Energy_conservation_policy• AI prompt engineering in 2025: What works and what doesn’t | Sander Schulhoff (Learn Prompting, HackAPrompt): https://www.lennysnewsletter.com/p/ai-prompt-engineering-in-2025-sander-schulhoff• Pantheon: https://www.imdb.com/title/tt11680642/• Retro: https://retro.app/• Thiel Fellowship: https://thielfellowship.org/—Recommended books:• Understanding Comics: The Invisible Art: https://www.amazon.com/Understanding-Comics-Invisible-Scott-McCloud/dp/006097625X• The Spy and the Traitor: The Greatest Espionage Story of the Cold War: https://www.amazon.com/Spy-Traitor-Greatest-Espionage-Story/dp/1101904216• Codex Seraphinianus: https://www.amazon.com/Codex-Seraphinianus-Anniversary-Luigi-Serafini/dp/0847871045Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email [email protected].—Lenny may be an investor in the companies discussed.My biggest takeaways from this conversation: To hear more, visit www.lennysnewsletter.com

Robby Stein is VP of Product at Google, where he oversees the core products of Google Search—including the new AI Overviews, AI Mode, search ranking, Google Lens, and more. Previously, he led consumer products at Instagram, where he and his teams built Stories, Reels, Close Friends, and other key features now used by billions.—What you’ll learn:Why Google’s AI products are suddenly taking off after years of perceived stagnation How AI is expanding Search rather than replacing it, contrary to what many predicted The three core product principles that have helped Robby build multiple billion-user products Inside Instagram’s decision to build its own version of Snapchat Stories His mantra of “relentless improvement” How Google developed AI Mode from concept to launch in just one year Why most teams give up too early on potentially transformative products —Brought to you by:Vanta—Automate compliance. Simplify security: https://vanta.com/lennyJira Product Discovery—Confidence to build the right thing: https://atlassian.com/lenny/?utm_source=lennypodcast&utm_medium=paid-audio&utm_campaign=fy24q1-jpd-imcOrkes—The enterprise platform for reliable applications and agentic workflows: https://www.orkes.io/—Transcript: https://www.lennysnewsletter.com/p/how-google-built-ai-mode-in-under-a-year—My biggest takeaways (for paid newsletter subscribers): ⁠https://www.lennysnewsletter.com/i/175041217/my-biggest-takeaways-from-this-conversation⁠—Where to find Robby Stein:• X: https://x.com/rmstein• LinkedIn: https://www.linkedin.com/in/robbystein/—Referenced:• Google Gemini: https://gemini.google.com/app• Nano Banana: https://aistudio.google.com/models/gemini-2-5-flash-image• Chat GPT: https://chatgpt.com/• Perplexity: https://www.perplexity.ai/• Google Lens: https://lens.google/• AI Google search: https://www.google.com/ai• Why ChatGPT will be the next big growth channel (and how to capitalize on it) | Brian Balfour (Reforge): https://www.lennysnewsletter.com/p/why-chatgpt-will-be-the-next-big-growth-channel-brian-balfour• Alex Rampell on X: https://x.com/arampell• A 4-step framework for building delightful products | Nesrine Changuel (Spotify, Google, Skype): https://www.lennysnewsletter.com/p/a-4-step-framework-for-building-delightful-products• Look broader, look closer, think younger: Tony Fadell speaks at TED2015: https://blog.ted.com/look-broader-look-closer-think-younger-tony-fadell-speaks-at-ted2015/• Jobs to Be Done: https://www.christenseninstitute.org/theory/jobs-to-be-done/• The ultimate guide to JTBD | Bob Moesta (co-creator of the framework): https://www.lennysnewsletter.com/p/the-ultimate-guide-to-jtbd-bob-moesta• Rinstagram or Finstagram? The curious duality of the modern Instagram user: https://www.theguardian.com/technology/2016/sep/26/rinstagram-finstagram-instagram-accounts• V03: https://v03ai.com/• Pirate GPT: https://www.kickstarter.com/projects/silentmeditation/pirate-gpt/• The Bear on Hulu: https://www.hulu.com/series/the-bear-05eb6a8e-90ed-4947-8c0b-e6536cbddd5f• Dune on HBO Max: https://www.hbomax.com/movies/dune/e7dc7b3a-a494-4ef1-8107-f4308aa6bbf7• Top Gun: Maverick: https://www.imdb.com/title/tt1745960/• Purple pillows: https://purple.com/pillows• Avocado pillow: https://www.avocadogreenmattress.com/products/green-pillow• Justin Bieber’s website: https://www.justinbiebermusic.com/• Scooter Braun’s website: https://scooterbraun.com/—Recommended books:• Competing Against Luck: The Story of Innovation and Customer Choice: https://www.amazon.com/Competing-Against-Luck-Innovation-Customer/dp/0062435612• The Design of Everyday Things: https://www.amazon.com/Design-Everyday-Things-Revised-Expanded/dp/0465050654• Aurora: https://www.amazon.com/Aurora-High-Stakes-Survival-Navigate-Darkness/dp/0062916475• Project Hail Mary: https://www.amazon.com/Project-Hail-Mary-Andy-Weir/dp/0593135202—Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email [email protected].—Lenny may be an investor in the companies discussed. To hear more, visit www.lennysnewsletter.com

Jason Droege is the CEO of Scale AI, a company that provides foundational training data to every major AI lab. He previously co-founded Scour with Travis Kalanick and built Uber Eats from idea to $20 billion in revenue. In this conversation, Jason shares lessons from getting sued for $250 billion, discovering restaurant economics by weighing sandwich ingredients, and over 25 years of launching transformative technology businesses.What you’ll learn:What actually happened with Meta’s $14 billion investment in Scale AIWhy AI models still need human experts to improve, and how that relationship is evolvingHow AI models learn from experts building websites and debugging codeThe business lessons from building Uber Eats from zero to $20 billionWhy most enterprise data is useless for AI models todayWhy urgent daily problems beat super-valuable occasional problems when building productsHow to think independently when building new products and businesses—Brought to you by:Merge—The fastest way to ship 220+ integrations: http://merge.dev/lennyFigma Make—A prompt-to-code tool for making ideas real: https://www.figma.com/lenny/Mercury—The art of simplified finances: https://mercury.com/—Transcript: https://www.lennysnewsletter.com/p/first-interview-with-scale-ais-ceo-jason-droege—My biggest takeaways (for paid newsletter subscribers): https://www.lennysnewsletter.com/i/174979621/my-biggest-takeaways-from-this-conversation—Where to find Jason Droege:• X: https://x.com/jdroege• LinkedIn: https://www.linkedin.com/in/jasondroege/—Where to find Lenny:• Newsletter: https://www.lennysnewsletter.com• X: https://twitter.com/lennysan• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/—In this episode, we cover:(00:00) Introduction to Jason Droege(06:01) Jason’s early career and lessons learned(10:27) The current state of Scale AI(12:37) The shift to expert data labeling(17:02) Challenges and strategies in finding experts(18:48) Reinforcement learning and AI environments(28:18) The future of AI and human involvement(31:21) The role of evals(35:25) What AI models will look like in the next few years(41:43) Building Uber Eats and understanding customer needs(48:19) The importance of independent thinking(50:45) Setting high standards for new businesses(53:03) Exploring and selecting business ideas(57:07) The McDonald’s story(01:00:13) The role of gross margins in business feasibility(01:04:49) Why Jason says, “Not losing is a precursor to winning”(01:09:12) Hiring and building teams(01:12:11) AI corner(01:14:47) Lightning round and final thoughts—Referenced:• Travis Kalanick on X: https://x.com/travisk• Scour: https://en.wikipedia.org/wiki/Scour_Inc.• Scale: https://scale.com/• Alexandr Wang on X: https://x.com/alexandr_wang• Why experts writing AI evals is creating the fastest-growing companies in history | Brendan Foody (CEO of Mercor): https://www.lennysnewsletter.com/p/experts-writing-ai-evals-brendan-foody• Brendan Foody’s post on X about knowledge work changing: https://x.com/BrendanFoody/status/1970163503702188048• MIT Finds 95% of GenAI Pilots Fail Because Companies Avoid Friction: https://www.forbes.com/sites/jasonsnyder/2025/08/26/mit-finds-95-of-genai-pilots-fail-because-companies-avoid-friction/• Uber Eats: https://www.ubereats.com/• Stephen Chau on X: https://x.com/thestephenchau• a16z Podcast: https://a16z.com/podcasts/a16z-podcast/• F1: The Movie: https://www.imdb.com/title/tt16311594/• V03: https://v03ai.com/• Careers at Scale: https://scale.com/careers—Recommended books:• The Selfish Gene: https://www.amazon.com/Selfish-Gene-Anniversary-Introduction/dp/0199291152• The Road Less Traveled: A New Psychology of Love, Traditional Values, and Spiritual Growth: https://www.amazon.com/Road-Less-Traveled-Timeless-Traditional/dp/0743243153/• Good to Great: Why Some Companies Make the Leap . . . And Others Don’t: https://www.amazon.com/Good-Great-Some-Companies-Others/dp/0066620996• Thinking, Fast and Slow: https://www.amazon.com/Thinking-Fast-Slow-Daniel-Kahneman/dp/0374533555/—Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email [email protected].—Lenny may be an investor in the companies discussed. To hear more, visit www.lennysnewsletter.com

Albert Cheng has led growth at three of the world’s most successful consumer subscription companies: Duolingo, Grammarly, and Chess.com. A former Google product manager (and serious pianist!), Albert developed a unique approach to finding and scaling growth opportunities through rapid experimentation and deep user psychology. His teams run 1,000 experiments a year, discovering counterintuitive insights that have driven tens of millions in revenue.What you’ll learn:1. How to use the explore-exploit framework to find new growth opportunities2. How showing premium features to free users doubled Grammarly’s upgrades to paid plans3. What good retention looks like for a consumer subscription app4. Why resurrected users drive 80% of mature product growth5. Why “reverse trials” work better than time-based trials6. The three pillars of successful gamification: core loop, metagame, and profile —Brought to you by:Vanta—Automate compliance. Simplify security.Jira Product Discovery—Confidence to build the right thingMiro—A collaborative visual platform where your best work comes to life—Where to find Albert Cheng:• X: https://x.com/albertc248• LinkedIn: https://www.linkedin.com/in/albertcheng1/• Chess.com: https://www.chess.com/member/Goniners—Where to find Lenny:• Newsletter: https://www.lennysnewsletter.com• X: https://twitter.com/lennysan• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/—Referenced:• How Duolingo reignited user growth: https://www.lennysnewsletter.com/p/how-duolingo-reignited-user-growth• Inside ChatGPT: The fastest-growing product in history | Nick Turley (Head of ChatGPT at OpenAI): https://www.lennysnewsletter.com/p/inside-chatgpt-nick-turley• Explore vs. Exploit: https://brianbalfour.com/quick-takes/explore-vs-exploit• Grammarly: https://www.grammarly.com/• Reforge: https://www.reforge.com/• Chess.com: https://www.chess.com/• Everyone’s an engineer now: Inside v0’s mission to create a hundred million builders | Guillermo Rauch (founder & CEO of Vercel, creators of v0 and Next.js): https://www.lennysnewsletter.com/p/everyones-an-engineer-now-guillermo-rauch• Building Lovable: $10M ARR in 60 days with 15 people | Anton Osika (CEO and co-founder): https://www.lennysnewsletter.com/p/building-lovable-anton-osika• Figma: https://www.figma.com/• Cursor: https://cursor.com/• The rise of Cursor: The $300M ARR AI tool that engineers can’t stop using | Michael Truell (co-founder and CEO): https://www.lennysnewsletter.com/p/the-rise-of-cursor-michael-truell• Claude Code: https://www.anthropic.com/claude-code• GitHub Copilot: https://github.com/features/copilot• Noam Lovinsky on LinkedIn: https://www.linkedin.com/in/noaml/• The happiness and pain of product management | Noam Lovinsky (Grammarly, Facebook, YouTube, Thumbtack): https://www.lennysnewsletter.com/p/the-happiness-and-pain-of-product• Kyla Siedband on LinkedIn: https://www.linkedin.com/in/kylasiedband/• The Duolingo handbook: https://blog.duolingo.com/handbook/• Lenny’s post on X about the Duolingo handbook: https://x.com/lennysan/status/1889008405584683091• The rituals of great teams | Shishir Mehrotra of Coda, YouTube, Microsoft: https://www.lennysnewsletter.com/p/the-rituals-of-great-teams-shishir• Duolingo on TikTok: https://www.tiktok.com/@duolingo• Kasparov vs. Deep Blue | The Match That Changed History: https://www.chess.com/article/view/deep-blue-kasparov-chess• Magnus Carlsen: https://en.wikipedia.org/wiki/Magnus_Carlsen• Elo rating system: https://www.chess.com/terms/elo-rating-chess• Stockfish: https://en.wikipedia.org/wiki/Stockfish_(chess)• AlphaGo on Prime Video: https://www.primevideo.com/detail/AlphaGo/0KNQHKKDAOE8OCYKQS9WSSDYN0• Statsig: https://www.statsig.com/• The State of Product in 2026: Navigating Change, Challenge, and Opportunity: https://www.atlassian.com/blog/announcements/state-of-product-2026• Erik Allebest on LinkedIn: https://www.linkedin.com/in/erikallebest/• Daniel Rensch on X: https://x.com/danielrensch• Chariot: https://en.wikipedia.org/wiki/Chariot_(company)• San Francisco 49ers: https://www.49ers.com/• Breville Barista Express: https://www.breville.com/en-us/product/bes870—Recommended books:• Snuggle Puppy!: A Little Love Song: https://www.amazon.com/Snuggle-Puppy-Little-Boynton-Board/dp/1665924985• Ogilvy on Advertising: https://www.amazon.com/Ogilvy-Advertising-David/dp/039472903X• Dark Squares: How Chess Saved My Life: https://www.amazon.com/Dark-Squares-Chess-Saved-Life/dp/1541703286—Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email [email protected] may be an investor in the companies discussed. To hear more, visit www.lennysnewsletter.com

Nesrine Changuel helped build Spotify, Google Chrome, and Google Meet. Her work has helped her discover the importance of emotional connection in building successful products. At Google, she served as a dedicated “delight PM,” a role specifically focused on making products more delightful. She recently published Product Delight, a book that provides a practical framework for creating products that serve both functional and emotional needs. Based in Paris, she now coaches founders and CPOs on implementing delight strategies in their organizations.What you’ll learn:1. Why delight is a business strategy, not just “sprinkling confetti” on top of functionality2. How to identify emotional motivators that drive product retention3. The 50-40-10 rule for balancing delight in your roadmap4. The 4-step delight model5. The origin story of Spotify’s Discover Weekly6. Why B2B products need delight just as much as B2C products7. How to get buy-in from skeptical leaders who think delight is a luxury—Brought to you by:DX—The developer intelligence platform designed by leading researchers: https://getdx.com/lennyJira Product Discovery—Confidence to build the right thing: https://atlassian.com/lennyLucidLink—Real-time cloud storage for teams: https://www.lucidlink.com/lenny—Transcript: https://www.lennysnewsletter.com/p/a-4-step-framework-for-building-delightful-products—My biggest takeaways (for paid newsletter subscribers): https://www.lennysnewsletter.com/i/174199489/my-biggest-takeaways-from-this-conversation—Where to find Nesrine Changuel:• LinkedIn: https://www.linkedin.com/in/nesrinechanguel/• Newsletter: https://nesrinechanguel.substack.com/• Website: https://nesrine-changuel.com/—Where to find Lenny:• Newsletter: https://www.lennysnewsletter.com• X: https://twitter.com/lennysan• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/—In this episode, we cover:(00:00) Introduction to Nesrine and product delight(04:56) Why delight matters(09:17) What makes a feature “delightful”(12:29) The three pillars of delight(13:03) Pillar 1: Removing friction (Uber refund example)(15:07) Pillar 2: Anticipating needs (Revolut eSIM example)(17:21) Pillar 3: Exceeding expectations (Edge coupon example)(18:35) The “confetti effect” and when it actually works(22:02) B2B vs. B2C: Why all products need emotional connection(29:52) The Delight Model: A 4-step framework(30:57) Step 1: Identifying user motivators (functional and emotional)(33:55) Step 2: Converting motivators into product opportunities(34:46) Step 3: Identifying solutions with the delight grid(36:46) Step 4: Validating ideas with the delight checklist(40:22) The Delight Model summarized(42:18) The importance of familiarity (Spotify Discover Weekly story)(45:21) Real examples: Chrome’s tab management solution(51:32) Google Meet’s solution for “Zoom fatigue”(55:02) Getting buy-in from skeptical leaders(59:39) Prioritizing delight: The 50-40-10 rule(1:02:41) Creating a culture of delight in your organization(1:06:45) The habituation effect(1:08:15) When delight goes wrong: Apple reactions example(1:10:21) How delight motivates product teams(1:12:24) Lightning round and final thoughts—Referenced:• Spotify: https://open.spotify.com/• Linear: https://linear.app/• How Linear builds product: https://www.lennysnewsletter.com/p/how-linear-builds-product• Jira: https://www.atlassian.com/software/jira• Asana: https://asana.com/• Monday: https://monday.com/• The Product Delight Model: https://nesrinechanguel.substack.com/p/the-product-delight-model• Revolut: https://www.revolut.com/• How Revolut trains world-class product managers: The “local CEO” model, raw intellect over experience, and a cultural obsession with building wow products | Dmitry Zlokazov (Head of Product): https://www.lennysnewsletter.com/p/how-revolut-trains-world-class-product-managers• Microsoft Cashback: https://www.microsoft.com/en-us/edge/features/shopping-cashback• Superhuman’s secret to success: Ignoring most customer feedback, manually onboarding every new user, obsessing over every detail, and positioning around a single attribute: speed | Rahul Vohra (CEO): https://www.lennysnewsletter.com/p/superhumans-secret-to-success-rahul-vohra• Brian Chesky’s secret mentor who died 9 times, started the Burning Man board, and built the world’s first midlife wisdom school | Chip Conley (founder of MEA): https://www.lennysnewsletter.com/p/chip-conley• Workday: https://www.workday.com/• SAP: https://www.sap.com/• ServiceNow: https://www.servicenow.com/• Salesforce: https://www.salesforce.com/• GitHub: https://github.com/• Atlassian: https://www.atlassian.com/• Snowflake: https://www.snowflake.com/• Data Superheroes: https://www.snowflake.com/en/data-superheroes/• Google Meet: https://meet.google.com/• Andy Nesling on LinkedIn: https://www.linkedin.com/in/andynesling/• Matic: https://maticrobots.com/• Diego Sanchez’s (Senior Product Manager at Buffer) post on LinkedIn: https://www.linkedin.com/feed/update/urn:li:activity:7365014292091346945/• Miro: https://miro.com/• Arc browser: https://arc.net/• Competing with giants: An inside look at how The Browser Company builds product | Josh Miller (CEO): https://www.lennysnewsletter.com/p/competing-with-giants-an-inside-look• Migros Supermarket: https://www.migros.ch/• The rise of Cursor: The $300M ARR AI tool that engineers can’t stop using | Michael Truell (co-founder and CEO): https://www.lennysnewsletter.com/p/the-rise-of-cursor-michael-truell• Building Lovable: $10M ARR in 60 days with 15 people | Anton Osika (CEO and co-founder): https://www.lennysnewsletter.com/p/building-lovable-anton-osika• Linear’s secret to building beloved B2B products | Nan Yu (Head of Product): https://www.lennysnewsletter.com/p/linears-secret-to-building-beloved-b2b-products-nan-yu• Suno: https://suno.com• Snapchat: https://www.snapchat.com/• Use Reactions, Presenter Overlay, and other effects when videoconferencing on Mac: https://support.apple.com/en-us/105117• Dr. Lipp: https://drlipp.com/• How to be the best coach to product people | Petra Wille (Strong Product People): https://www.lennysnewsletter.com/p/how-to-be-the-best-coach-to-product• The Great American Baking Show: https://www.imdb.com/title/tt21822674/• Le Meilleur Pâtissier: https://en.wikipedia.org/wiki/Le_Meilleur_P%C3%A2tissier• The Upside on Amazon Prime: https://www.amazon.com/gp/video/detail/amzn1.dv.gti.3cb8500f-31af-9f4f-5dec-701e086d58e8• The Intouchables: https://www.imdb.com/title/tt1675434/• Yoyo stroller: https://www.stokke.com/USA/en-us/category/strollers/yoyo-strollers• UppaBaby strollers: https://uppababy.com/strollers/—Recommended books:• Product Delight: How to Make Your Product Stand Out with Emotional Connection: https://www.amazon.com/Product-Delight-Stand-Emotional-Connection-ebook/dp/B0FGZ93D9Y/• Factfulness: Ten Reasons We’re Wrong About the World—and Why Things Are Better Than You Think: https://www.amazon.com/Factfulness-Reasons-World-Things-Better/dp/1250107814• STRONG Product Communities: The Essential Guide to Product Communities of Practice: https://www.amazon.com/STRONG-Product-Communities-Essential-Practice/dp/3982235189/r—Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email [email protected] may be an investor in the companies discussed. To hear more, visit www.lennysnewsletter.com

Hamel Husain and Shreya Shankar teach the world’s most popular course on AI evals and have trained over 2,000 PMs and engineers (including many teams at OpenAI and Anthropic). In this conversation, they demystify the process of developing effective evals, walk through real examples, and share practical techniques that’ll help you improve your AI product.What you’ll learn:1. WTF evals are2. Why they’ve become the most important new skill for AI product builders3. A step-by-step walkthrough of how to create an effective eval4. A deep dive into error analysis, open coding, and axial coding5. Code-based evals vs. LLM-as-judge6. The most common pitfalls and how to avoid them7. Practical tips for implementing evals with minimal time investment (30 minutes per week after initial setup)8. Insight into the debate between “vibes” and systematic evals—Brought to you by:Fin—The #1 AI agent for customer serviceDscout—The UX platform to capture insights at every stage: from ideation to productionMercury—The art of simplified finances—Where to find Shreya Shankar• X: https://x.com/sh_reya• LinkedIn: https://www.linkedin.com/in/shrshnk/• Website: https://www.sh-reya.com/• Maven course: https://bit.ly/4myp27m—Where to find Hamel Husain• X: https://x.com/HamelHusain• LinkedIn: https://www.linkedin.com/in/hamelhusain/• Website: https://hamel.dev/• Maven course: https://bit.ly/4myp27m—In this episode, we cover:(00:00) Introduction to Hamel and Shreya(04:57) What are evals?(09:56) Demo: Examining real traces from a property management AI assistant(16:51) Writing notes on errors(23:54) Why LLMs can’t replace humans in the initial error analysis(25:16) The concept of a “benevolent dictator” in the eval process(28:07) Theoretical saturation: when to stop(31:39) Using axial codes to help categorize and synthesize error notes(44:39) The results(46:06) Building an LLM-as-judge to evaluate specific failure modes(48:31) The difference between code-based evals and LLM-as-judge(52:10) Example: LLM-as-judge(54:45) Testing your LLM judge against human judgment(01:00:51) Why evals are the new PRDs for AI products(01:05:09) How many evals you actually need(01:07:41) What comes after evals(01:09:57) The great evals debate(1:15:15) Why dogfooding isn’t enough for most AI products(01:18:23) OpenAI’s Statsig acquisition(1:23:02) The Claude Code controversy and the importance of context(01:24:13) Common misconceptions around evals(1:22:28) Tips and tricks for implementing evals effectively(1:30:37) The time investment(1:33:38) Overview of their comprehensive evals course(1:37:57) Lightning round and final thoughts—LLM Log Open Codes Analysis Prompt:Please analyze the following CSV file. There is a metadata field which has an nested field called z_note that contains open codes for analysis of LLM logs that we are conducting. Please extract all of the different open codes. From the _note field, propose 5-6 categories that we can create axial codes from.—Referenced:• Building eval systems that improve your AI product: https://www.lennysnewsletter.com/p/building-eval-systems-that-improve• Mercor: https://mercor.com/• Brendan Foody on LinkedIn: https://www.linkedin.com/in/brendan-foody-2995ab10b• Nurture Boss: https://nurtureboss.io/• Braintrust: https://www.braintrust.dev/• Andrew Ng on X: https://x.com/andrewyng• Carrying Out Error Analysis: https://www.youtube.com/watch?v=JoAxZsdw_3w• Julius AI: https://julius.ai/• Brendan Foody on X—“evals are the new PRDs”: https://x.com/BrendanFoody/status/1939764763485171948• Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferences: https://dl.acm.org/doi/abs/10.1145/3654777.3676450• Lenny’s post on X about evals: https://x.com/lennysan/status/1909636749103599729• Statsig: https://statsig.com/• Claude Code: https://www.anthropic.com/claude-code• Cursor: https://cursor.com/• Occam’s razor: https://en.wikipedia.org/wiki/Occam%27s_razor• Frozen: https://www.imdb.com/title/tt2294629/• The Wire on HBO: https://en.wikipedia.org/wiki/The_Wire—Recommended books:• Pachinko: https://www.amazon.com/Pachinko-National-Book-Award-Finalist/dp/1455563935• Apple in China: The Capture of the World’s Greatest Company: https://www.amazon.com/Apple-China-Capture-Greatest-Company/dp/1668053373/• Machine Learning: https://www.amazon.com/Machine-Learning-Tom-M-Mitchell/dp/1259096955• Artificial Intelligence: A Modern Approach: https://www.amazon.com/Artificial-Intelligence-Modern-Approach-Global/dp/1292401133/Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email [email protected].—Lenny may be an investor in the companies discussed.My biggest takeaways from this conversation: To hear more, visit www.lennysnewsletter.com

Julie Zhuo is the former VP and Head of Design at Facebook (now Meta), author of the bestselling book The Making of a Manager, and co-founder of Sundial, an AI-powered data analysis company. Also, my first-ever podcast guest over 3 years ago!In our conversation, we discuss:1. The three core manager skills that translate directly to managing AI agents2. How her team uses AI to learn new skills 10x faster3. The “diagnose with data, treat with design” framework for balancing gut and data4. Why hypergrowth AI companies have terrible data infrastructure (and why it doesn’t matter)5. How to give feedback that actually lands—including Julie’s exact script for difficult conversations6. What Julie’s teaching her kids about an AI future (hint: it’s not coding or STEM)—Brought to you by:Mercury — The art of simplified financesDX — The developer intelligence platform designed by leading researchersPostHog—How developers build successful products—Transcript: https://www.lennysnewsletter.com/p/from-managing-people-to-managing-ai-julie-zhuo—My biggest takeaways (for paid newsletter subscribers): https://www.lennysnewsletter.com/i/172723725/my-biggest-takeaways-from-this-conversation—Where to find Julie Zhuo:• X: https://x.com/joulee• LinkedIn: https://www.linkedin.com/in/julie-zhuo/• Website: https://www.juliezhuo.com/• Newsletter: https://lg.substack.com/• Sundial: https://sundial.so/—Where to find Lenny:• Newsletter: https://www.lennysnewsletter.com• X: https://twitter.com/lennysan• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/—In this episode, we cover:(00:00) Welcome back, Julie!(05:18) The success of The Making of a Manager(08:41) Why AI will make everyone a manager(11:38) The future of management roles(14:00) Empowering teams with AI(21:30) Specific roles being accelerated by AI(26:53) Data analysis in AI companies(32:02) The role of data in design(37:21) The evolving role of managers in the AI era(40:22) Embracing change and uncertainty(42:14) Timeless lessons for managers(49:03) Balancing strengths and weaknesses(57:49) Building a feedback culture(01:05:33) Creating win-win situations(01:09:27) Being aware of your own energy and conviction(01:12:12) Navigating disagreements with higher-ups(01:15:57) AI corner(01:20:08) Contrarian corner(01:23:14) Lightning round and final thoughts—Referenced:• Julie Zhuo on accelerating your career, impostor syndrome, writing, building product sense, using intuition vs. data, hiring designers, and moving into management: https://www.lennysnewsletter.com/p/episode-2-julie-zhuo• Waymo: https://waymo.com/• How we restructured Airtable’s entire org for AI | Howie Liu (co-founder and CEO): https://www.lennysnewsletter.com/p/how-we-restructured-airtables-entire-org-for-ai• Cursor: https://cursor.com/• The rise of Cursor: The $300M ARR AI tool that engineers can’t stop using | Michael Truell (co-founder and CEO): https://www.lennysnewsletter.com/p/the-rise-of-cursor-michael-truell• Inside ChatGPT: The fastest growing product in history | Nick Turley (Head of ChatGPT at OpenAI): https://www.lennysnewsletter.com/p/inside-chatgpt-nick-turley• Behind the founder: Marc Benioff: https://www.lennysnewsletter.com/p/behind-the-founder-marc-benioff• OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter): https://www.lennysnewsletter.com/p/kevin-weil-open-ai• Anthropic’s CPO on what comes next | Mike Krieger (co-founder of Instagram): https://www.lennysnewsletter.com/p/anthropics-cpo-heres-what-comes-next• The Magic Loop: https://www.lennysnewsletter.com/p/the-magic-loop• Dunning-Kruger effect: https://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect• Eric Antonow on LinkedIn: https://www.linkedin.com/in/antonow/• Methaphone: https://methaphone.com/• Replit: https://replit.com/• “Baby” by Justin Bieber on Spotify: https://open.spotify.com/track/6epn3r7S14KUqlReYr77hA• Kingdom Rush: https://www.kingdomrush.com/• Dr. Becky on TikTok: https://www.tiktok.com/@drbeckyatgoodinside• Emily Oster on TikTok: https://www.tiktok.com/@profemilyoster• La La Land on Netflix: https://www.netflix.com/title/80095365• Granola: https://www.granola.ai/• Matic robots: https://maticrobots.com/• Limitless pendant: https://www.limitless.ai/• How I AI: https://www.youtube.com/@howiaipodcast—Recommended books:• The Making of a Manager: What to Do when Everyone Looks to You: https://www.amazon.com/Making-Manager-What-Everyone-Looks/dp/0525540423• High Output Management: https://www.amazon.com/High-Output-Management-Andrew-Grove/dp/0679762884/• Zen and the Art of Motorcycle Maintenance: An Inquiry into Values: https://www.amazon.com/Zen-Art-Motorcycle-Maintenance-Inquiry/dp/0061673730• Conscious Business: How to Build Value Through Values: https://www.amazon.com/Conscious-Business-Build-through-Values/dp/1622032020• Good Inside: A Practical Guide to Resilient Parenting Prioritizing Connection Over Correction: https://www.amazon.com/Good-Inside-Guide-Becoming-Parent/dp/0063159481/—Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email [email protected] may be an investor in the companies discussed. To hear more, visit www.lennysnewsletter.com

Brendan Foody is the CEO and co-founder of Mercor, the fastest-growing company in history to go from $1M to $500M in revenue (in just 17 months!). At 22, he is also the youngest American unicorn founder ever. Mercor works with 6 of the Magnificent 7 and all top 5 AI labs to help them hire experts to create evaluations and training data that improve their models. In this conversation, Brendan explains why evals have become the critical bottleneck for AI progress, how he discovered this massive opportunity, and what the future of work might look like in an AI-driven economy.What you’ll learn:1. Why evals are becoming the primary bottleneck for AI progress and what this means for AI startups2. How Mercor grew to $500M revenue in 17 months (fastest in history)3. Brendan’s meeting with xAI that changed his company’s trajectory4. Which skills and jobs will remain most valuable as AI continues to advance (hint: jobs with “elastic” demand)5. Why Brendan believes AGI and superintelligence are not happening anytime soon6. The three unique core values that drove Mercor’s success7. How Harvard Lampoon writers are making Claude funnier—Brought to you by:WorkOS—Modern identity platform for B2B SaaS, free up to 1 million MAUsJira Product Discovery—Atlassian’s new prioritization and roadmapping tool built for product teamsEnterpret—Transform customer feedback into product growth—Transcript: https://www.lennysnewsletter.com/p/experts-writing-ai-evals-brendan-foody—My biggest takeaways (for paid newsletter subscribers): https://www.lennysnewsletter.com/i/173303790/my-biggest-takeaways-from-this-conversation—Where to find Brendan Foody:• X: https://x.com/BrendanFoody• LinkedIn: https://www.linkedin.com/in/brendan-foody-2995ab10b/—Where to find Lenny:• Newsletter: https://www.lennysnewsletter.com• X: https://twitter.com/lennysan• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/—In this episode, we cover:(00:00) Introduction to Brendan Foody and Mercor(05:38) The “era of evals”(09:26) Understanding the AI training landscape(17:10) The future of work and AI(25:54) The evolution of labor markets(29:55) Understanding how AI models are trained(38:58) Building Mercor(53:27) Lessons from past ventures(56:55) The future of AI and model improvement(01:00:41) His personal use of AI and final thoughts—References: https://www.lennysnewsletter.com/p/experts-writing-ai-evals-brendan-foody—Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email [email protected] may be an investor in the companies discussed. To hear more, visit www.lennysnewsletter.com