The science behind DORA | Derek DeBellis (Google)

Click here to view the episode transcript. In this episode, Emanuel Mueller Ramos, Head of Developer Experience at Skyscanner, discusses the evolution of his team as they transitioned from focusing on frameworks and middleware to becoming a customer-centric, impact-driven organization. Emanuel details the strategies he used to gain stakeholder buy-in, why it's crucial to rethink traditional productivity metrics, and how they made a cultural shift to prioritize developer happiness and effectiveness. This conversation highlights the steps necessary to build a developer experience function that delivers meaningful impact.Mentions and links:Follow Emanuel on LinkedInMeasuring developer productivity with the DX Core 4Discussion points:(1:14) The beginning of Skyscanner's developer productivity division(3:53) Gaining stakeholder buy-in and refocusing the teams(5:57) Redefining success metrics for developer productivity(8:57) Pitching the developer experience focus to leadership(17:26) Moving from frameworks to feedback loops(20:45) Fostering a customer-centric culture(23:20) Defining the collaboration between platform and developer experience teams(26:41) Choosing the right metrics for developer experience success (31:31) Risks and challenges ahead

Click here to view the episode transcript. In this week's episode, Abi is joined by industry leaders Idan Gazit from GitHub, Anna Sulkina from Airbnb, and Alix Melchy from Jumio. Together, they discuss the impact of GenAI tools on developer productivity, exploring challenges in measurement and enhancement. They delve into AI's evolving role in engineering, from overcoming friction points to exploring real-world applications and the future of technology. Gain insights into how AI-driven chat assistants are reshaping workflows and the vision for coding.Links: How to measure GenAI adoption and impactTimestamps:(2:58) Challenges of Measuring AI Productivity(6:02) Use cases for GenAI within the Airbnb developer organization(10:26) GitHub’s process for developing and testing new GenAI tools for developers(12:42) Driving GenAI adoption strategies at Airbnb(14:20) Research impact and productivity gains with GenAI tools at Airbnb(17:03) Copilot use cases surveyed among Jumio's developers(18:46) Challenges measuring impact of AI products at GitHub(21:33) Biggest gains of GenAI usage at Airbnb(24:19) Future opportunities in GenAI(30:31) Challenges in GenAI for developers

Click here to view the episode transcript. In this week's episode, Abi welcomes Jared Wolinsky, Vice President of Platform Engineering at SiriusXM, to delve into the inner workings of platform engineering at SiriusXM. Jared sheds light on their innovative approach to prioritizing projects, emphasizing alignment with overarching business goals. They explore how these strategies boost developer speed and drive technological advancement within the organization.Links: When is the right time to establish a DevProd team reportTimestamps:(1:39) SiriusXM's major rebuild(4:46) Challenges of building a platform during a major product revamp(7:22) Navigating trade-offs(10:06) Defining the ideal developer journey at SiriusXM(17:28) Navigating the path to a user-centric developer experience(23:05) Collaborating with leadership to iterate and gain approval(25:05) Balancing enablement and platform(28:28) Implementing a data-driven prioritization framework(34:29) Aligning projects with business goals

Click here to view the episode transcript'. In this episode, Michelle Swartz, Vice president of Developer Enablement American Express, shares insights on improving developer experience. She discusses the creation of an onboarding bootcamp and the development of the AmEx Way Library for better knowledge management. Michelle explains how AmEx balances standardization and flexibility with the concept of Paved Roads. She also highlights the importance of measuring success, fostering community, and elevating the company's tech credibility.Mentions and linksGenAI GuideTimestamps(5:45) Challenges of advocating for DevEx in non-tech companies(7:43) Importance of senior leadership buy-in for DevEx(9:58) Genesis of the DevEx organization and Jedi Council(12:12) Transition to a dedicated DevEx function(13:17) Formalizing investment in DevEx(18:02) Initial efforts and learning in improving DevEx(19:25) Using sentiment surveys to prioritize DevEx areas(27:26) Addressing knowledge management challenges(29:49) Balancing standardization and freedom in DevEx(36:21) Implementing Paved Roads: evolution vs. revolution

Click here to view the episode transcript'. This week’s episode is a recording from a recent event hosted by Abi Noda (CEO of DX) and Laura Tacho (CTO at DX). The episode begins with an overview of the DORA, SPACE, and DevEx frameworks, including where they overlap and common misconceptions about each. Laura and Abi discuss the advantages and drawbacks of each framework, then discuss how to choose which framework to use.Mentions and Links: Dora.devDiscussion points:2:50- DORA, SPACE, DevEx overview10:35- Choosing which framework to use13:15- Using DORA22:42-Using SPACE

Click here to view the episode transcript. In this week's episode, we welcome Derek DeBellis, lead researcher on Google's DORA team, for a deep dive into the science and methodology behind DORA's research. We explore Derek's background, his role at Google, and how DORA intersects with other research disciplines. Derek takes us through DORA's research process step by step, from defining outcomes and factors to survey design, analysis, and structural equation modeling.Mentions and Links:Derek DeBellis on LinkedInDX’s guide to measuring GenAI adoption and impact2023 Accelerate State of DevOps ReportDiscussion points:(3:00) Derek’s transition from Microsoft to the DORA team at Google(4:28) Derek talks about his connection to surveys(6:16) Derek’s journey to becoming a quantitative user experience researcher(7:48) Derek simplifies DORA(8:19) DORA - Philosophy vs practice(11:09) Understanding desired outcomes(12:45) Self reported outcomes vs objective outcomes(16:16) Derek and Abi discuss the nuances of literature review(19:57) Derek details survey development(27:55) Pretesting issues(29:30) Designing surveys for other companies(35:02) Derek simplifies model analysis and validation techniques(38:48) Benchmarks: Balancing data limitations with method sensitivity

Click here to view the episode transcript. This week we’re joined by Sean Mcllroy from Slack’s Release Engineering team to learn about how they’ve fully automated their deployment process. This conversation covers Slack’s original release process, key changes Sean’s team has made, and the latest challenges they’re working on today. Mentions and links:Read Sean’s blog post, The Scary Thing About DeploysFollow Sean on LinkedInTime Stamps:(1:34): The Release Engineering team(2:13): How the monolith has served Slack (3:24): How the deployment process used to work (6:23): The complexity of the deploy itself(7:39): Early ideas for improving the deployment process(9:07): Why anomaly detection is challenging(10:32): What a Z-score is(13:23): Managing noise with Z-scores(16:49): Presenting this information to people that need it(19:54): Taking humans out of the process(23:13): Handling rollbacks(25:27): Not overloading developers with information(28:26): Handling large deployments

Click here to view the episode transcript. This week’s episode is the recording of a live conversation between Abi and Chris Westerhold (Thoughtworks Head of Developer Experience). This conversation is useful for anyone early in their journey with developer portals or platforms: Abi and Chris discuss common approaches to solving these problems, pitfalls to avoid, building vs. buying, and more. Mentions and LinksFollow Chris on LinkedInWatch the recording of this conversation Watch part 2 of this conversation on the market landscapeLearn about PlatformX, DX’s product mentioned in the conversationTime Stamps:(3:09) Why there’s an increased interest in developer portals(5:33) Chris’ background with dev portals(6:37) Homegrown solutions for developer portals(9:22) How developer portal initiatives begin(11:24) Internal developer portal vs service catalogs and IDPs(16:18) Mistakes companies make with developer portals(21:05) Approaches to solving this problem(24:28) How can developer portals drive value(32:07) Common traps to avoid

Click here to listen to the episode transcript. On this week's episode, Abi interviews Kent Wills, Director of Engineering Effectiveness at Yelp.  He shares insights into the evolution of their developer productivity efforts over the past decade. From tackling challenges with their monolithic architecture to scaling productivity initiatives for over 1,300 developers. Kent also touches on his experience in building a business case for developer productivity.Discussion points:(1:42) Forming the developer productivity team(3:25) Naming the team engineering effectiveness(4:30) Getting leadership buy-in for focusing on this work(7:54) Managing code ownership in Yelp’s monolith(12:23) Supporting the design system(16:00) The business case for forming a dedicated team (19:45) How to standardize (23:50) How their approach to standardization might be different in another company(27:08) Demonstrating the value of their work (32:21) Building an insights platform(38:47) How Yelp is using LLM’sMentions and LinksConnect with Kent Wills on LinkedInWatch Kent’s 2023 talk at ElevateListen to the interview with Peter Seibel (“Let 1,000 flowers bloom”)Download the recently published benchmarks on developer productivity team headcount

This week we’re joined by Gail Carmichael, Principal Instructional Engineer at Splunk. At Splunk, Gail’s team is responsible for improving developer onboarding, which they do through a multi-day learning program. Here, Gail shares how this program works and how they measure developer onboarding. The conversation also covers what instructional engineers are generally, and how Gail demonstrates the impact of her team’s work. Discussion points:(1:16) The Engineering Enablement & Engagement Team at Splunk(8:01) What an Instructional Engineer is(14:36) The developer onboarding program at Splunk(16:05) Components of a good onboarding program(21:11) Why having an onboarding program matters(28:17) Measuring onboarding at Shopify (Gail’s previous company)(31:39) Measuring developer onboarding at SplunkMentions and LinksConnect with Gail on LinkedInDownload the report on Developer productivity metrics at top tech companies