It Depends: Lessons in Technology Leadership

When Kevin Goldsmith joined Microsoft, new graduates were often pushed into Software Development Engineer in Test, with the promise they could become “real” developers later. Decades later, a startup leader he mentors is asking a version of the same question about a senior manual tester on a team where agents now write much of the code and many of the tests: is this role worth what it costs? Kevin’s answer traces how the QA role has been rebuilt over his career. It starts with Microsoft’s adversarial test org, moves through a startup whose Extreme Programming champion dismissed manual testing, the industry’s shift to offshore testing, and continues to Adobe, where testers were involved from the start of every project, and Spotify, where they were full members of the squad. The pattern he finds is about context. The value of testers, and how valuable they were seen to be, followed how much context they had and how close they sat to the work. Agents have their own kind of context, and Kevin explains why he’s come to think of an agent as the ultimate offshore tester. He offers three quality questions for checking whether a team still has holistic quality judgment, whoever owns it: who decides what matters, who sees the whole, and who uses it like a customer. He also covers how to evaluate your existing testers, why you should staff quality to risk rather than headcount, and three ways leaders undermine quality while trying to move faster. It’s for engineering leaders deciding what to do with their QA function as agents take on more testing, and for testers figuring out where their value lies now. Other episodes referenced Fail Safe, Fail Smart... Succeed!: https://itdependspod.com/episodes/fail-safe-fail-smart-succeed/ The Interesting Changes Aren’t Happening in the IDE: https://itdependspod.com/episodes/the-interesting-changes-arent-happening-in-the-ide/ Paying for Nothing to Happen: Budgeting for Security, Compliance, and Reliability: https://itdependspod.com/episodes/paying-for-nothing-to-happen-budgeting-for-security-compliance-and-reliability/ Send questions for the holiday Q&A episodes: [email protected], or on Bluesky at https://bsky.app/profile/kevingoldsmith.com Listener Survey: https://forms.gle/JVeKHsHJKhEM3dvK6 The book: It Depends: Writing on Technology Leadership 2012-2022 (https://itdependsbook.net) Your host: Kevin Goldsmith (https://kevingoldsmith.com)

Someone gets into a company’s systems, takes over its connection to an email vendor, and starts sending phishing emails. Everything else stops. The head of security runs the response, which is exactly right. The CTO’s job turns out to be something different, and Kevin Goldsmith argues it’s the part of crisis work that almost nobody writes about. The episode separates crisis response, fixing the problem, from crisis leadership, everything around the fix. Kevin draws on that security incident at a company where he was CTO and on Onfido during COVID. At Onfido, a fully in-office company with a large BPO operation, had to go remote almost overnight, and he skipped a step he already knew to take: putting a business continuity plan in place before he needed one. From those two crises, he lays out five jobs for the leader during the response: name the owner, clear the path, set the rhythm, keep it calm, and save the why for later. He also gets practical about planning for the categories of crisis that hit almost everyone, knowing who outside the company will be in it with you, what a complete update actually contains, and why the person running the response shouldn’t also be the one fielding everyone’s questions. For crises that don’t end, like a pandemic, one more job remains. At some point someone has to tell people the emergency is over, even if the situation isn’t. Show notes The Google SRE book, “Managing Incidents” chapter (free online): https://sre.google/sre-book/managing-incidents/ Site Reliability Engineering (O’Reilly): https://link.amazon/B03QgOSJp (Amazon link) Every Organization Is a System: https://itdependspod.com/episodes/every-organization-is-a-system-are-you-designing-it-or-just-living-in-it/ Don’t Let Your Boss Do Your Job: https://itdependspod.com/episodes/dont-let-your-boss-do-your-job/ The Conversation Before the Conversation: https://itdependspod.com/episodes/the-conversation-before-the-conversation/ Fail Safe, Fail Smart... Succeed!: https://itdependspod.com/episodes/fail-safe-fail-smart-succeed/ Listener Survey: https://forms.gle/JVeKHsHJKhEM3dvK6 The book: It Depends: Writing on Technology 2012-2022 (https://itdependsbook.net) Your host: Kevin Goldsmith (https://kevingoldsmith.com)

A leader is running an initiative, and at an executive meeting his boss gives three of his peers time to talk about it. He gets added at the end, almost as an afterthought. A few days later, Kevin Goldsmith had his own version: a meeting he’d contributed slides to went ahead without him. Both of them were reacting to the small part of the situation they could see. The obvious question is whether the boss meant it. The more useful one is how to read someone whose context you mostly can’t see. That starts with understanding the game, which Kevin defines as a company’s real system and culture: what actually gets rewarded, and how decisions really get made. Kevin draws on eight years at Microsoft that trained him to read every interaction as political, and a moment at Adobe when that wiring led him to badly misread his own boss. He lays out a five-step framework for telling a slight from a signal. It starts with separating what you saw from the story you told yourself, and ends with deciding whether the game is one you want to play. Along the way, he brings in logical empathy and David Foster Wallace’s idea of default settings. The episode also turns the question around. Managers create these moments for their own teams, usually through omission, and Kevin shares a time he got it wrong, and what he does now to close the loops his reports can’t see. It’s for engineering leaders who answer to a boss and have people answering to them. John Godfrey Saxe, “The Blind Men and the Elephant” (1872) Episode: “Every Organization Is a System. Are You Designing It or Just Living in It?” (Mar 2026) Episode: “Values → Culture → Everything: Why Company Culture Actually Matters” (Sept 2025) Chris Argyris’s Ladder of Inference Episode: “Logical Empathy” David Foster Wallace, “This Is Water,” Kenyon College commencement (2005) Listener Survey: https://forms.gle/JVeKHsHJKhEM3dvK6 The book: It Depends: Writing on Technology 2012-2022 (https://itdependsbook.net) Your host: Kevin Goldsmith (https://kevingoldsmith.com) The theme song artist: Answers to April

A CTO needs to hire more engineers, and the numbers are working against the argument. Engineering spend has fallen as a share of revenue because the company grew faster than the team, so on paper it looks like efficiency. In practice, one resignation could sink the roadmap. Kevin Goldsmith's advice was to stop arguing from within and examine the organization as an acquirer would. Kevin has led due diligence as an acquirer at large companies and startups, and he's been examined as a target through acquisitions and investor rounds. From both seats, the same pattern keeps showing up: the organizations that come through well are the ones whose leaders already know where the problems are and have a plan for them. The catch is that the longer a leader has been in an organization, the less its workarounds look like problems. The framework here is the Outsider's Audit, four questions anyone doing diligence will ask, turned on your own team: Worth, Dependencies, Flow, and Risk. Kevin walks through each one practically, including building spare capacity at the org level so a resignation doesn't cancel a quarter of work, tracing a single ticket to find where work actually stalls, trading audits with a peer, and handling the people side, right up to the moment the team starts wondering if the company is being sold. The episode is aimed at CTOs and VPs who may face funding or acquisition diligence someday, but the approach works just as well for a director or engineering manager who needs to make the case for their team with evidence rather than instinct. Due Diligence in Funding & Acquisition, A CTO's Guide Listener Survey: https://forms.gle/JVeKHsHJKhEM3dvK6 The book: It Depends: Writing on Technology 2012-2022 (https://itdependsbook.net) Your host: Kevin Goldsmith (https://kevingoldsmith.com) The theme song artist: Answers to April

A vendor the security team wanted, budgeted two years in a row, and never actually bought. When Kevin Goldsmith finally tried to spend the money, his CFO asked what the ROI was, and he didn't have an answer good enough to win the argument. Not long after, a new board arrived and told him he was underinvesting in cybersecurity. Same company, same customers, same risk, completely different verdict. Every technology leader owns budgets where the best possible outcome is that nothing happens. Security, compliance, reliability, backups, disaster recovery. They're the easiest money in the company to cut and the hardest to defend, because success looks identical to having never spent the money at all. This episode is about how to decide how much to spend, and how to defend the number when finance asks. Kevin lays out four price tags that other people are already attaching to your risk: the deal price your customers set, the insurance price your insurer sets, the compliance price regulators set, and the survival price you have to estimate yourself. He compares three companies where the right answer ranged from a large security team to a fraction of one person, revisits COVID as the disaster recovery test nobody scheduled, and covers the failure modes that show up again and again, including the raided line item and the backup nobody has ever restored from. Useful for CTOs, VPs of engineering, and directors heading into a budgeting cycle who need a defensible number rather than a benchmark, and for anyone who has been told in the same year that they're spending too much and not enough. Becoming a Business Leader: https://itdependspod.com/episodes/becoming-a-business-leader/ The Google SRE book, free online at https://sre.google/books/ Listener Survey: https://forms.gle/JVeKHsHJKhEM3dvK6 The book: It Depends: Writing on Technology 2012-2022 (https://itdependsbook.net) Your host: Kevin Goldsmith (https://kevingoldsmith.com) The theme song artist: Answers to April

A developer builds a working replacement for an internal platform over a weekend using AI tools, then asks leadership why they can't ship it on Tuesday. The code is clean, the architecture is real, and the team is confused about the holdup. Kevin Goldsmith explores what happens when AI-powered development speed outpaces the organizational guardrails that once kept teams aligned. This episode explores the tension between team autonomy and organizational coherence in an era when building software has become dramatically faster, but deciding what to build has not. Kevin draws on Spotify's aligned autonomy model, his own experience watching founder-speed culture collide with a growing engineering team, and a practical framework he calls the Blast Radius Test: three questions that help teams determine whether a decision is theirs to make or requires a wider conversation. The episode also tackles the blurring line between prototypes and MVPs. When an AI agent produces a prototype with clean architecture, test coverage, and error handling, the output can look identical to a first production implementation. Kevin makes the case that the distinction now has to come from intent and organizational readiness rather than code quality, and shares a story about handing a team his own prototype only to watch them spend weeks trying to ship his throwaway code. This is for engineering leaders trying to figure out how to give teams room to move fast without losing organizational coherence. Henrik Kniberg's Aligned Autonomy video It Depends Episode: Every Decision Makes a Policy It Depends Episode: Fail Safe, Fail Smart... Succeed! Listener Survey: https://forms.gle/JVeKHsHJKhEM3dvK6 The book: It Depends: Writing on Technology 2012-2022 (https://itdependsbook.net) Your host: Kevin Goldsmith (https://kevingoldsmith.com) The theme song artist: Answers to April

The spreadsheet says building is cheaper now, and the spreadsheet isn't wrong. A vendor's annual license on one line, a couple of developers and some tokens on the other. What that comparison leaves out is that building has always had two costs, and AI only made one of them smaller. Kevin has spent most of his career firmly on the buy side, with a mantra he gave his teams for years: don't invent things that already exist. This episode is him revising that position rather than abandoning it, and being specific about how much smaller the revision is than the enthusiasm in the industry suggests. He walks through a four-question test to run before agreeing to build something that already exists, covering blast radius, ownership cost, differentiation, and whether there's an exit path. Along the way: where building genuinely does make sense now, where it still doesn't regardless of how cheap it has become, the license cost trap that catches CFOs, why supply chain risk has started cutting in both directions, and the version of this decision that shows up in headcount instead of software. Two stories anchor it: an auth decision that gave back every dollar it saved and came within about a month of being much worse, and a build made deliberately with an exit already planned. Listener Survey: https://forms.gle/JVeKHsHJKhEM3dvK6 The book: It Depends: Writing on Technology 2012-2022 (https://itdependsbook.net) Your host: Kevin Goldsmith (https://kevingoldsmith.com)

Chapter 3 from It Depends: Writing on Technology Leadership 2012-2022 In this episode, we discuss my experience building an exercise to better understand what is important to me in a role or company, and how I used the results to prepare for job interviews. I also discuss some of the mistakes people make when interviewing for roles during a downturn, particularly in how they present themselves. If you're currently looking for a job, or think you will in the future (we all will), this episode will be helpful. Some links: It Depends book information: https://itdependsbook.net Newsletter: https://kevingoldsmith.substack.com Questions/contact (I'll answer them in the podcast or newsletter): [email protected] Living in an Employer’s Market as a Developer article: https://blog.kevingoldsmith.com/2020/03/31/living-in-an-employers-market-as-a-developer/

This chapter discusses what you can do as a manager to protect your team from layoffs: Ensure the team is aligned with company goals. Raising the visibility of the good work of the individuals on the team. Managing performance. All of these things will help when someone is deciding whether anyone on your team should be laid off. This episode provides more information for individuals and explains that managers are judged more by their team's work and less by their own contributions. Links: The book: It Depends: Writing on Technology 2012-2022 Your host: Kevin Goldsmith The newsletter My talk from Leading Eng 2024: The Director to CTO Path  

Teams create their own languages organically, which is essential to team bonding and belonging. Understand this and use it to improve how your team works together and make onboarding new members easier. Links: The newsletter with the text of the chapter: https://kevingoldsmith.substack.com/p/building-a-vernacular-with-your-engineering "It Depends: Writing on Technology Leadership 2012-2022" https://itdependsbook.net It Depends newsletter: https://kevingoldsmith.substack.com Questions/comments: [email protected]