364: AI is Changing Who Builds Software

In this episode, Dave speaks with Jean-Baptiste Tristan, senior principal scientist at AWS, about how automated reasoning can help verify software correctness and enforce governance guardrails for autonomous AI agents.Jean-Baptiste described automated reasoning as computer science and mathematical logic implemented through algorithms and software. Unlike data-driven AI that generalizes through induction or transduction, it makes inferences through deduction using axioms and rules.Automated provers can attempt to prove a formal specification or find counterexamples; interactive theorem provers mechanize reasoning. A sorting algorithm, for example, can be specified to preserve list values and order them correctly, then checked against that specification.

Dave talks with Justin Collier, Vice President of Product Management at SmartBear, about the need for AI testing to mature, to keep pace with AI-generated code. About 69% of surveyed U.S. software experts said AI writes or accelerates at least 41% of their code, while only about 46% validate specifications before AI generates code. Code coverage remains important, but it is not sufficient by itself. Teams should identify critical tests early and target tests to changed code and release risks, since running every regression test may not keep pace with frequent changes.AI-generated tests need to verify software intent and should not simply let AI grade its own work. Teams need traceable records of test authorship and execution, release decisions, changes, and the rationale for shipping or holding software.

In this episode, Dave speaks with Christian Bremeau, founder and CEO of Meta Integration as a nearly 30-year-old metadata technology provider whose components support lineage and metadata harvesting across data integration, catalog, modeling, and BI products.Version 12 is now generally available after nearly two years of development and about six months of pre-release deployment. It includes multiple MCP servers, AI-assisted metadata search and question-answering, and the MetaKarta product brand.Semantic HubThe Semantic Hub reuses existing BI metrics, KPIs, lineage, and semantic models to provide context for AI rather than requiring organizations to start from scratch.It compiles governed definitions into database semantic or metric views and BI tools including Tableau and Power BI, enabling AI agents and BI users to receive the same answers.Christian emphasized deterministic SQL generation from compiled semantic models instead of relying on AI to generate SQL at runtime, reducing AI cost and hallucination risk.

Dave discusses npm version 12’s decision to disable lifecycle scripts by default to improve security while potentially shifting malicious-package execution from installation time to runtime. Joining him is Darren Meyer , security leader at  Checkmarx. Darren explained that lifecycle scripts, including post-install scripts, have been a common way for attackers to execute malicious code in development environments. With scripts disabled by default, attackers may move deployment and infection code into module-loading behavior, such as execution during require or import. The change is a useful barrier, but it does not eliminate the underlying threat.

Dave discusses the evolution of quality assurance in an AI-driven development life cycle with Wei Wei Wu, CEO of Momentic.  Wei Wei argues that traditional test scripts like Selenium and Playwright are becoming obsolete as AI agents increasingly write code. Instead, he proposes a future where independent agents act as verifiers to close the feedback loop autonomously. Wei Wei suggests the role of QA is shifting toward "context engineering," where humans provide business logic and edge cases to AI.

Dave welcomes Cassie Shum, RelationalAI adviser Cassie Shun, former VP ecosystem, product engineering there and formerly at Thoughtworks, to discuss the evolving role of developers in the era of AI. Shum highlights that while AI handles commoditized coding, developers are still essential for design patterns, testing and validation. She identifies two types of organizations: those with weak foundations and those with strong ones. Shum emphasizes the importance of documenting architectural intent and onboarding developers quickly using AI. She suggests that junior developers should focus on understanding design principles and working with AI, rather than just coding.

Dave and Sreenivasan Rajagopal discuss the rising costs of AI in organizations, emphasizing the role of the Tokenomics Foundation, a Linux foundation formed by industry leaders, including Broadcom. Sreenivasan explains that AI spend is often exceeding budgets quickly, and the foundation aims to measure and optimize AI investments. They highlight Broadcom's Value Ops AI Tokenomics, which integrates AI costs into infrastructure management, focusing on governance, security, and value realization. The discussion also touches on the importance of understanding AI architecture costs and the need for effective AI governance to ensure productivity and value.

Dave Rubinstein interviews Gavriel Cohen, creator of NanoClaw, about the state of the project, an open-source multi-agent orchestration tool released in February. NanoClaw quickly gained popularity, with over 30,000 stars and 13,000 forks on GitHub, and a community of 5,000 in Discord. Cohen emphasizes the project's focus on agent isolation, credential management, and policy enforcement to prevent agents from going rogue. He discusses the importance of continuous testing and end-to-end frameworks to adapt to rapidly changing AI models. Cohen also highlights the need for robust security measures, including dedicated AI security agents, to protect against sophisticated attacks and zero-day vulnerabilities.

Dave Rubinstein discusses with Tim Qi, lead data analyst at Linear, how AI is changing who builds software. Linear's study found that AI is accelerating team velocity and changing behavior on platforms. Companies are adopting AI in two main ways: top-down, tightly controlled rollouts, and bottom-up, allowing experimentation. Training varies, often evolving with use cases. AI adoption is similar across company sizes. The study also revealed that AI tools are making previously cumbersome tasks more manageable, leading to higher quality outputs. The industry is still figuring out how to manage the shadow AI issue.

Dave Rubinstein and Matt Whitbourne discuss the evolution and potential of mainframe modernization, particularly with the integration of AI. They highlight the long-standing challenge of maintaining legacy systems, such as COBOL, and the role of AI in enhancing innovation and efficiency. Whitbourne emphasizes the importance of understanding business logic before modernizing and the potential for AI to assist in this process. He also notes the benefits of AI in improving code serviceability, risk management, and compliance. The conversation underscores the need for a strategic approach to mainframe modernization, focusing on business goals and the unique characteristics of mainframe workloads.