From Coordination Chaos to Mission Focus: The Waypoints Framework

Major software acquisition programs face pressure to meet delivery milestones. Far too often, by the time program leaders turn their attention to sustainment, the window for strategic decision making has closed and the program is locked into costlier and riskier paths. Drawing on lessons from a recent technical engagement, the latest podcast episode from the Carnegie Mellon University Software Engineering Institute (SEI) explores how software sustainment programs could benefit from an approach that shifts the emphasis from depot activation to the continuous evolution and management of technical baselines. Eileen Wrubel, technical director for smart software acquisition at the SEI, is joined by Brigid O'Hearn, the SEI's Department of War software modernization policy lead, and Kevin Dooley, a senior Agile transformation leader, to discuss this latest technical engagement as well as three critical questions that help avoid the sustainment trap.   

AI components, whether traditional machine learning or generative AI, are embedded in many present-day systems. However, despite advances in automation, infrastructure, and tooling for developing AI components, many do not leave the prototype stage or reach production because they fail to meet overall system quality expectations. In our latest podcast from the Carnegie Mellon University Software Engineering Institute, Rachel Brower-Sinning and Robert Edman, both machine learning scientists in the SEI's Tactical Edge and AI-Enabled Systems Initiative, sit down with Grace Lewis, the initiative's lead and an SEI principal researcher, to discuss a quality model for machine learning components to support their proper testing and evaluation. The research team that developed the model also included Alex Derr, Sebastián Echeverría, and Ipek Ozkaya from the SEI, and, from Carnegie Mellon University, Kate Maffey. Colin Beaudoin, a PhD intern at the SEI in the summer of 2025 who is now an assistant professor at Fairfield University, also worked on the model.

The recent explosion in large language model (LLM) technology has highlighted the challenges of using public generative artificial intelligence tools in classified environments, especially for software analysis. Currently, software analysis falls on the shoulders of static analysis tools and manual code review, which tend to provide limited technical depth and are often time-consuming in practice. In our latest podcast from the Carnegie Mellon University Software Engineering Institute (SEI), Ryan Karl, an SEI embedded engineer, and John Robert, deputy director of the SEI's Software Solutions Division, discuss their work on using LLMs in unclassified environments to rapidly develop tools that accelerate software analysis in classified environments with improved accuracy for certain software analysis tasks. 

Software-defined warfare is today's reality for national security, shifting the emphasis in military operations from hardware to software. In the latest podcast from the Carnegie Mellon University Software Engineering Institute, SEI director Paul Nielsen recently sat down with Matthew Butkovic, technical director of Risk and Resilience in the SEI's CERT Division, to discuss the evolution of software-defined warfare and the ways in which software engineering practices can meaningfully address the challenges of implementing software on the battlefield.  

In aviation, waypoints guide pilots through complex flight plans, providing some structure but maintaining flexibility. Kevin Dooley, a senior Agile transformation leader at the SEI, adopted this concept to solve one of defense acquisition's most persistent challenges: synchronizing dozens of interdependent teams without drowning in administrative noise and overhead. In the latest podcast from the Carnegie Mellon University Software Engineering Institute (SEI), Dooley, who co-developed the Waypoints Framework with Air Force Major Adam Satterfield, sits down with Eileen Wrubel, SEI technical director for Smart Software Acquisition to discuss Waypoints and how it can help teams visualize their work and own processes so they can start collaborating.  

Experimentation and validation of LLM performance is critical when building LLM-driven systems that must reliably deliver a service, from customer service chat bots to intelligence analysis tools. To help teams meet the need for rigorous evaluation methods, a research team in the SEI's AI Division led by Violet Turri has developed the Expanding Large Language Model Metrics (ELM) library, which is built on best practices for LLM evaluation and benchmarking. In the latest episode from the Carnegie Mellon University Software Engineering Institute Podcast Series, Turri sits down with Katie Robinson, a design researcher also in the SEI's AI division, to discuss the library, which turns evaluation from an ad-hoc process into a repeatable, extensible framework.

Data poisoning—where adversaries tamper with training data to corrupt model behavior—poses significant risks as AI adoption expands across critical sectors. Organizations without mechanisms in place to detect or prevent data poisoning are open to an avenue of attack that, once exploited, is difficult to remediate. Machine unlearning and model retraining are not always viable or effective solutions. In today's operational climate, where threat actors look to influence models and degrade the trust of users through incorrect behaviors, preventing data poisoning is more important than ever.  In this episode of the SEI Podcast Series, Julie Lawler and James Cunningham—AI security researchers at Carnegie Mellon University's Software Engineering Institute—discuss the growing threat of data poisoning in AI systems and highlight emerging mitigation strategies, including chain-of-custody controls.  

As recently as December 2025, the Carnegie Mellon University Software Engineering Institute (SEI's) CERT Coordination Center (CERT/CC) documented a UEFI-related vulnerability in certain motherboard models, illustrating that early-boot firmware behavior continues to present security challenges despite requiring local physical access to exploit. While CERT/CC reported seven UEFI vulnerabilities in 2025, that number remains small compared to reported vulnerabilities in other software. However, the consequences of a potential UEFI attack are often more serious given the extremely high privileges UEFI firmware possesses. In our latest SEI Podcast, Vijay Sarvepalli, a senior information security architect specializing in vulnerability and threat analysis in CERT, sits down with Michael Winter, deputy technical director of threat analysis in CERT, to discuss research and mitigation of UEFI vulnerabilities and discuss a new tool, the CERT UEFI parser, an open source tool that uses program analysis to reveal the architecture of UEFI software, and explore this veiled source of vulnerabilities. 

In February 2026, Paul Nielsen announced that he will transition out of his role as director and chief executive officer of the Software Engineering Institute (SEI) at Carnegie Mellon University. During Nielsen's tenure, the SEI has marked major institutional milestones that underscore its enduring role in strengthening the security, resilience, and reliability of the nation's software- and AI-intensive systems. The institute recently celebrated 40 years of innovation and saw its contract renewed, which paved the way for CMU to operate the SEI for another five years. In our latest SEI podcast, Nielsen recently sat down with Matthew Butkovic, technical director of Risk and Resilience in the SEI's CERT Division, to discuss his legacy at the SEI, the impact of mentors, and the importance of encouraging scientists and engineers to do their best work.

Cybersecurity staffing shortages are a major concern in the government given the increasingly sophisticated cyber attacks on the nation's critical infrastructure. In the FY2023 National Defense Authorization Act (NDAA), Congress tasked the Pentagon with finding flexible options to address cyber staffing needs. The Pentagon commissioned the SEI to conduct an independent study to assess the feasibility and advisability of creating a civilian cybersecurity reserve (CCR) that could harness cyber expertise from the private sector to mobilize a mission-ready workforce capable of operating in contested environments. In our latest podcast from the Carnegie Mellon University Software Engineering Institute (SEI), the lead authors on the report, Marie Baker, a technical manager in the SEI's CERT Division, and Chris May, technical director of the CERT Cyber Mission Readiness directorate, sit down with Mike Winter, deputy technical director of threat analysis, to discuss their findings.