Episode 83: When Should Machines Decide?

What characteristics make a person trustworthy? Under what circumstances would a person delegate life or death decisions to artificial intelligence (AI)? Does it matter that AI systems reflect trustworthy humans’ decision-making preferences, morals, and ethics? If so, what characteristics are most important?These are some of the fundamental questions DARPA researchers are exploring for the In the Moment (ITM) program, which aims to support the development of algorithms that are trusted to independently make decisions in difficult domains, particularly in significant trauma events such as battlefield triage.DARPA’s research has identified the need for fundamentally different approaches to advance AI technology to a place where we’re willing to trust it and not be foolish to do so. Continuing themes from our mini-series on ELSI – ethical, legal, and societal implications of new technologies and capabilities – we meet with DARPA’s ITM program manager, Dr. Patrick Shafto, and the ITM performers and ELSI advisors, who break down how they’re tackling the fundamental question of alignment in the context of human decision-makers and autonomous decision-making tools.In case you missed them, check out our previous ELSI series episodes at the following links:Episode 79: Integrating ELSIEpisode 78: Introducing ELSIOur special thanks to the following ITM performers and advisors for their contributions to this episode (in order of their appearance):·        Alice Leung, RTX BBN·        Joseph Cohn, SoarTech·        Matthew Molineaux, Parallax Advanced Research·        Arslan Basharat, Kitware Inc.·        Jennifer McVay, CACI·        Dave Cotting, Institute for Defense Analyses (IDA)·        Sarah Daly, IDA·        Lauren Diaz, University of Maryland Applied Research Laboratory for Intelligence and Security (ARLIS)·        Ellie Tyler, ARLIS

Accomplishing DARPA’s mission of creating and preventing strategic surprise is as much a business challenge as it is a technology challenge.In this episode, team members from DARPA’s Contracts Management Office – Office Director Effie Fragogiannis and Deputy Director Catherine Stevens, along with Senior Advisor Scott Ulrey – explore what it takes to innovate contracting processes and mechanisms to enable the development of breakthrough technologies at the speed of relevance.From DARPA’s pioneering work with Other Transactions, to fast-pitch proposals, to the exploration of previously unrealized authorities, hear how the agency is breaking down the barriers of government contracting, providing companies a clearer path to the national security mission.Links:Acquisition Innovation Collaborative Disruption at DoD – Kathleen Hicks – American Dynamism SummitAcquisition Innovation: From Other Transactions to Fast-Pitch ProposalsOffice of the Undersecretary of Defense for Acquisition and Sustainment Other Transactions GuideDARPAConnect

In this episode, Dr. Jean-Paul Chretien and Elissa Rupley from our Biological Technologies Office provide an exciting update from the recent DARPA Triage Challenge (DTC) workshop at the Guardian Centers in Perry, GA.  The DARPA Triage Challenge, or DTC, aims to drive breakthrough innovations in identification of “signatures” of injury that will help medical responders perform scalable, timely, and accurate triage. Of particular interest are mass casualty incidents, in both civilian and military settings, when medical resources are limited relative to the need. We also hear from Alix Donnelly, from the U.S. Army's Telemedicine & Advanced Technology Research Center (TATRC) and some of the participating DTC Team Members on their experience thus far in the competition. There are still opportunities to get involved – listen to learn more!Links:DARPA Triage ChallengeDr. Jean-Paul Chretien bioDTC Workshop 1 highlight video DTC YouTube Playlist

In this episode, Dr. Vishnu Sundaresan from our Defense Sciences Office highlights several technology programs designed to precisely control chemical processes to enable distributed, small-batch manufacturing of chemical products while retaining efficiencies of large-scale industrial production. Colloquially calling this portfolio “decentralized chemistry for everything,” the concept aims to shift the paradigm from a few centralized production facilities producing medicines in large batches and requiring a costly purification process, to direct manufacturing of pure pharmaceuticals via desktop printer-sized machines that would create — at the push of a button — doses of a variety of medicines whenever and wherever needed. Such a revolutionary capability — if successful — would circumvent brittle international chemical supply chains and would serve military members deployed in remote locations as well as benefit rural civilian communities.Sundaresan describes programs aiming to achieve elements of this vision: Spin-COntrolled chemical Process Engineering (SCOPE), Recycling at the Point of Disposal (RPOD), and Establishing Qualification Processes for Agile Pharmaceutical Manufacturing (EQUIP-A-Pharma).Listen to Sundaresan describe his journey to becoming a DARPA program manager, the fascinating world of controlling electron spins, and the ethical, legal, and societal challenges of preparing the market for such revolutionary tech.

In this episode, we’ll be taking a deeper dive into ELSI – ethical, legal, and societal implications of new technologies and capabilities – and specific examples of how DARPA programs have incorporated those considerations into their structure. We’re highlighting three examples of how DARPA integrated ELSI throughout the program lifecycle via the counsel of experts from the medical, scientific, legal, and ethics communities to assist program managers and performers in identifying and mitigating any potential issues. The first program, out of our Biological Technologies Office, is Safe Genes, which supported force protection and military health and readiness by developing tools and methodologies to control, counter, and even reverse the effects of genome editing—including gene drives—in biological systems across scales. The second program, Urban Reconnaissance through Supervised Autonomy (URSA) from our Tactical Technology Office (TTO) aimed to enable improved techniques for rapidly discriminating hostile intent and filtering out threats in complex urban environments. And, finally, the current In the Moment program in our Information Innovation Office (I2O) seeks to identify key attributes underlying trusted human decision-making in dynamic settings and computationally representing those attributes, to generate a quantitative alignment framework for a trusted human decision-maker and an algorithm. YouTube: https://www.youtube.com/watch?v=PYSbEnamSDA  iTunes: https://podcasts.apple.com/us/podcast/voices-from-darpa/id1163190520 Show notes and links: DARPA currently is seeking applicants for the 2025 ELSI Visiting Scholar. The deadline to apply is June 3, 2024.Voices from DARPA Episode 78: Introducing ELSI : https://blubrry.com/voices_from_darpa/132452728/episode-78-introducing-elsi/  ELSI History: https://www.ncbi.nlm.nih.gov/books/NBK231976/  [quoted in Roberts, 1989]. Highlighted Programs: In the Moment (ITM): https://www.darpa.mil/program/in-the-moment  Urban Reconnaissance through Supervised Autonomy (URSA): https://www.darpa.mil/program/urban-reconnaissance-through-supervised-autonomy   Safe Genes: https://www.darpa.mil/program/safe-genes   Safe Genes Publications:   Interdisciplinary development of a standardized introduction to gene drives for lay audiences  https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-020-01146-0  A typology of community and stakeholder engagement based on documented examples in the field of novel vector control https://journals.plos.org/plosntds/article?id=10.1371/journal.pntd.0007863#:~:text=The%20typology%20names%20three%20types,and%203)%20engagement%20to%20involve.  Codes https://www.liebertpub.com/doi/full/10.1089/crispr.2020.0096 https://www.science.org/doi/10.1126/science.abd1908 

As a global leader in innovation, DARPA starts an average of 50 new programs each year. These programs span a variety of technical disciplines to develop breakthrough technologies for national security, all of which have the potential to raise ethical, legal, and societal implication – or, ELSI – considerations.Taking time to consider ELSI’s role in a program can contribute to the responsible development of emerging technologies by guiding innovation, maximizing the potential application space, and facilitating dialogue with future end-users, and the public, to ensure diverse perspectives and implications are considered. It can improve research by fostering conversations that identify unknowns, anticipate consequences, and make design decisions to maximize benefits and opportunities and minimize risks and harms.In this episode of Voices from DARPA, we’ll hear from DARPA Director, Dr. Stefanie Tompkins, to explain the agency's perspective on those implications, as well as Dr. Bart Russell, deputy director of the Defense Sciences Office, on what it would mean to incorporate ELSI across the agency more formally. Finally, Dr. Rebecca Crootof, DARPA’s inaugural ELSI Visiting Scholar, will discuss her journey to the agency and her approach to developing a process to ensure that ELSI can inform — and even improve —DARPA programs.That sounds like a lot of responsibility, influence, and potential impact – for some, maybe, too good to be true?Ethical, Legal, and Societal Implications of Emerging Technologies at DARPA DARPA currently is seeking applicants for the 2025 ELSI Visiting Scholar. The deadline to apply is June 3, 2024.Origin of the term ELSI: Three decades of ethical, legal, and social implications research: Looking back to chart a path forwardThe piece references a class from Dr. Oona A. Hathaway

Good ideas can come from anywhere, but what is the best way to find them, or help them find you?  In 2022, DARPA hit the road in pursuit of the answer. Comprising six regional events, DARPA Forward took the agency across the country to engage untapped talent and strengthen the nationwide innovation ecosystem. The event series offered a powerful lesson in breaking down barriers of entry in pursuit of national security breakthroughs.  To sustain this momentum, DARPA launched DARPAConnect, an initiative that aims to further broaden the agency’s reach and foster greater collaboration with underrepresented, diverse, and nontraditional institutions new to the national security space.  In this episode of Voices from DARPA, we’re taking a deep dive on DARPAConnect, talking with several of those involved in the initiative to get a sense of how it all works. We’ll explore its goals, its offerings, and what success looks like at DARPA, home to some of the biggest – and riskiest – bets on U.S. technological innovation.  DARPAConnect website: https://www.darpaconnect.us/home  

In this episode we hear from quantum physicist Dr. Mukund Vengalattore, a program manager in DARPA’s Defense Sciences Office, who oversees a portfolio of fundamental research programs aimed at unlocking new quantum insights and overcoming challenges to enable revolutionary capabilities for defense. These include harnessing atoms and superconducting structures for novel sensing applications (imagine tiny, super-sensitive antennas, infrared detectors or gyroscopes that vastly outperform much larger antennas, IR cameras, and gyroscopes of today); developing better quantum bits (qubits) for quantum computing (including using photons to encode information in novel ways); enabling field-deployable, tactical-grade mobile atomic clocks for our troops; and discovering new quantum materials for applications ranging from quantum computing to biomedical imaging. We’re also joined by Dr. Mikhail Lukin, professor of physics at Harvard University, who led a team on Vengalattore’s Optimization with Noisy Intermediate-Scale Quantum devices (ONISQ) program that made a major quantum breakthrough published in Nature recently. Lukin’s team exploited characteristics of Rydberg neutral atoms tocreate logical qubits and used them to demonstrate the first-ever quantum circuit, a key step to advancing novel quantum computing architectures (Vengalattore provides a primer on the Rydberg atomic state). You’ll also hear about “optical tweezers” – which use laser beams that can be controlled to precisely grab and move around individual qubits without destroying their quantumness — and how they helped enable the breakthrough. To read more about the ONISQ logical qubit breakthrough visit: https://www.darpa.mil/news-events/2023-12-06 Normally, we’d recommend you jump right into the episode, but this time, a primer may be helpful. We suggest starting with our recent Quantum Mechanic episode before you take a deep technical dive to the subatomic level for a fascinating window on the vast frontiers of quantum exploration… and potential applications in the real world. 

We usually think of materials based on our experience in the natural world. For example, something that’s light is usually fragile (like a feather) or something heavy is usually strong (like a brick). But what if we could engineer a material that had completely new characteristics that defied properties found in nature? Engineered materials, also known as metamaterials, allow us to do just that. DARPA Program Manager Dr. Rohith Chandrasekar in DARPA’s Defense Sciences Office has led programs designing metamaterials that revolutionize how light interacts with matter. His programs are enabling new concepts for improving Warfighter effectiveness and health on the battlefield with new optics and materials. In this episode, Dr. Chandrasekar discusses several of these programs including Enhanced Night Vision in Eyeglass Form (ENVision), which has developed metamaterials to replace heavy and bulky binocular-like night-vision goggles lenses with lightweight lenses providing more infrared information and near eyesight field of view, in a form factor like a pair of glasses. He also discusses his Coded Visibility program, which focused on developing novel obscurants (aka smoke) used on the battlefield to provide friendly forces with visibility of the environment, while simultaneously hiding them from detection by an adversary. The catch, however, is that the smoke particles needed to be safe to breathe and potentially even tunable using active sources. Finally, he talks about the Accelerating discovery of Tunable Optical Materials (ATOM) program. This effort seeks to identify new materials whose properties could be rapidly changed to enable different functions. Imagine a massive telephoto camera on the sideline of a sporting event replaced with a planar imaging system that could zoom, or a thin filter that can rapidly collect critical data across infrared bands for spectroscopy, all with no moving parts. Sounds like magic, but it’s not! Enjoy listening to DARPA’s Metamaterial Visionary.

Established in 2006, the Young Faculty Award (YFA) program aims to identify and engage rising academics in early-career research positions - particularly those without prior DARPA funding - and expose them to Department of Defense (DOD) needs and DARPA's mission to create and prevent technological surprise. The YFA program provides high-impact funding toresearchers at U.S. institutions early in their careers to advance innovative research enabling transformative DOD capabilities. The long-term goal of the YFA program is to build a pipeline for the next generation of academic scientists, engineers, and mathematicians who will focus a significant portion of their career on DOD and national security issues.In this episode you'll hear from Dr. Rohith Chandrasekar, who oversees DARPA's YFA program, as well as from DARPA Program Managers Dr. Chris Bettinger and Dr. Sunil Bhave, who reflect on their experience as YFA awardees early in their academic careers and the opportunities it has afforded them.DARPA recently published the 2024 YFA Research Announcement that features almost two dozen new technical topics and an additional open topic covering six thrust areas specific to DARPA's Defense Sciences Office (DSO). To view the full 2024 YFA Research Announcement visit SAM.gov: https://sam.gov/opp/f2bf469a50e7433fa758f0125831754b/view or Grants.gov: https://www.grants.gov/search-results-detail/350899. Executive summaries, which are encouraged, are due by Dec. 13, 2023, 4:00 p.m. ET. Full proposals are due Feb. 22, 2024, 4 p.m. ET.