The problem of algorithmic bias in AI-based military decision support systems

In the debate on how artificial intelligence (AI) will impact military strategy and decision-making, a key question is who makes better decisions — humans or machines? Advocates for a better leveraging of artificial intelligence point to heuristics and human error, arguing that new technologies can reduce civilian suffering through more precise targeting and greater legal compliance. The counter argument is that AI-enabled decision-making can be as bad, if not worse, than that done by humans, and that the scope for mistakes creates disproportionate risks. What these debates overlook is that it may not be possible for machines to replicate all dimensions of human decision-making. Moreover, we may not want them to. In this post, Erica Harper, Head of Research and Policy at the Geneva Academy of International Humanitarian Law and Human Rights, sets out the possible implications of AI-enabled military decision-making as this relates to the initiation of war, the waging of conflict, and peacebuilding. She highlights that while such use of AI may create positive externalities — including in terms of prevention and harm mitigation — the risks are profound. These include the potential for a new era of opportunistic warfare, a mainstreaming of violence desensitization and missed opportunities for peace. Such potential needs to be assessed in terms of the current state of multilateral fragility, and factored into AI policy-making at the regional and international levels.

The military decision-making process is facing a challenge by the increasing number of interconnected sensors capturing information on the battlefield. The abundance of information offers advantages for operational planning – if it can be processed and acted upon rapidly. This is where AI-assisted decision-support systems (DSS) enter the picture. They are meant to empower military commanders to make faster and more informed decisions, thus accelerating and improving the decision-making process. Although they are just meant to assist – and not replace – human decision-makers, they pose several ethical challenges which need to be addressed. In this post, Matthias Klaus, who has a background in AI ethics, risk analysis and international security studies, explores the ethical challenges associated with a military AI application often overshadowed by the largely dominating concern about autonomous weapon systems (AWS). He highlights a number of ethical challenges associated specifically with DSS, which are often portrayed as bringing more objectivity, effectivity and efficiency to military decision-making. However, they could foster forms of bias, infringe upon human autonomy and dignity, and effectively undermine military moral responsibility by resulting in peer pressure and deskilling.

The Fourth Geneva Convention was the first humanitarian law convention dedicated to protections for civilians during armed conflict. Amongst its numerous protective rules, it also provides the main rules of international humanitarian law (IHL) governing the exceptional practice of internment of protected persons – detention of such persons for security reasons during international armed conflict. In this post, and in commemoration of the 75th anniversary of the Geneva Conventions this year, Group Captain Tim Wood, Provost Marshal of the New Zealand Defence Force, shares his views and practical insights with regards to procedures for internment review of civilians. Drawing on operational experience, he considers some of the characteristics of review bodies which are essential for them to properly fulfil their role.

When the very first Geneva Convention was adopted in 1864, it was the culmination of several interwoven humanitarian projects of the ICRC’s principal founder, Henry Dunant. One of those ambitions was the conception, standardization, and integration into what would become known as international humanitarian law (IHL) of the distinctive emblem of the Convention. Designed to signal the specific protections IHL accords to the medical services and certain humanitarian operations, the emblem – today the red cross, red crescent, and red crystal – is displayed on different persons and objects in the physical world, including on buildings, transports, units, equipment, and personnel that are accorded these protections. Over its 160-year history, the distinctive emblem has saved countless lives. Today, the ICRC is again engaged in a project to conceive, standardize, and integrate into IHL a means to identify those very same specific protections, but in a way the drafters of the original 1864 Geneva Convention could not have imagined: a digital emblem specifically designed to identify the digital assets of the medical services and certain humanitarian operations. In this post, building on previous work on this topic, ICRC Legal Adviser Samit D’Cunha summarizes some of the key milestones of the history and development of the distinctive emblem and explores how these milestones serve as a lodestone – or compass – for the Digital Emblem Project’s path forward.

During armed conflict and other situations of violence, timely access to reliable information can save lives. Affected people need to know where danger and risks come from, how and where they can find assistance, and how to protect themselves and access needed services. At the same time, the information dimensions of conflict have become part of the digital frontlines, where harmful information can spread at greater scale, speed, and reach than ever before. The information space can be riddled with narratives that distort facts that are essential for people to make decisions regarding shelter or their security, that undermine humanitarian operations, or that influence people’s behavior, fueling polarization and hate speech or triggering or inciting violence against civilian populations. The International Committee of the Red Cross (ICRC) is concerned that the spread of misleading or hateful narratives may undermine the protection and safety of people affected by armed conflict and other situations of violence. The ICRC focuses on the potential for harmful effects resulting from the distortion of information or the absence of reliable information. In this post, ICRC Digital Risks Adviser Joelle Rizk presents four risks associated with the spread of harmful information in situations of armed conflict and elaborates on the ICRC approach that focuses on addressing its harmful effects on people.

Over the past decade, discussions surrounding artificial intelligence (AI) in the military domain have largely focused on autonomous weapon systems. This is partially due to the ongoing debates of the Group of Governmental Experts on Lethal Autonomous Weapons Systems of the Convention on Certain Conventional Weapons. While autonomous weapon systems are indeed a pressing concern, the critical reality is that AI is hastily deployed to gather intelligence and, even more worrisome, to support militaries to select and engage targets. As AI-based decision support systems (AI DSS) are increasingly used in contemporary battlefields, Jimena Sofía Viveros Álvarez, member of the United Nations Secretary General’s High-Level Advisory Body on AI, REAIM Commissioner and OECD.AI Expert, advocates against the reliance on these technologies in supporting the target identification, selection and engagement cycle as their risks and inefficacies are a permanent fact which cannot be ignored, for they actually risk exacerbating civilian suffering.

Algorithmic bias has long been recognized as a key problem affecting decision-making processes that integrate artificial intelligence (AI) technologies. The increased use of AI in making military decisions relevant to the use of force has sustained such questions about biases in these technologies and in how human users programme with and rely on data based on hierarchized socio-cultural norms, knowledges, and modes of attention. In this post, Dr Ingvild Bode, Professor at the Center for War Studies, University of Southern Denmark, and Ishmael Bhila, PhD researcher at the “Meaningful Human Control: Between Regulation and Reflexion” project, Paderborn University, unpack the problem of algorithmic bias with reference to AI-based decision support systems (AI DSS). They examine three categories of algorithmic bias – preexisting bias, technical bias, and emergent bias – across four lifecycle stages of an AI DSS, concluding that stakeholders in the ongoing discussion about AI in the military domain should consider the impact of algorithmic bias on AI DSS more seriously.

The desire to develop technological solutions to help militaries in their decision-making processes is not new. However, more recently, we have witnessed militaries incorporating increasingly complex forms of artificial intelligence-based decision support systems (AI DSS) in their decision-making process, including decisions on the use of force. The novelty of this development is that the process by which these AI DSS function challenges the human’s ability to exercise judgement in military decision-making processes. This potential erosion of human judgement raises several legal, humanitarian and ethical challenges and risks, especially in relation to military decisions that have a significant impact on people’s lives, their dignity, and their communities. It is in light of this development that we must urgently and in earnest discuss how these systems are used and their impact on people affected by armed conflict. With this post, Wen Zhou, Legal Adviser with the International Committee of the Red Cross (ICRC), and Anna Rosalie Greipl, Researcher at the Geneva Academy of International Humanitarian Law and Human Rights, launch a new series on artificial intelligence (AI) in military decision-making. To start the discussion, they outline some of the challenges and risks, as well as the potential, that pertain to the use of AI DSS in preserving human judgement in legal determinations on the use of force. They also propose some measures and constraints regarding the design and use of AI DSS in these decision-making processes that can inform current and future debates on military AI governance, in order to ensure compliance with international humanitarian law (IHL) and support mitigating the risk of harm to people affected by those decisions.

In cities from Gaza to those in Sudan and Ukraine, childhoods are irrevocably changed by urban warfare. Yet despite the number of children affected and the increasingly urbanized nature of conflict, the detail of the child-specific nature of the harm caused remains poorly understood by practitioners and decision-makers. To address this gap, in 2023 the ICRC published a new report – Childhood in Rubble: The Humanitarian Consequences of Urban Warfare for Children – drawing from existing literature, 52 interviews with experts, and the organization’s firsthand experience. In this post, three of the report’s contributors set out eight overlooked ways that children are affected by urban warfare and outline a set of legal, policy and operational recommendations that states, non-state armed groups and humanitarians could implement to elevate the protection of children from media rallying cry to political priority.

Next week marks the 75th anniversary of the four Geneva Conventions of 1949. Against a backdrop of over 120 armed conflicts worldwide, this should prompt us not to celebrate, but to reflect: how were these now universally accepted humanitarian norms drafted, and are they still fit for purpose today? In this post, ICRC Legal Adviser Ellen Policinski examines how the Geneva Conventions are interpreted and applied today, providing concrete examples from the ICRC’s updated Commentaries. She goes on to investigate the recurring critique that international humanitarian law (IHL) is somehow outdated, examining who benefits from the narrative that the Geneva Conventions and IHL more generally are not appropriate legal tools to govern armed conflicts today.