The Moral Machine Experiment

A St Cross Special Ethics Seminar, recorded at St Cross College, Oxford in February 2024. Demandingness objections have become a stock argument in ethics claiming that single moral demands or entire moral theories must be given up or altered if they ask too much of agents. But can we clearly distinguish an acceptable level of demandingness from one that is too high? I argue that demandingness objections inevitably fail to make that distinction without borderline cases because they are sorites-susceptible. First, I show that the heap paradox applies to demandingness objections and the expression “overdemanding” because two conditions are met. There is an ordering of values on one dimension decisive for the expression’s application: the cost to the agent. Also, the expression “overdemanding” is tolerant, because the difference between two neighbouring levels of demandingness is so small that it does not allow us to say that this is the difference between an acceptable level of demandingness and critical overdemandingness. Second, I discuss attempts to overcome or bypass the vagueness of demandingness objections. I will argue that these strategies are not very promising and that we should rather embrace the vagueness.

Professor Predrag uses a comparison of money and morality to explore the mutual relationship between morality and personality. To clarify the tension that exists between morality and personality, Cicovacki opens his talk by comparing the development of the money economy and morality. Money and morality play a similar function with respect to social interactions: they make most diverse things commensurable and impose the rules that should have universal validity, regardless of to whom they apply. Personality is characterized by the uniqueness of each individual, as well as by a need for continuous development. To close an unhealthy gap between morality and personality, morality should be conceived not on the model of the money economy, but by becoming more sensitive to who we are and in what kind of situations we find ourselves. Cicovacki argues that we should favor a maximalist rather than a minimalist conception of morality: the one that urges us to become as good human beings as we can, rather than to focus merely on enabling acceptable social intercourse. The questions that such a conception of morality should ask are: 1. What is the moral cost of being who you are? and 2. What is the moral cost of not being who you are?

Professor Mark Coeckelbergh considers whether AI poses a risk for democracy n this St Cross Special Ethics Seminar Cases such as Cambridge Analytica or the use of AI by the Chinese government suggest that the use of artificial intelligence (AI) creates some risks for democracy. This paper analyzes these risks by using the concept of epistemic agency and argues that the use of AI risks to influence the formation and the revision of beliefs in at least three ways: the direct, intended manipulation of beliefs, the type of knowledge offered, and the creation and maintenance of epistemic bubbles. It then suggests some implications for research and policy.

In the first St Cross Special Ethics Seminar of 2023, Dr Larry Lengbeyer explores 'shallow cognizing' as a form of self-control Shallow cognizing is a familiar but overlooked practice of self-control, typically initiated without conscious intention, that enables us to short-circuit potential upwellings of emotion and desire in ourselves. We will consider the range of contexts in which the practice is manifest, speculate about its roots in the compartmentalized structure of our cognitive systems, ponder its benefits and costs (its uses and misuses), and contemplate its relation to virtue. We will then continue in this exploratory vein by asking whether taking account of this neglected phenomenon might improve our understanding of issues in practical ethics, such as duties of doctors to obtain informed consent from patients, and how to balance free expression with proper care for others' sensibilities, in the classroom and perhaps elsewhere.

In this St Cross Special Ethics Seminar, Dr Edmond Awad discusses his project, the Moral Machine, an internet-based game exploring the ethical dilemmas faced by driverless cars. I describe the Moral Machine, an internet-based serious game exploring the many-dimensional ethical dilemmas faced by autonomous vehicles. The game enabled us to gather 40 million decisions from 3 million people in 200 countries/territories. I report the various preferences estimated from this data, and document interpersonal differences in the strength of these preferences. I also report cross-cultural ethical variation and uncover major clusters of countries exhibiting substantial differences along key moral preferences. These differences correlate with modern institutions, but also with deep cultural traits. I discuss how these three layers of preferences can help progress toward global, harmonious, and socially acceptable principles for machine ethics. Finally, I describe other follow up work that build on this project.

In this St Cross Special Ethics Seminar, Professor Stephen Clarke the role of hope in patients undergoing major healthcare procedures, and how it relates to decision-making in situations of risk and uncertainty. It is widely supposed that it is important to imbue patients undergoing medical procedures with a sense of hope. But why is hope so important in healthcare, if indeed it is? We examine the answers that are currently on offer and show that none do enough to properly explain the importance that is often attributed to hope in healthcare. We then identify a hitherto unrecognised reason for supposing that it is important to imbue patients undergoing significant medical procedures with hope, which draws on prospect theory, Kahneman and Tversky’s hugely influential descriptive theory about decision making in situations of risk and uncertainty. We also consider some concerns about patient consent and the potential manipulation of patients, that are raised by our account. We then consider some complications for the account raised by religious sources of hope, which are commonly drawn on by patients undergoing major healthcare procedures.

In this St Cross Special Ethics Seminar, Dr Brian Earp argues that all medically unnecessary genital cutting of non-consenting persons should be opposed on moral and legal grounds. Defenders of male circumcision increasingly argue that female ‘circumcision’ (ritual cutting of the clitoral hood or labia) should be legally allowed in Western liberal democracies even when non-consensual. In a recent article, Richard Shweder (2021) gives perhaps the most persuasive articulation of this argument to have so far appeared in the literature. In my own work, I argue that no person should be subjected to medically unnecessary genital cutting of any kind without their own informed consent, regardless of the sex characteristics with which they were born or the religious or cultural background of their parents. Professor Shweder and I agree that Western law and policy on child genital cutting is currently beset with cultural, religious, and sex-based double-standards. We disagree about what should be done about this. In this talk, I argue that ‘legalizing’ childhood female genital cutting so as to bring it into line with current treatment of childhood male genital cutting is not an acceptable solution to these problems. Instead, all medically unnecessary genital cutting of non-consenting persons should be opposed equally on moral and legal grounds and discouraged by all appropriate means.

In this St Cross Special Ethics Seminar, Dr Sarah Chan outlines some risks arising from the deliberate infection of human participants to infectious agents for research purposes In this St Cross Special Ethics Seminar, Dr Sarah Chan explores three key areas of risk in ‘challenge trials’ – the deliberate infection of human participants to infectious agents as a tool for vaccine development and improving our knowledge of disease biology. Dr Chan explores a) whether some forms of challenge trials cannot be ethically justified; b) why stratifying populations for vaccine allocation by risk profile can result in unjust risk distribution; and c) how comparing these cases and the evaluation of relative risk reveals flaws in approach to pandemic public health.

Marcello Ienca discusses moral and legal issues surrounding the decoding – ‘mind reading’ - of brain activity In the 1990s, following rapid advances in the use of technologies such as functional magnetic resonance imaging (fMRI), an ethical debate arose around the concept of 'mind reading': the possibility of decoding a person's mental states (including their conscious experience) based on quantitative measurements of their brain activity. This debate concerned the moral and legal status of information about mental states (mental information) and the definition of normative principles to justify the collection and processing of such information. However, the poor replicability of fMRI-based studies combined with conceptual clarifications within the philosophy of mind (such as Dennett's vehicle-content distinction) showed that this debate rested on weak empirical and conceptual grounds. As a result, the interest of the bioethics/neuroethics community waned. A couple of decades later, however, the wide availability of brain data outside the clinical setting combined with the use of artificial intelligence models to process and decode such data (through a process known as 'reverse inference') make the ethical debate on mind-reading topical again, albeit on a different conceptual ground. In addition, AI approaches such as affective computing and natural language processing have shown the possibility of inferring a person's mental states also from non-neural data (e.g., social media). This presentation will discuss the moral and legal status of mental information and the conditions for legitimate access to and alteration of such information. In particular, it will examine the concepts of ‘mental privacy’ and ‘cognitive liberty’ and defend the thesis that every person should enjoy a right to mental self-determination. This includes both the negative freedom from coercive or otherwise non-voluntary access to one's mental sphere and the positive freedom of the individual to modify their mental states and processes (e.g. through cognitive or affective enhancement).

Professor Gopal Sreenivasan delivers a New St Cross Special Ethics Seminar on the topic of Informed Consent. This talk develops a novel argument to show that prospective research subjects can validly consent to participate in a study without understanding (most of) the content of the required disclosure. Its point of departure is the right subjects standardly have to waive (most of) the investigator’s duty to disclose. Things get worse for autonomy based defences of informed consent because this right to waive is very well grounded in an individual’s autonomy.