Base Camp for Mount Ethics (2022)

By Nick Bostrom.Abstract:There may well exist a normative structure, based on the preferences or concordats of a cosmic host, and which has high relevance to the development of A-eye. In particular, we may have both moral and prudential reason to create superintelligence that becomes a good cosmic citizen—that is conforms to cosmic norms and contributes positively to the cosmopolis. An exclusive focus on promoting the welfare of the human species and other terrestrial beings, or an insistence that our own norms must at all cost prevail, may be objectionable and unwise. Such attitudes might be analogized to the selfishness of one who exclusively pursues their own personal interest, or the arrogance of one who acts as if their own convictions entitle them to run roughshod over social norms—though arguably they would be worse, given our present inferior status relative to the membership of the cosmic host. An attitude of humility may be more appropriate.Read the full paper:https://nickbostrom.com/papers/ai-creation-and-the-cosmic-host.pdfMore episodes at:https://radiobostrom.com

Nick Bostrom’s latest book, Deep Utopia: Life and Meaning in a Solved World, will be published on 27th March, 2024. It’s available to pre-order now: https://nickbostrom.com/deep-utopia/ The publisher describes the book as follows: A greyhound catching the mechanical lure—what would he actually do with it? Has he given this any thought? Bostrom’s previous book, Superintelligence: Paths, Dangers, Strategies changed the global conversation on AI and became a New York Times bestseller. It focused on what might happen if AI development goes wrong. But what if things go right? Suppose that we develop superintelligence safely, govern it well, and make good use of the cornucopian wealth and near magical technological powers that this technology can unlock. If this transition to the machine intelligence era goes well, human labor becomes obsolete. We would thus enter a condition of “post-instrumentality”, in which our efforts are not needed for any practical purpose. Furthermore, at technological maturity, human nature becomes entirely malleable. Here we confront a challenge that is not technological but philosophical and spiritual. In such a solved world, what is the point of human existence? What gives meaning to life? What do we do all day? Deep Utopia shines new light on these old questions, and gives us glimpses of a different kind of existence, which might be ours in the future.

By Nick Bostrom, Thomas Douglas & Anders Sandberg.Abstract:In some situations a number of agents each have the ability to undertake an initiative that would have significant effects on the others. Suppose that each of these agents is purely motivated by an altruistic concern for the common good. We show that if each agent acts on her own personal judgment as to whether the initiative should be undertaken, then the initiative will be undertaken more often than is optimal. We suggest that this phenomenon, which we call the unilateralist’s curse, arises in many contexts, including some that are important for public policy. To lift the curse, we propose a principle of conformity, which would discourage unilateralist action. We consider three different models for how this principle could be implemented, and respond to an objection that could be raised against it.Read the full paper:https://nickbostrom.com/papers/unilateralist.pdfMore episodes at:https://radiobostrom.com/ ---Outline:(00:00) Intro(01:20) 1. Introduction(10:02) 2. The Unilateralist's Curse: A Model(11:31) 3. Lifting the Curse(13:54) 3.1. The Collective Deliberation Model(15:21) 3.2. The Meta-rationality Model(18:15) 3.3. The Moral Deference Model(33:24) 4. Discussion(37:53) 5. Concluding Thoughts(41:04) Outro & credits

By Nick Bostrom.Abstract:Positions on the ethics of human enhancement technologies can be (crudely) characterized as ranging from transhumanism to bioconservatism. Transhumanists believe that human enhancement technologies should be made widely available, that individuals should have broad discretion over which of these technologies to apply to themselves, and that parents should normally have the right to choose enhancements for their children-to-be. Bioconservatives (whose ranks include such diverse writers as Leon Kass, Francis Fukuyama, George Annas, Wesley Smith, Jeremy Rifkin, and Bill McKibben) are generally opposed to the use of technology to modify human nature. A central idea in bioconservativism is that human enhancement technologies will undermine our human dignity. To forestall a slide down the slippery slope towards an ultimately debased ‘posthuman’ state, bioconservatives often argue for broad bans on otherwise promising human enhancements. This paper distinguishes two common fears about the posthuman and argues for the importance of a concept of dignity that is inclusive enough to also apply to many possible posthuman beings. Recognizing the possibility of posthuman dignity undercuts an important objection against human enhancement and removes a distortive double standard from our field of moral vision.Read the full paper:https://nickbostrom.com/ethics/dignityMore episodes at:https://radiobostrom.com/ ---Outline:(00:02) Introduction(00:21) Abstract(01:57) Transhumanists vs. bioconservatives(06:42) Two fears about the posthuman(19:44) Is human dignity incompatible with posthuman dignity?(29:03) Why we need posthuman dignity(34:38) Outro & credits

By Nick Bostrom.Abstract:Rarely does philosophy produce empirical predictions. The Doomsday argument is an important exception. From seemingly trivial premises it seeks to show that the risk that humankind will go extinct soon has been systematically underestimated. Nearly everybody's first reaction is that there must be something wrong with such an argument. Yet despite being subjected to intense scrutiny by a growing number of philosophers, no simple flaw in the argument has been identified.Read the full paper:https://anthropic-principle.com/q=anthropic_principle/doomsday_argument/More episodes at:https://radiobostrom.com/

By Nick Bostrom and Carl Shulman. Draft version 1.10.Abstract:AIs with moral status and political rights? We'll need a modus vivendi, and it’s becoming urgent to figure out the parameters for that. This paper makes a load of specific claims that begin to stake out a position.Read the full paper:https://nickbostrom.com/propositions.pdfMore episodes at:https://radiobostrom.com/ ---Outline:(00:00) Introduction(00:36) Disclaimer(01:07) Consciousness and metaphysics(06:48) Respecting AI interests(21:41) Security and stability(32:04) AI-empowered social organization(38:07) Satisfying multiple values(42:23) Mental malleability, persuasion, and lock-in(47:20) Epistemology(53:36) Status of existing AI systems(59:52) Recommendations regarding current practises and AI systems(01:07:08) Impact paths and modes of advocacy(01:11:11) Closing credits

By Nick Bostrom.Draft version 0.9Abstract:New theoretical ideas for a big expedition in metaethics.Read the full paper:https://nickbostrom.com/papers/mountethics.pdfMore episodes at:https://radiobostrom.com/ ---Outline:(00:17) Metametaethics/preamble(02:48) Genealogy(09:41) Metaethics(21:30) Value representors(26:56) Moral motivation(30:02) The weak(33:25) Hedonism(41:38) Hierarchical norm structure and higher morality(55:30) Questions for future research

By Nick Bostrom.Abstract:Transhumanism is a way of thinking about the future that is based on the premise that the human species in its current form does not represent the end of our development but rather a comparatively early phase. We formally define it as follows:(1) The intellectual and cultural movement that affirms the possibility and desirability of fundamentally improving the human condition through applied reason, especially by developing and making widely available technologies to eliminate aging and to greatly enhance human intellectual, physical, and psychological capacities.(2) The study of the ramifications, promises, and potential dangers of technologies that will enable us to overcome fundamental human limitations, and the related study of the ethical matters involved in developing and using such technologies.Transhumanism can be viewed as an extension of humanism, from which it is partially derived. Humanists believe that humans matter, that individuals matter. We might not be perfect, but we can make things better by promoting rational thinking, freedom, tolerance, democracy, and concern for our fellow human beings. Transhumanists agree with this but also emphasize what we have the potential to become. Just as we use rational means to improve the human condition and the external world, we can also use such means to improve ourselves, the human organism. In doing so, we are not limited to traditional humanistic methods, such as education and cultural development. We can also use technological means that will eventually enable us to move beyond what some would think of as “human”.Read the full paper:https://nickbostrom.com/views/transhumanist.pdfMore episodes at:https://radiobostrom.com/ ---Outline:(00:25) 1 GENERAL QUESTIONS ABOUT TRANSHUMANISM(00:31) 1.1 What is transhumanism?(05:48) 1.2 What is a posthuman?(10:11) 1.3 What is a transhuman?(12:57) 2 TECHNOLOGIES AND PROJECTIONS(13:02) 2.1 Biotechnology, genetic engineering, stem cells, and cloning – what are they and what are they good for?(19:51) 2.2 What is molecular nanotechnology?(31:24) 2.3 What is superintelligence?(39:58) 2.4 What is virtual reality?(44:52) 2.5 What is cryonics? Isn’t the probability of success too small?(49:52) 2.6 What is uploading?(57:26) 2.7 What is the singularity?(01:00:26) 3 SOCIETY AND POLITICS(01:00:34) 3.1 Will new technologies only benefit the rich and powerful?(01:03:50) 3.2 Do transhumanists advocate eugenics?(01:10:17) 3.3 Aren’t these future technologies very risky? Could they even cause our extinction?(01:19:57) 3.4 If these technologies are so dangerous, should they be banned? What can be done to reduce the risks?(01:27:47) 3.5 Shouldn’t we concentrate on current problems such as improving the situation of the poor, rather than putting our efforts into planning for the “far” future?(01:31:17) 3.6 Will extended life worsen overpopulation problems?(01:40:53) 3.7 Is there any ethical standard by which transhumanists judge “improvement of the human condition”?(01:45:25) 3.8 What kind of society would posthumans live in?(01:48:43) 3.9 Will posthumans or superintelligent machines pose a threat to humans who aren’t augmented?(01:53:38) 4 TRANSHUMANISM AND NATURE(01:53:44) 4.1 Why do transhumanists want to live longer?(01:56:51) 4.2 Isn’t this tampering with nature?(02:00:01) 4.3 Will transhuman technologies make us inhuman?(02:01:48) 4.4 Isn’t death part of the natural order of things?(02:07:28) 4.5 Are transhumanist technologies environmentally sound?(02:09:49) 5 TRANSHUMANISM AS A PHILOSOPHICAL AND CULTURAL VIEWPOINT(02:09:56) 5.1 What are the philosophical and cultural antecedents of transhumanism?(02:27:36) 5.2 What currents are there within transhumanism? Is extropianism the same as transhumanism?(02:33:07) 5.3 How does transhumanism relate to religion?(02:35:59) 5.4 Won’t things like uploading, cryonics, and AI fail because they can’t preserve or create the soul?(02:38:02) 5.5 What kind of transhumanist art is there?(02:41:17) 6 PRACTICALITIES(02:41:21) 6.1 What are the reasons to expect all these changes?(02:44:46) 6.2 Won’t these developments take thousands or millions of years?(02:48:10) 6.3 What if it doesn’t work?(02:49:45) 6.4 How can I use transhumanism in my own life?(02:51:27) 6.5 How could I become a posthuman?(02:53:23) 6.6 Won’t it be boring to live forever in a perfect world?(02:58:16) 6.7 How can I get involved and contribute?(03:00:41) ACKNOWLEDGEMENTS AND DOCUMENT HISTORY

By Nick Bostrom.Abstract:Evolutionary development is sometimes thought of as exhibiting an inexorable trend towards higher, more complex, and normatively worthwhile forms of life. This paper explores some dystopian scenarios where freewheeling evolutionary developments, while continuing to produce complex and intelligent forms of organization, lead to the gradual elimination of all forms of being that we care about. We then consider how such catastrophic outcomes could be avoided and argue that under certain conditions the only possible remedy would be a globally coordinated policy to control human evolution by modifying the fitness function of future intelligent life forms.Read the full paper:https://nickbostrom.com/fut/evolutionMore episodes at:https://radiobostrom.com/ ---Outline:(00:16) Abstract(01:06) 1. The Panglossian view(07:50) 2. Two dystopian "upward" evolutionary scenarios(17:35) 3. Ours is an evolutionary disequilibrium(24:55) 4. Costly signaling and flamboyant display?(30:07) 5. Two senses of outcompeted(32:31) 6. Could we control our own evolution?(35:30) 7. Preventing non-eudaemonic agents from arising(39:56) 8. Modifying the fitness function(43:34) 9. Policies for evolutionary steering(49:48) 10. Detour(52:44) 11. Only a singleton could control evolution(59:22) 12. Conclusion

By Nick Bostrom.Abstract:The purpose of this paper, boldly stated, is to propose a new type of philosophy, a philosophy whose aim is prediction. The pace of technological progress is increasing very rapidly: it looks as if we are witnessing an exponential growth, the growth-rate being proportional to the size already obtained, with scientific knowledge doubling every 10 to 20 years since the second world war, and with computer processor speed doubling every 18 months or so. It is argued that this technological development makes urgent many empirical questions which a philosopher could be well-suited to help answering. I try to cover a broad range of interesting problems and approaches, which means that I won't go at all deeply into any of them; I only try to say enough to show what some of the problems are, how one can begin to work with them, and why philosophy is relevant. My hope is that this will whet your appetite to deal with these questions, or at least increase general awareness that they worthy tasks for first-class intellects, including ones which might belong to philosophers.Read the full paper:https://nickbostrom.com/old/predictMore episodes at:https://radiobostrom.com/ ---Outline:(00:19) Abstract(01:52) 1. The Polymath(08:41) 2. The Carter-Leslie Doomsday Argument and the Anthropic Principle(15:28) 3. The Fermi Paradox(43:49) 4. Superintelligence(58:27) 5. Uploading, Cyberspace and Cosmology(01:16:14) 6. Attractors and Values(01:28:41) 7. Transhumanism