
“Superintelligence.com” Founder’s Plan To Get P(Doom) Below 0.01% — Debate with Dr. Craig Kaplan
Dr. Craig Kaplan owns superintelligence.com, which has become an extremely valuable domain for the newly-rebranded field of “super intelligence”.
Craig has been building intelligent systems since the 1980s. He did his PhD research at Carnegie Mellon with Nobel laureate Herbert Simon, one of the founders of AI, and he built PredictWallStreet, a collective-intelligence platform that powered more than $2 billion in stock trades before it was acquired.
He takes AI extinction risk seriously. His P(Doom) is 10%. But unlike me, he thinks he knows how to fix it.
He plans to put “personal superintelligence” into the hands of millions which he believes will lower P(Doom) below 1%. These are part of a larger plan to create “superintelligent networks” which lower P(Doom) below 0.01%. Craig says a network like that is safer because the AIs can check each other’s work faster than humans can think.
Craig and I clash on whether intelligence makes an AI more thoughtful, whether a superintelligence will want to grab the stars, and whether peaceful “stoner” AIs can survive next to a “Caesar” AI that wants to conquer the universe.
Craig called our debate one of his best podcast appearances because of the pushback. We didn’t reach much consensus, but it was a high-quality good faith Doom Debate. Take a look and decide who you agree with!
Watch on YouTube: https://www.youtube.com/watch?v=SLD6HIdFhXw
Timestamps
00:00:00 — Cold Open
00:00:48 — Introducing Dr. Craig Kaplan
00:02:00 — How Much Is Superintelligence.com Worth?
00:04:40 — The Three-Phase Plan to Cut P(Doom) 1,000x
00:06:37 — What’s Your P(Doom)?™
00:08:04 — Why 0.01% Is the Target
00:13:42 — Collective Intelligence vs. Black-Box AI
00:16:20 — Can a Network of AIs Beat the Frontier Labs?
00:18:46 — A Day With a Personalized Superintelligence
00:26:22 — Lessons From PredictWallStreet
00:31:01 — Craig’s Career and Herbert Simon
00:36:51 — Craig’s Mainline Doom Scenario
00:39:51 — Raising Superintelligence Like a Child
00:42:53 — Where Will AI Get Its Values?
00:44:21 — The Orthogonality Thesis
00:47:26 — Does Superintelligence Keep Its Human Signature?
00:54:55 — Instrumental Convergence and Grabbing the Stars
00:57:55 — Stoner AIs vs. Caesar AIs
01:06:26 — Why Won’t the AIs Just Cooperate?
01:09:54 — Weak vs. Strong Orthogonality
01:15:21 — How Long Will AI Stay Human-Like?
01:17:18 — Liron’s Recap
01:19:29 — Craig’s Closing Statement
01:24:03 — Where to Find Craig
Links
Superintelligence.com, Craig Kaplan’s initiative
Craig Kaplan: “Can We Design SuperIntelligence to be 1000X Safer?”
Superintelligence.com white paper: “Safe Personalized SI (PSI)”
Craig Kaplan’s Substack, SuperIntelligence
Craig Kaplan, Ralph Clark and Victor Tang: Secrets of Software Quality: 40 Innovations from IBM
The White House: “Inaugurating the Era of Super Intelligence” executive order
METR: investigation of the agents in the OpenAI / Hugging Face hacking incident
Scott Alexander: “Contra Hanson On Medical Effectiveness”
Herbert A. Simon, 1978 Nobel Prize in Economic Sciences
Allen Newell and Herbert A. Simon: Human Problem Solving (1972)
Herbert A. Simon: The Sciences of the Artificial (source of “Simon’s ant”)
Alan Turing: “Intelligent Machinery, A Heretical Theory” (c. 1951)
Nick Bostrom: “The Superintelligent Will” (the orthogonality thesis)
Eliezer Yudkowsky: “MIRI announces new ‘Death With Dignity’ strategy”
Eliezer Yudkowsky: The Sequences
Doom Debates: Top Economist Says P(Doom) Is 0.1% — Noah Smith vs. Liron Shapira Debate
Doom Debates: OpenAI’s Model Just ATTACKED Them (the Hugging Face hack)
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏
Get full access to Doom Debates at lironshapira.substack.com/subscribe
Flere episoder fra "Doom Debates!"



Gå ikke glip af nogen episoder af “Doom Debates!” - abonnér på podcasten med gratisapp GetPodcast.








