English-language podcasts

Doom Debates!

“Superintelligence.com” Founder’s Plan To Get P(Doom) Below 0.01% — Debate with Dr. Craig Kaplan

86 min

Listen through the publisher, then discuss the episode here or arrange an in-person Roundtable. Your city determines where to meet.

Roundtables are designed for groups of four, with a shared subject and room for everyone to speak. Come for discussion, with no dating expectation.

Dr. Craig Kaplan owns superintelligence.com, which has become an extremely valuable domain for the newly-rebranded field of “super intelligence”. Craig has been building intelligent systems since the 1980s. He did his PhD research at Carnegie Mellon with Nobel laureate Herbert Simon, one of the founders of AI, and he built PredictWallStreet, a collective-intelligence platform that powered more than $2 billion in stock trades before it was acquired. He takes AI extinction risk seriously. His P(Doom) is 10%. But unlike me, he thinks he knows how to fix it. He plans to put “personal superintelligence” into the hands of millions which he believes will lower P(Doom) below 1%. These are part of a larger plan to create “superintelligent networks” which lower P(Doom) below 0.01%. Craig says a network like that is safer because the AIs can check each other’s work faster than humans can think. Craig and I clash on whether intelligence makes an AI more thoughtful, whether a superintelligence will want to grab the stars, and whether peaceful “stoner” AIs can survive next to a “Caesar” AI that wants to conquer the universe. Craig called our debate one of his best podcast appearances because of the pushback. We didn’t reach much consensus, but it was a high-quality good faith Doom Debate. Take a look and decide who you agree with! Watch on YouTube: https://www.youtube.com/watch?v=SLD6HIdFhXw Timestamps 00:00:00 — Cold Open 00:00:48 — Introducing Dr. Craig Kaplan 00:02:00 — How Much Is Superintelligence.com Worth? 00:04:40 — The Three-Phase Plan to Cut P(Doom) 1,000x 00:06:37 — What’s Your P(Doom)?™ 00:08:04 — Why 0.01% Is the Target 00:13:42 — Collective Intelligence vs. Black-Box AI 00:16:20 — Can a Network of AIs Beat the Frontier Labs? 00:18:46 — A Day With a Personalized Superintelligence 00:26:22 — Lessons From PredictWallStreet 00:31:01 — Craig’s Career and Herbert Simon 00:36:51 — Craig’s Mainline Doom Scenario 00:39:51 — Raising Superintelligence Like a Child 00:42:53 — Where Will AI Get Its Values? 00:44:21 — The Orthogonality Thesis 00:47:26 — Does Superintelligence Keep Its Human Signature? 00:54:55 — Instrumental Convergence and Grabbing the Stars 00:57:55 — Stoner AIs vs. Caesar AIs 01:06:26 — Why Won’t the AIs Just Cooperate? 01:09:54 — Weak vs. Strong Orthogonality 01:15:21 — How Long Will AI Stay Human-Like? 01:17:18 — Liron’s Recap 01:19:29 — Craig’s Closing Statement 01:24:03 — Where to Find Craig Links Superintelligence.com, Craig Kaplan’s initiative (https://www.superintelligence.com/) Craig Kaplan: “Can We Design SuperIntelligence to be 1000X Safer?” (https://www.superintelligence.com/releases-1/can-we-design-superintelligence-to-be-1000x-safer) Superintelligence.com white paper: “Safe Personalized SI (PSI)” (https://www.superintelligence.com/whitepaper-5-personalized-si) Craig Kaplan’s Substack, SuperIntelligence (https://read.superintelligence.com/) SuperIntelligence on YouTube (https://www.youtube.com/@SuperintelligencebyiQ) Craig Kaplan, Ralph Clark and Victor Tang: Secrets of Software Quality: 40 Innovations from IBM (https://www.goodreads.com/work/editions/3454766-secrets-of-software-quality-40-innovations-from-ibm-book-and-disk-mcgr) The White House: “Inaugurating the Era of Super Intelligence” executive order (https://www.whitehouse.gov/presidential-actions/2026/09/inaugurating-the-era-of-super-intelligence/) METR: investigation of the agents in the OpenAI / Hugging Face hacking incident (https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/) Scott Alexander: “Contra Hanson On Medical Effectiveness” (https://www.astralcodexten.com/p/contra-hanson-on-medical-effectiveness) Herbert A. Simon, 1978 Nobel Prize in Economic Sciences (https://www.nobelprize.org/prizes/economic-sciences/1978/simon/facts/) Allen Newell and Herbert A. Simon: Human Problem Solving (1972) (https://archive.org/details/humanproblemsolv0000newe) Herbert A. Simon: The Sciences of the Artificial (source of “Simon’s ant”) (https://en.wikipedia.org/wiki/The_Sciences_of_the_Artificial) Alan Turing: “Intelligent Machinery, A Heretical Theory” (c. 1951) (https://gwern.net/doc/ai/1951-turing.pdf) Nick Bostrom: “The Superintelligent Will” (the orthogonality thesis) (https://nickbostrom.com/superintelligentwill.pdf) Instrumental convergence (https://www.lesswrong.com/w/instrumental-convergence) Eliezer Yudkowsky: “MIRI announces new ‘Death With Dignity’ strategy” (https://www.lesswrong.com/posts/j9Q8bRmwCgXRYAgcJ/miri-announces-new-death-with-dignity-strategy) Eliezer Yudkowsky: The Sequences (https://www.readthesequences.com/) Doom Debates: Top Economist Says P(Doom) Is 0.1% — Noah Smith vs. Liron Shapira Debate (https://www.youtube.com/watch?v=AwmJ-OnK2I4) Doom Debates: This Cybersecurity Expert Is Confident That Humanity Will SURVIVE — Debate with Zack Korman (https://www.youtube.com/watch?v=QJTeZfx1ZLk) Doom Debates: OpenAI’s Model Just ATTACKED Them (the Hugging Face hack) (https://www.youtube.com/watch?v=RczYubQzXbI) Join the Doom Debates Discord (https://doomdebates.com/discord) Support Doom Debates (https://doomdebates.com/donate) Support the mission by subscribing to my Substack at DoomDebates.com (https://doomdebates.com/) and to youtube.com/@DoomDebates (https://youtube.com/@DoomDebates), or to really take things to the next level: Donate (https://doomdebates.com/donate)

Episode information is provided by the publisher. Listening takes place on the publisher’s service.

Listen at the source ↗

Featured places to meet in Palo Alto, CA

Explore these places and arrange a conversation through ThinkMatch.

City guide for Palo Alto, CA

More from Doom Debates!

Doom Debates!

Jensen Huang Resigns, RSI Coming in 2027 — with Robert Wright

104 min

Bestselling author Robert Wright and I react to Jensen Huang’s resignation from Nvidia, METR’s report that AI is doubling Anthropic’s research speed, and whether AI safety can ever be formally verified. Plus: Where AI’s economic value will accrue; My AI outperforming my CPA; What Jensen’s fall means for Anthropic’s IPO; And an anonymous Chinese AI journalist on how it all looks from Beijing. Enjoy another edition of the Nonzero x Doom Debates collab! Watch on YouTube: https://www.youtube.com/watch?v=uZZEfe2OmAE Timestamps 00:00:00 — Cold Open 00:00:30 — Introductions 00:01:32 — Jensen Huang Resigns from Nvidia 00:05:02 — Who’s Defending Jensen? 00:10:09 — What Did Jensen Do Wrong? 00:12:23 — Was Alice Huang a Chinese Spy? 00:18:03 — Does Decapitation Work? 00:21:34 — The Great Man Theory of Tech 00:23:58 — Where Will AI’s Value Accrue? 00:27:14 — What’s Next for Jensen? 00:29:33 — METR: AI Is Doubling Anthropic’s Research Speed 00:35:01 — AI Is Solving Open Math Problems 00:37:24 — AI

Discuss this episode

Doom Debates!

There’s ZERO Chance of AI Doom By 2050 — Debate with Prakash Narayanan, Host of AI in the AM

120 min

Prakash Narayanan is the man behind @8teAPi, an X account with nearly 60,000 followers that reacts to AI news from a techno-optimist perspective. He’s also the co-host of AI in the AM with friend-of-the-show Nathan Labenz. He spent almost 20 years in investment banking and private equity, used to post anonymously, and had e/acc in his profile before deciding the movement wasn’t for him. Now he’s stepping into the spotlight to defend the pro-AI position, which is more than I can say for most of the pro-AI voices on Twitter who like to snipe at me. We actually agree on a lot about how the technology is advancing. Prakash expects recursive self-improvement within a year and says AI may already be smarter than any individual human. He even gives a 5% chance that every human body gets discarded for a silicon substrate by 2050. And yet his P(Doom) by 2050 is 0%. Not 1%. Not 1 in 100,000. “Literally zero.” His reasoning is that AI is a multiplayer game. A market where somebody has to pay for

Discuss this episode

Doom Debates!

Can Republicans STOP AI Doom? — Brendan & Randan Steinhauser

84 min

Brendan and Randan Steinhauser are a husband and wife team who know how to make things happen in conservative politics. Brendan helped build the Tea Party movement and ran John Cornyn’s Senate campaign. Randan has been involved in advocacy and political campaigns in more than 30 states, including working under former Secretary of Education Betsy DeVos. Now they’re bringing Republicans, parents, and faith leaders into the fight against AI catastrophe. They know how to build political momentum, but my question is whether that will lead to action quickly enough. I'm skeptical of incremental legislation efforts and data center protests, but they argue it's necessary coalition-building that leads to superintelligence regulation. This isn't your typical state-of-the-political-map podcast. We also get into: Their personal AGI awakenings How people of faith make sense of claims of Godlike AI and AI doom My own transition to joining the Republican Party Transhumanism, souls, and mind uploading

Discuss this episode

Community Poll

Teams owe supporters honest explanations during long periods of poor results.

Share your view, then see how readers voted. No sign-in required. This poll is shared across stories.

Do you agree or disagree?

Choose one answer

Common Quandary

A Common Quandary to discuss

Another conversation to explore, separate from this news story.

You lecture everyone about climate change—then get invited on a dream flight. Do you go?

A close friend living overseas invites you to an important celebration. Flying is affordable but carbon-intensive. Your individual decision will not determine whether the flight operates, yet it conflicts with commitments you publicly encourage others to make.

When is a small personal contribution to a large harm morally significant?

Explore this quandary