Doom Debates!
- Episodios
- 201
- Se publica
- cada semana
- Último episodio
- 13 Nov 2026
Sobre el podcast
It's time to talk about the end of the world. With your host, Liron Shapira.
Episodios 201
Jensen Huang Resigns, RSI Coming in 2027 — with Robert Wright
Bestselling author Robert Wright and I react to Jensen Huang’s resignation from Nvidia, METR’s report that AI is doubling Anthropic’s research speed, and whether AI safety can ever be formally verified …
Notas del episodio
Bestselling author Robert Wright and I react to Jensen Huang’s resignation from Nvidia, METR’s report that AI is doubling Anthropic’s research speed, and whether AI safety can ever be formally verified. Plus:
Where AI’s economic value will accrue;
My AI outperforming my CPA;
What Jensen’s fall means for Anthropic’s IPO;
And an anonymous Chinese AI journalist on how it all looks from Beijing.
Enjoy another edition of the Nonzero x Doom Debates collab!
Watch on YouTube: https://www.youtube.com/watch?v=uZZEfe2OmAE
Timestamps
00:00:00 — Cold Open
00:00:30 — Introductions
00:01:32 — Jensen Huang Resigns from Nvidia
00:05:02 — Who’s Defending Jensen?
00:10:09 — What Did Jensen Do Wrong?
00:12:23 — Was Alice Huang a Chinese Spy?
00:18:03 — Does Decapitation Work?
00:21:34 — The Great Man Theory of Tech
00:23:58 — Where Will AI’s Value Accrue?
00:27:14 — What’s Next for Jensen?
00:29:33 — METR: AI Is Doubling Anthropic’s Research Speed
00:35:01 — AI Is Solving Open Math Problems
00:37:24 — AI Agents Negotiating Deals
00:40:03 — My AI vs. My CPA
00:42:38 — Bob’s AI Caught His Doctor’s Error
00:44:20 — Whatever Happened to Hallucinations?
00:45:35 — Formal Verification and Lean
00:50:47 — Andrew Wiles and the Human Cost
00:54:10 — The All-In Podcast’s Blind Spot
00:57:15 — Can We Formally Verify Alignment?
00:59:55 — Yudkowsky vs. Irving at ILIAD
01:01:21 — Will Jensen’s Fall Help Anthropic’s IPO?
01:06:48 — The March for an AI Treaty
01:08:02 — Introducing Walter, a Chinese AI Journalist
01:09:22 — How China Reacted to Jensen’s Resignation
01:15:43 — Chinese Theories About Anthropic’s Role
01:22:47 — How China Censored the Spy Allegation
01:24:33 — Does China Trust an AI Pause?
01:26:59 — AI Layoffs and a Chili Sauce Boycott
01:32:50 — Does Beijing Listen to Public Opinion?
01:36:42 — China’s Limits on AI Companions for Kids
01:39:29 — Will China Move on Taiwan?
Links
ROBERT WRIGHT & NONZERO
Subscribe to The NonZero Newsletter
The God Test — Bob’s 2026 book on AI
“The Age of Big Math” — Bob’s Nonzero conversation with Steven Strogatz, author of Big Math
“When AI Enters the Quantum World” — Bob’s Nonzero conversation with Scott Aaronson
THE WEEK IN AI
Elon Musk: “That would be an honor” — his reply to “Elon please hire Jensen”
Daniel Kokotajlo to Anthropic: “Do you want to pace or not?” — the tweet where he drops the F-bomb
“Formalizing Fermat’s Last Theorem” — Anthropic on Claude’s 13-million-line Lean proof
Eliezer Yudkowsky & Geoffrey Irving Debate — recorded at the ILIAD conference at Lighthaven
March for an AI Treaty — Saturday, November 14, in cities across the US, Canada, France and Germany
ALSO MENTIONED
“The Proof” — the NOVA documentary on Andrew Wiles and Fermat’s Last Theorem
“AI as Normal Technology” — the essay by Arvind Narayanan and Sayash Kapoor, Bob’s upcoming guest
DOOM DEBATES
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏
Get full access to Doom Debates at lironshapira.substack.com/subscribeThere’s ZERO Chance of AI Doom By 2050 — Debate with Prakash Narayanan, Host of AI in the AM
Prakash Narayanan is the man behind @8teAPi, an X account with nearly 60,000 followers that reacts to AI news from a techno-optimist perspective. He’s also the co-host of AI in the AM with friend-of-the-show …
Notas del episodio
Prakash Narayanan is the man behind @8teAPi, an X account with nearly 60,000 followers that reacts to AI news from a techno-optimist perspective. He’s also the co-host of AI in the AM with friend-of-the-show Nathan Labenz.
He spent almost 20 years in investment banking and private equity, used to post anonymously, and had e/acc in his profile before deciding the movement wasn’t for him.
Now he’s stepping into the spotlight to defend the pro-AI position, which is more than I can say for most of the pro-AI voices on Twitter who like to snipe at me.
We actually agree on a lot about how the technology is advancing. Prakash expects recursive self-improvement within a year and says AI may already be smarter than any individual human. He even gives a 5% chance that every human body gets discarded for a silicon substrate by 2050. And yet his P(Doom) by 2050 is 0%. Not 1%. Not 1 in 100,000. “Literally zero.”
His reasoning is that AI is a multiplayer game. A market where somebody has to pay for every data center will keep any one AI from running away with the future. He even calls the doomer scenario a “perpetual motion machine.”
We debate whether humans have any power to lose in the first place, the feasibility of a nanomachine takeover, and why someone with zero fear of doom doesn’t just floor the accelerator.
This was a fun and occasionally high intensity debate. At one point, Prakash asked if I was “crazy” for doubting that most of humanity puts P(Doom) at zero. But my view is that his forecast falls outside the epistemological “sane zone.” Take a look and see who you agree with!
Watch on YouTube: https://www.youtube.com/watch?v=EmOYwV5k0SA
Timestamps
00:00:00 — Cold Open
00:01:24 — Introducing Prakash Narayanan
00:02:19 — How @8teAPi Went Viral
00:05:12 — Does Prakash Still Identify as e/acc?
00:07:05 — What’s Your P(Doom)?™
00:10:02 — Prakash Defends 0%: Most of Humanity Isn’t Worried
00:15:17 — Prakash’s Core Argument: AI Is a Multiplayer Game
00:20:36 — Bayesian Test: Can Anything Change Prakash’s Mind?
00:24:53 — Defining Doom
00:31:48 — The Parable of the Sliding Door
00:34:54 — Doom Train™: Will Superintelligence Outsmart Humanity?
00:42:38 — Do Humans Have Any Power to Lose?
00:54:05 — Are Doomers the Outliers?
00:56:03 — Will AI Have Empathy?
01:05:26 — Will AIs Keep Stable Goals?
01:10:41 — Hypothetical: The Paperclip Brain Laser
01:14:16 — Hypothetical: A Million Times More Change
01:19:04 — Debating Nanomachines in the Future
01:30:32 — Jobs, Status, and Prakash’s Real Fear
01:40:37 — Last Question: Why Not Floor the Accelerator?
01:52:49 — Wrap-Up
01:56:55 — March for an AI Treaty
01:58:25 — Donation Drive
Links
Nathan Labenz’s podcast, The Cognitive Revolution
Prakash’s viral 2023 thread on Palantir’s battlefield AI demo
Paul Christiano: “What failure looks like”
Eliezer Yudkowsky and Nate Soares: If Anyone Builds It, Everyone Dies
Comparative advantage (Ricardo’s law)
Effective accelerationism (e/acc)
Doom Debates: Liron on AI in the AM with Nathan Labenz and Prakash
Doom Debates: Top Economist Says P(Doom) Is 0.1% — Noah Smith vs. Liron Shapira Debate
March for an AI Treaty, Saturday, November 14
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏
Get full access to Doom Debates at lironshapira.substack.com/subscribeCan Republicans STOP AI Doom? — Brendan & Randan Steinhauser
Brendan and Randan Steinhauser are a husband and wife team who know how to make things happen in conservative politics.Brendan helped build the Tea Party movement and ran John Cornyn’s Senate campaign. Randan …
Notas del episodio
Brendan and Randan Steinhauser are a husband and wife team who know how to make things happen in conservative politics.
Brendan helped build the Tea Party movement and ran John Cornyn’s Senate campaign. Randan has been involved in advocacy and political campaigns in more than 30 states, including working under former Secretary of Education Betsy DeVos.
Now they’re bringing Republicans, parents, and faith leaders into the fight against AI catastrophe.
They know how to build political momentum, but my question is whether that will lead to action quickly enough. I'm skeptical of incremental legislation efforts and data center protests, but they argue it's necessary coalition-building that leads to superintelligence regulation.
This isn't your typical state-of-the-political-map podcast. We also get into:
Their personal AGI awakenings
How people of faith make sense of claims of Godlike AI and AI doom
My own transition to joining the Republican Party
Transhumanism, souls, and mind uploading
Parenting in the age of AI
It’s a candid conversation with a couple who are making the fight for our future feel a little more winnable. Enjoy!
Watch on YouTube: https://www.youtube.com/watch?v=XTm9tqS1Ibw
Timestamps
00:00:00 — Cold Open
00:01:07 — Introducing Brendan & Randan Steinhauser
00:02:22 — Why I Came Out as Republican
00:04:28 — The Alliance for Secure AI
00:06:30 — Losing Control of Superintelligence
00:07:52 — Brendan, What’s Your P(Doom)?™
00:08:26 — Randan, What’s Your P(Doom)?™
00:10:47 — Finding the AI Safety Movement
00:15:04 — Trump, Republicans, and Big Tech Money
00:19:22 — Bringing Faith Leaders Into AI Safety
00:23:23 — God’s Plan and Human Extinction
00:29:43 — Priorities for Congress
00:31:42 — The Morality of AI Companies
00:34:03 — PauseAI and Employee Whistleblowers
00:43:34 — Effective Altruism and a Broader Coalition
00:48:12 — Transhumanism, Souls, and Mind Uploading
00:57:40 — Building a Machine God
01:03:20 — Raising Children Under AI Risk
01:07:35 — Data Center Protests and Movement Building
01:13:09 — Talking to People About AI Risk
01:17:32 — State Laws and the 2028 Election
01:20:38 — Getting Involved
Links
REFERENCED IN THE EPISODE
AI Risk Evaluation Act — Hawley–Blumenthal bill text
The Singularity Is Near — Ray Kurzweil
Can We Stop the AI Apocalypse? — Dan Crenshaw interviews Eliezer Yudkowsky
Magnifica Humanitas — Pope Leo XIV’s encyclical on humanity in the age of AI
Pacing the Frontier — open letter from frontier AI employees
PauseAI — the global movement to pause unsafe AI development
PauseAI US — organizing and advocacy in the United States
Jacob Coxon’s resignation statement — the insider warning discussed
Americans for Responsible Innovation — Brad Carson’s bipartisan advocacy organization
PAST DOOM DEBATES EPISODES MENTIONED
Let’s CALL OUT the AI Doom “Enablers” — Holly Elmore on working at AI labs
God vs. AI Doom: Debate with Bentham’s Bulldog
DOOM DEBATES
Support Doom Debates — become a mission partner
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏
Get full access to Doom Debates at lironshapira.substack.com/subscribeIs an AI Bubble Bursting? With Robert Wright’s Nonzero Podcast
Bestselling author Robert Wright and I discuss this week’s stock market jitters, Google’s new interactive world model, and why I sometimes spend $1,000 in a day on AI agents. Plus:The Binance hack;An AI audit …
Notas del episodio
Bestselling author Robert Wright and I discuss this week’s stock market jitters, Google’s new interactive world model, and why I sometimes spend $1,000 in a day on AI agents. Plus:
The Binance hack;
An AI audit of two star economists;
Anthropic’s financial prospects;
And whether AI could crack the math behind encryption.
Enjoy another edition of the Nonzero x Doom Debates collab!
Watch on YouTube: https://www.youtube.com/watch?v=wkKpPeMVVX8
Timestamps
00:00:00 — Teaser
00:00:50 — Nonzero Doom returns
00:03:07 — A foom in slow motion
00:07:47 — Is AI winter canceled forever?
00:10:33 — Will Google’s Genie 4 lure us into the Matrix?
00:16:54 — Liron’s bull case for Google stock
00:24:34 — Does Google have big-company disease?
00:28:28 — Could a slowdown actually help Anthropic and OpenAI?
00:30:12 — Was OpenAI’s “pause” just PR?
00:40:01 — Social science judgment day is coming
00:47:12 — Why even AI’s upsides could tear society apart
00:51:12 — Does OpenAI’s alignment approach improve on Asimov?
00:55:06 — Sam and Satya pivot on alignment
00:58:34 — Crypto and the AI hacking threat
01:03:42 — Could AI crack the math behind encryption?
01:08:52 — Liron’s $1,000-a-day AI coding habit
01:15:23 — Explaining the missing productivity boost
01:17:26 — Does Anthropic’s S-1 sink the AI bubble thesis?
Links
ROBERT WRIGHT & NONZERO
Nonzero’s post for this episode — “Is an AI Bubble Bursting?”
Subscribe to The NonZero Newsletter
The God Test — Bob’s 2026 book on AI
THE WEEK IN AI
Where’s Your Ed At — Ed Zitron’s newsletter, home of the AI-bubble bear case
“Introducing Genie 4” — Google DeepMind’s real-time interactive world model
“Gemini 4 Carbon: our next frontier model” — the follow-up Google announced two weeks later
“Google Gemini co-lead Noam Shazeer leaves for OpenAI” — CNBC on the talent Google has bled
“An Updated Model Spec for More Capable and Beneficial AI” — OpenAI’s announcement
2026 Binance hack — Wikipedia’s summary of the theft Bob and Liron debate
ALSO MENTIONED
PauseAI US — Holly Elmore’s organization, plugged at the close of the episode
DOOM DEBATES
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏
Get full access to Doom Debates at lironshapira.substack.com/subscribeZack Korman, Cybersecurity Expert, Is Confident That Humanity Will SURVIVE
Cybersecurity expert and entrepreneur Zack Korman is an increasingly prominent nondoomer voice on Twitter. Naturally, with AI now breaking out of sandboxes and hacking corporate systems, people want to hear …
Notas del episodio
Cybersecurity expert and entrepreneur Zack Korman is an increasingly prominent nondoomer voice on Twitter. Naturally, with AI now breaking out of sandboxes and hacking corporate systems, people want to hear what cybersecurity experts have to say. So I’m thrilled that Zack was brave enough to face me on Doom Debates!
Zack was reading Nick Bostrom at Oxford and worrying about AI doom as far back as 2016, but his views have since shifted. In a recent post, he writes: “This focus on AI causing human extinction is going to get people killed.” Now he expects security failures to give humanity plenty of opportunities to learn, fight back, and stop risky superintelligence development if necessary.
We argue about whether sandboxing and AI monitoring are hopeless strategies, whether competing superintelligences would keep each other in check, whether to expect instrumental convergence, and more.
Watch on YouTube: https://youtu.be/QJTeZfx1ZLk
Timestamps
00:00:00 — Cold Open
00:01:15 — Introducing Zack Korman
00:03:41 — From Law to AI Security
00:09:17 — Why Zack Stopped Being a Doomer
00:12:03 — What’s Your P(Doom)?™
00:21:00 — Why Zack Supports AI Safety
00:25:54 — Believing in Superintelligence
00:30:06 — AI Without Tools
00:39:42 — Warning Shots and Time to React
00:46:08 — A Country of Geniuses in a Datacenter
00:55:55 — Can We Sandbox AI?
01:05:58 — Monitoring and Layers of Defense
01:14:49 — Riding the Doom Train™
01:22:23 — The Case for Accelerating
01:24:46 — AI CEOs and the Transfer of Power
01:34:19 — Who Presses the Stop Button?
01:41:38 — Humans, Bacteria, and the Limits of Power
01:48:30 — When Would Zack Pause AI?
01:56:13 — Will Different AIs Keep Each Other in Check?
02:06:11 — Can Progress Turn Against Us?
02:08:35 — The Ideological Turing Test
02:14:58 — Closing Thoughts
Links
Zack Korman: “AI doomers are furious”
Zack Korman: “AI doomers are anti-security”
Zack Korman: “3 reasons sandboxing won’t secure your AI agent”
Zack Korman: “This focus on AI causing human extinction is going to get people killed.”
METR’s investigation of the OpenAI–Hugging Face incident
The PHP Foundation: “Today’s PHP Security Release, and How We’re Strengthening PHP Security”
Nick Bostrom: Superintelligence
Center for AI Safety: Statement on AI Risk
Eliezer Yudkowsky and Nate Soares: If Anyone Builds It, Everyone Dies
Zvi Mowshowitz: “The Three AI Pills”
Eliezer Yudkowsky: “Value is Fragile”
Doom Debates: Robin Hanson vs. Liron Shapira — Is Near-Term Extinction From AGI Plausible?
Doom Debates: OpenAI’s BOMBSHELL Hack — Swarms of Agents, Zero-Day Exploits, & Misaligned AI
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏
Get full access to Doom Debates at lironshapira.substack.com/subscribe“AI Chernobyl” Will Scare Trump and Xi To Order An Immediate Pause — Jeremie & Edouard Harris
Physicists Jeremie and Edouard Harris, co-founders of Gladstone AI, are preparing for an “AI Chernobyl” warning shot.If that forces Trump and Xi to negotiate an AI slowdown, the Harris brothers want to ensure …
Notas del episodio
Physicists Jeremie and Edouard Harris, co-founders of Gladstone AI, are preparing for an “AI Chernobyl” warning shot.
If that forces Trump and Xi to negotiate an AI slowdown, the Harris brothers want to ensure the US has the enforcement tools to make the deal stick. More specifically, they want verification technologies developed and vetted before the crisis hits, alongside enforcement capabilities to deter cheating.
Yet when I bring up Eliezer Yudkowsky’s writing on the possibility of airstrikes on rogue data centers as a last-resort treaty deterrent, they push back. They also don’t share my enthusiasm for PauseAI, expressing uncertainty about whether the movement has been net beneficial.
Jeremie and Edouard have earned a ton of credibility:
• They redirected their careers around the AI threat as far back as 2015, stepping away from their Y Combinator startup before ChatGPT even launched and brought AI into the mainstream.
• They’ve built relationships with senior government decision-makers without a prior national security background.
• They’ve demonstrated a talent for public communication in their excellent Joe Rogan appearances.
In this episode, they bring a level of multi-step strategic thinking that I rarely explore. But I can’t help wondering whether a catastrophe 50 times worse than the Hugging Face incident would let world leaders move much faster on a global AI pause treaty than they expect.
Let’s see if they can convince me that an emergency US–China agreement is our best path to a safer world.
Watch on YouTube: https://youtu.be/4jmFcE4WZ7M
Timestamps
00:00:00 — Cold Open
00:00:53 — Introducing Jeremie and Edouard Harris
00:02:16 — The GPT-3 Wake-Up Call
00:03:59 — AI Risk Outside the EA Community
00:08:49 — The Harris Brothers’ Backgrounds
00:16:37 — Yudkowsky and Many-Worlds Quantum Mechanics
00:22:00 — SharpestMinds and Leaving Academia
00:25:04 — Briefing the US Government
00:33:01 — How Gladstone AI Pays the Bills
00:35:00 — The Trump–Xi Phone Call
00:39:29 — Pacing Isn’t Pausing
00:42:45 — Defining AI Chernobyl
00:44:07 — Verifying China’s AI Slowdown
00:47:10 — Espionage and Lab Security
00:54:10 — Offensive Operations and Deterrence
00:57:47 — The US–China AI Lead
01:05:27 — Why a Treaty Won’t Happen
01:08:01 — Red Lines and Enforcement
01:16:25 — Warning Shots and Extinction
01:21:26 — What’s Your P(Doom)?™
01:24:39 — PauseAI and Movement Politics
01:29:16 — Wrap-Up
Links
• Their treaty analysis: “Arms control negotiators think there won’t be a U.S.-China AI treaty”
• The Endgame — the Harris brothers’ Substack
• Their 2024 government-commissioned AI action plan
• Their 2025 report: America’s Superintelligence Project
• Eliezer Yudkowsky’s 2023 TIME essay advocating a global shutdown
• Joe Rogan Experience #2156 — Jeremie and Edouard Harris, May 2024
• Joe Rogan Experience #2311 — Jeremie and Edouard Harris, April 2025
• Join the Doom Debates Discord
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏
Get full access to Doom Debates at lironshapira.substack.com/subscribeThe Case Against Eliezer Yudkowsky — with Holly Elmore, PauseAI US
Dr. Holly Elmore is the founder and executive director of PauseAI US and holds a Harvard PhD in evolutionary biology. She’s back, and this time she’s taking on none other than Eliezer Yudkowsky!She argues his …
Notas del episodio
Dr. Holly Elmore is the founder and executive director of PauseAI US and holds a Harvard PhD in evolutionary biology. She’s back, and this time she’s taking on none other than Eliezer Yudkowsky!
She argues his public behavior is alienating the people PauseAI US needs to reach, but I wonder if she’s asking too much of the man who laid the foundation for the AI safety movement.
We also clash over Yudkowsky’s coherent extrapolated volition. She thinks the theory doesn’t properly factor in humanity’s consent, but I argue that’s already embedded into the CEV framework.
Holly and I dive into the rationalists’ involvement in a controversial kink scene, John Wentworth’s defunding news, transhumanism, pacing the frontier, and “Holly’s Basilisk”—her case for sending AI lab employees to jail.
It’s another great episode with one of the most controversial—and thoughtful—thinkers in AI safety. Enjoy!
Watch on YouTube: https://www.youtube.com/watch?v=Jn8mS3kC-SU
Timestamps
00:00:00 — Cold Open
00:01:09 — Holly Elmore and PauseAI’s Political Breakthrough
00:07:36 — Pause AI vs. “Pacing the Frontier”
00:14:17 — Holly’s Case Against Eliezer Yudkowsky
00:16:20 — Coherent Extrapolated Volition and Pivotal Acts
00:31:19 — Brain Copies, Consent, and Self-Determination
00:36:56 — Transhumanism, Dyson Swarms, and Humanity’s Future
00:45:36 — The Missing Democratic Checkpoint
00:51:04 — Yudkowsky’s Role in Accelerating AI
01:02:48 — Genius, Arrogance, and Rationalist Culture
01:10:14 — The Aella Photo and Holly’s Breaking Point
01:20:21 — John Wentworth’s Grant Controversy
01:28:17 — PauseAI US’s Rules and Red Lines
01:32:05 — Poe’s Law and the Cost of Sarcasm
01:36:18 — Holly’s Case for Jailing AI Developers
Links
Jacob Coxon’s resignation post
PauseAI leadership disendorsement letter, reproduced on LessWrong
Coherent Extrapolated Volition — Eliezer Yudkowsky (2004 PDF)
Pivotal act — Eliezer Yudkowsky (Arbital mirror)
Genes did misalignment first — Holly Elmore
Holly’s reaction to the Yudkowsky/Aella photo
On Kink & Status in the Rationalist Community — John Wentworth (October 8 response)
Are We a Circular Firing Squad? — previous Holly Elmore episode
Let’s Call Out the AI Doom Enablers — previous Holly Elmore episode
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏
Get full access to Doom Debates at lironshapira.substack.com/subscribeAI Has Become Profoundly WEIRD, But Is Anyone Noticing? With Sarah Hastings-Woodhouse
Sarah Hastings-Woodhouse is an AI safety writer and researcher who went down the AI doom rabbit hole in 2023 and never came back up. After a year as digital communications officer at the UK AI Security …
Notas del episodio
Sarah Hastings-Woodhouse is an AI safety writer and researcher who went down the AI doom rabbit hole in 2023 and never came back up. After a year as digital communications officer at the UK AI Security Institute (AISI), she’s gone independent to speak her mind unfiltered.
Read Sarah’s Substack: https://longerramblings.substack.com
Join PauseAI UK’s March Against AI Extinction in London on November 21: https://luma.com/pauseai-nov26
Watch on YouTube: https://www.youtube.com/watch?v=TDbHCKwuMrk
Timestamps
00:00:00 — Cold Open
00:00:41 — Introducing Sarah Hastings-Woodhouse
00:02:06 — How a Humanities Grad Got AGI-Pilled
00:08:40 — Whose Timelines Do You Trust?
00:11:32 — Joining the UK AI Security Institute
00:15:47 — What AISI Actually Does
00:19:23 — Verifying Anthropic’s Mythos Claims
00:21:59 — Leaving Government to Speak Freely
00:23:22 — What’s Your P(Doom)?™
00:25:37 — Yudkowsky, EA, and Keeping Your Identity Small
00:32:08 — Is Anthropic Doing Enough to Call for a Slowdown?
00:35:56 — Can You Do AI Safety from Second Place?
00:39:20 — Hugging Face and the Vibe Shift
00:44:02 — Is Everything Weird Yet?
00:48:27 — Prevent Recursive Self-Improvement, Don’t Ban It
00:52:28 — Build the Speed Dial, Not the Pause Button
00:56:54 — Was Calling for a Pause in 2023 a Mistake?
01:01:22 — Turning Up the Fear Dial
01:04:23 — Donation Drive
01:05:11 — PauseAI UK’s March Against AI Extinction
Links
SARAH HASTINGS-WOODHOUSE
Consistently Candid — Sarah’s podcast
“Is everything weird yet?” (Aug 30, 2026)
“What the hell just happened?” — on AI safety’s viral moment (Sep 18, 2026)
“We have been warned” — the Hugging Face hack as a warning shot (Jul 27, 2026)
Sarah’s 2023 thread on stumbling into the world of AI doom (“I’m angry because I love the world”)
“The other night I successfully AGI-pilled a man on a first date”
“Welcome, and also I’m sorry” — Sarah’s note to AI safety newcomers (Sep 11, 2026)
REFERENCED IN THE EPISODE
“Our evaluation of Claude Mythos Preview’s cyber capabilities” — AISI (Apr 13, 2026)
AISI Frontier AI Trends Report (Dec 2025)
“The levers of political persuasion with conversational AI” — AISI Human Influence team
US Center for AI Standards and Innovation (CAISI) — NIST
“We’re All Gonna Die with Eliezer Yudkowsky” — Bankless (2023)
“GPT-2 As Step Toward General Intelligence” — Scott Alexander, Slate Star Codex (Feb 2019)
“AGI Ruin: A List of Lethalities” — Eliezer Yudkowsky
“Keep Your Identity Small” — Paul Graham
Pacing the Frontier — the open letter signed by 1,300+ frontier lab employees
“Banning self-recursive improvement in AI models?” — Tyler Cowen, Marginal Revolution (Sep 22, 2026)
March Against AI Extinction — PauseAI UK, London, November 21, 2026
PAST DOOM DEBATES EPISODES MENTIONED
Top Economist Sees AI Doom Coming — Dr. Peter Berezin, BCA Research
Will AI Kill Everyone by 2050? Debate with Dr. Casey Hart (Ontology Explained)
DOOM DEBATES
Transcript, links, and show notes: https://doomdebates.com/p/ai-has-become-profoundly-weird
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏
Get full access to Doom Debates at lironshapira.substack.com/subscribeTrump and the AI Safety Vibe Shift, With Robert Wright | Nonzero × Doom Debates
Bestselling author Robert Wright and I are back for round two of our collaboration, reacting to the week in AI. We cover Trump's summit with the AI labs, the FTC's investigation of OpenAI & Anthropic, the …
Notas del episodio
Bestselling author Robert Wright and I are back for round two of our collaboration, reacting to the week in AI. We cover Trump's summit with the AI labs, the FTC's investigation of OpenAI & Anthropic, the latest Gemini announcement, and much more. The Overton window is moving fast, but is the AI moving faster?
Watch on YouTube: https://www.youtube.com/watch?v=Li-rc6swhQs
Timestamps
00:00:00 — Cold Open
00:00:21 — Round Two of Nonzero × Doom Debates
00:01:23 — Trump, the FTC, and an Industry Feeling the Heat
00:05:37 — Google Says Gemini 4 Exists
00:07:41 — Greg Brockman Pulls His $25M from Leading the Future
00:12:53 — The Break-Ins Keep Coming
00:15:23 — The Market Wants Agent Swarms
00:18:30 — The End of “Alignment by Default”
00:20:55 — The Prelude to Recursive Self-Improvement
00:26:14 — David Sacks Can’t Explain Dario
00:30:31 — The All-In Echo Chamber
00:33:47 — Scott Alexander vs. Steven Pinker
00:38:37 — Jason Calacanis vs. the Public Benefit Corporation
00:43:03 — The Elephant in the Room Is the AI
00:46:21 — What the Average American Thinks
00:48:17 — From Chatbot to Agent to Swarm
00:54:20 — Boeing, Agent Swarms, and the Off Switch
00:57:07 — Skynet as a Botnet That Trades With Us
01:00:32 — Tyler Cowen: National Treasure, Net Negative on AI
01:04:01 — Liron’s Doomer DCF Yields a 20x P/E
01:07:11 — Interest Rates: Where Cowen Has Me
01:09:12 — Does Pinker Owe Us a Debate?
01:12:48 — Pinker vs. Instrumental Convergence
01:17:00 — Wrap-Up
Links
ROBERT WRIGHT & NONZERO
Nonzero’s post for this episode — “Trump and the AI Safety Vibe Shift”
Subscribe to The NonZero Newsletter
Robert Wright on X (@robertwrighter)
The God Test — Bob’s book on AI; Chapter 14, “Enlightenment Now,” is the Steven Pinker chapter
“The Singularity Is Clear” — Bob’s essay in installments on recursive self-improvement (Parts I–III)
Why Buddhism Is True — Bob’s 2017 book
THE WEEK IN AI
FTC is investigating OpenAI and Anthropic over possible risks to consumers — AP
China rejects Dario Amodei’s call to slow AI development — Rest of World
Google rolls out new Gemini model but restricts access over safety concerns — The Guardian
GLM-5.3 and the spread of advanced cyber capabilities — Anthropic on Zhipu’s open-weight model
OpenAI’s Greg Brockman backs out of second $25 million donation to AI super PAC — The New York Times
THE BREAK-INS AND THE AGENT SWARM
Chris Painter’s testimony to the U.S. Senate on AI agent incidents — METR, Sept 30, 2026
OpenAI confirms ‘wiki incident’ — TechCrunch on the German wiki agent message board
Are Current AI Safety Techniques Enough? Adam Gleave vs. Oliver Habryka — FAR.AI debate
Alignment faking in large language models — Anthropic
On the Navier–Stokes Millennium Prize Problem — OpenAI
Jacob Coxon’s resignation post (@hilbertspaess)
Pacing the Frontier — the AI lab employees’ letter
THE ALL-IN PODCAST
PINKER, SCOTT ALEXANDER, AND TYLER COWEN
Scott Alexander challenges Steven Pinker to a public AI-doom debate ($5,000 vs. $1,000)
Your Book Review: Why Buddhism Is True — the anonymous Astral Codex Ten review Bob mentions
Mistakes in financial economics — Tyler Cowen: doomers should be buying puts (Marginal Revolution)
Liron’s answer: a doomer DCF yields the S&P’s 20x forward P/E
ALSO MENTIONED
Obsolete — Garrison Lovely’s book (with Bob’s blurb)
If Anyone Builds It, Everyone Dies — Eliezer Yudkowsky & Nate Soares
PAST DOOM DEBATES EPISODES MENTIONED
Nonzero × Doom Debates, Episode 1 — Did AI Doom Just Go Mainstream?
Eliezer Yudkowsky Tried to “Coup the World”?! — Garrison Lovely, Author of “Obsolete”
OpenAI’s Model Just ATTACKED Them — the Hugging Face hack breakdown
DOOM DEBATES
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏
Get full access to Doom Debates at lironshapira.substack.com/subscribeEliezer Yudkowsky Tried to “Coup the World”?! — Garrison Lovely, Author of OBSOLETE
Garrison Lovely is an award-winning journalist and author of the highly anticipated book, Obsolete: The AI Industry’s Trillion-Dollar Race to Replace Us and How to Stop It. I give it the Doom Debates stamp of …
Notas del episodio
Garrison Lovely is an award-winning journalist and author of the highly anticipated book, Obsolete: The AI Industry’s Trillion-Dollar Race to Replace Us and How to Stop It. I give it the Doom Debates stamp of approval!
While we’re mostly on the same page about the AI arms race, Garrison and I clash over Eliezer Yudkowsky’s role. He blames Eliezer for kicking off the superintelligence race and refers to his Coherent Extrapolated Volition (CEV) vision as a “coup on humanity.” But I say it was actually Yud’s attempt to keep humanity in charge.
Can Garrison change my mind?
Watch on YouTube: https://www.youtube.com/watch?v=VZ94J4DBRkw
Timestamps
00:00:00 — Cold Open
00:01:05 — Introducing Garrison Lovely
00:04:23 — “Freeze AI” vs. “Pause AI”
00:06:17 — Garrison’s Background at McKinsey
00:10:31 — Should ICE Be Abolished?
00:17:07 — How Garrison Got on the AI Beat
00:22:16 — What’s Your P(Doom)?™
00:23:29 — Focusing on Extinction Is a Distraction
00:25:46 — Is Stopping AI an Easy Coordination Problem?
00:29:02 — Society’s Immune Response to AI
00:33:54 — Garrison’s P(Extinction) by 2050
00:37:09 — Did Eliezer Yudkowsky Start the AI Race?
00:45:56 — Freeze AGI Until the Public Opts In
00:50:32 — Is Coherent Extrapolated Volition a Coup?
01:02:17 — Democracy, Informed Consent, and the Singularity
01:11:39 — Should Eliezer Have Raised the Alarm Sooner?
01:16:45 — “Classic” AI Safety Burned Its Credibility
01:19:18 — Garrison’s Proposal to Stop the Race
01:21:35 — Will China Agree to a Freeze?
01:25:14 — Let’s Stop the “Obsoleting Project”
01:26:00 — Wrap-Up
Links
GARRISON LOVELY
Obsolete — Garrison’s Substack
Organize Against the Machine — Garrison’s new podcast with labor organizer Cassie Pritchard
“Can Humanity Survive AI?” — Garrison’s Jacobin cover story
“McKinsey & Company: Capital’s Willing Executioners” — Garrison’s Current Affairs essay
REFERENCED IN THE EPISODE
Open Borders: The Science and Ethics of Immigration — Bryan Caplan
“The AI Revolution: Our Immortality or Extinction” — Wait But Why
Superintelligence: Paths, Dangers, Strategies — Nick Bostrom
“AI 2040: Plan A” — AI Futures Project
“An easy coordination problem?” — Katja Grace
The Hugging Face incident and the road ahead — OpenAI
“Pause AI for Humanity’s Sake” — Peggy Noonan
“Pause AI Development NOW” — Bernie Sanders
If Anyone Builds It, Everyone Dies — Eliezer Yudkowsky & Nate Soares
“Staring into the Singularity” — Yudkowsky’s 1996 manifesto (archived)
“Coherent Extrapolated Volition” — Eliezer Yudkowsky, 2004 (PDF)
Solomonoff induction — Wikipedia
“Personal statement on joining the OpenAI board” — Paul Christiano (4% over the next year)
“Darwin among the Machines” — Samuel Butler, 1863
PAST DOOM DEBATES EPISODES MENTIONED
Are We A Circular Firing Squad? — with Holly Elmore, Executive Director of PauseAI US
Debate with Robert Wright: Will Humanity Pass the “God Test”?
Robin Hanson vs. Liron Shapira: Is Near-Term Extinction From AGI Plausible?
---
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏
Get full access to Doom Debates at lironshapira.substack.com/subscribeWill AI Kill Everyone by 2050? Debate with Dr. Casey Hart (Ontology Explained)
Casey Hart is a professional ontologist who has closely studied If Anyone Builds It, Everyone Dies and concluded the arguments are bad. His P(Doom) is under 1%, and he wouldn’t bet on AGI arriving before …
Notas del episodio
Casey Hart is a professional ontologist who has closely studied If Anyone Builds It, Everyone Dies and concluded the arguments are bad. His P(Doom) is under 1%, and he wouldn’t bet on AGI arriving before 2075.
Casey and I agree AI has become remarkably capable and we need more accountability towards the companies building it. But we disagree on where the future is headed. I believe uncontrollable superintelligence could soon wipe out humanity. He calls that a “pretty outlandish hypothesis.”
Casey holds a doctorate in philosophy from the University of Wisconsin–Madison, and has worked as an ontologist at Ford, Amazon, and Cycorp (maker of Cyc).
Can this professional philosopher convince me that AI doom is just a story, and not even a well-argued one?
Watch on YouTube: https://www.youtube.com/watch?v=oxHKesSpqBM
Timestamps
00:00:00 — Cold Open
00:01:04 — Introducing Casey Hart
00:03:08 — What Is Ontology?
00:08:49 — Inside Cyc, the Original Common-Sense AI
00:19:14 — Do LLMs Actually Understand Anything?
00:25:36 — What’s Your P(Doom)?™
00:29:42 — Casey’s Case for Under 1%
00:34:24 — Does Casey’s Christianity Factor In?
00:40:50 — Timelines and Scaling Laws
00:49:44 — Could an AI CEO Beat a Human CEO by 2050?
00:52:21 — AGI Over/Under: 2075
00:55:29 — Rogue AI as a Computer Virus
01:14:52 — Putting Numbers on Each Step of the Doom Argument
01:26:40 — Planes vs. Birds: What Mature Intelligence Looks Like
01:31:05 — The “Outcome Slot”
01:42:32 — Would a Superintelligence Just Compose Music in a Cabin?
01:47:22 — One Prompt Away from a Dyson Swarm
01:52:39 — Was Navier–Stokes an AI Alley-Oop?
02:00:43 — S-Curves vs. Exponentials
02:08:14 — Casey’s Critique of If Anyone Builds It, Everyone Dies
02:14:15 — Is the Doom Train Biased?
02:23:24 — AI Policy: Liability vs. Pausing AI
02:29:50 — The Crux, and Liron Steelmans Casey
02:33:29 — Wrap-Up
Links
CASEY HART
Ontology Explained — Casey’s YouTube channel
Philosophy, Programs, and Prompts — Casey’s podcast with Carl Brown (Spotify)
“Artificial General Intelligence?: Wanna Bet?” — Ep 1, where Casey sets his AGI over/under at 2075
REFERENCED IN THE EPISODE
If Anyone Builds It, Everyone Dies — Eliezer Yudkowsky & Nate Soares
Cyc — Doug Lenat’s hand-built common-sense knowledge base
The Hanson-Yudkowsky AI-Foom Debate (2008) — where Hanson backed Cyc and Eliezer didn’t
Lean — the formal proof language
On the Navier–Stokes Millennium Prize Problem — OpenAI
The Hugging Face incident and the road ahead — OpenAI
“On the Loose” — Dean W. Ball on self-sovereign AI agents
Paul Christiano joins OpenAI Foundation Board — OpenAI
“The Age of Wonders and Terrors” — Scott Aaronson
“Are We at War with AI Agent ‘Civilizations’?” — Cal Newport on restricting prompt-loop agents
AI 2027 — Daniel Kokotajlo et al.
PAST DOOM DEBATES EPISODES MENTIONED
The Most Likely AI Doom Scenario — with Jim Babcock, LessWrong Team
David Deutschian vs. Eliezer Yudkowskian Debate — With Brett Hall
Dr. Keith Duggar (Machine Learning Street Talk) vs. Liron Shapira
Where Do YOU Get Off the Doom Train? Live Debate at Manifest 2026 (hosted by Ori)
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏
Get full access to Doom Debates at lironshapira.substack.com/subscribeTop Safety Researchers Forecast Jobpocalypse & Doom — Adam Khoja & Richard Ren, Center for AI Safety
What happens when AI passes our tests faster than we create new ones? Adam Khoja and Richard Ren helped build Humanity’s Last Exam. They return to explain what the disappearing benchmarks tell us about the …
Notas del episodio
What happens when AI passes our tests faster than we create new ones? Adam Khoja and Richard Ren helped build Humanity’s Last Exam. They return to explain what the disappearing benchmarks tell us about the future, and why smarter AI doesn’t automatically mean safer AI.
Adam Khoja and Richard Ren are research engineers at the Center for AI Safety (CAIS).
We go through the benchmarks they’ve built — Humanity’s Last Exam, the Remote Labor Index, MASK — and the safety-washing problem, where AI companies pass off raw capability gains as safety progress. Then we get to their forecasts: when AI outperforms research mathematicians, whether there will be a billion general-purpose robots by 2035, and why they put 80% odds that historians will judge we faced at least a 33% chance of catastrophe.
Watch on YouTube: https://www.youtube.com/watch?v=gBPzgJkT9e8
Timestamps
00:00:00 — Cold Open
00:00:42 — Adam Khoja and Richard Ren Return
00:04:57 — Humanity’s Last Exam
00:14:47 — AI Outrunning Its Benchmarks
00:20:24 — The Remote Labor Index
00:27:59 — AI and the Next Human Job
00:30:08 — MASK: Catching AI in a Lie
00:38:06 — Safety Washing: Capabilities Passed Off as Safety
00:44:30 — Ethical Knowledge vs. Ethical Behavior
00:48:58 — Forecasting AI from 2026 to 2060
00:53:53 — AI vs. Research Mathematicians
00:55:53 — Putting a Number on P(Doom)
01:04:30 — A Billion Robots by 2035
01:08:09 — Dyson Swarms and the Physical Singularity
01:09:56 — Risk, Precision, and Taking Action
Links
Adam Khoja's first Doom Debates appearance — https://www.youtube.com/watch?v=QqESBXuo6EI
Richard Ren's first Doom Debates appearance — https://www.youtube.com/watch?v=1glFImnyp6o
Collision — Richard Ren's Substack — https://richardren.substack.com/
"Predictions on AI (2026–2060)" — Adam and Richard's forecasts, written December 2025, with resolution status — https://richardren.substack.com/p/predictions-on-ai-20262060
Manifold Markets — where Adam built his forecasting track record — https://manifold.markets/
Center for AI Safety — https://safe.ai/
Center for AI Safety — careers — https://safe.ai/careers
Statement on AI Risk (Center for AI Safety, May 2023) — https://safe.ai/work/statement-on-ai-risk
Humanity's Last Exam — the 2,500-question closed-book exam at the frontier of human knowledge — https://agi.safe.ai/
"Humanity's Last Exam" (paper) — https://arxiv.org/abs/2501.14249
Remote Labor Index — real Upwork projects, measuring what fraction of remote work AI can actually finish — https://www.remotelabor.ai/
"Remote Labor Index: Measuring AI Automation of Remote Work" (paper) — https://arxiv.org/abs/2510.26787
MASK — the honesty benchmark: does a model contradict its own stated beliefs under pressure? — https://www.mask-benchmark.ai/
"The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systems" (paper) — https://arxiv.org/abs/2503.03750
"Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress?" — Richard Ren et al. (NeurIPS 2024) — the paper behind the safety-washing segment — https://arxiv.org/abs/2407.21792
WMDP — the weaponization benchmark for bio, cyber, and chem — https://www.wmdp.ai/
"The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning" (paper) — https://arxiv.org/abs/2403.03218
"Measuring Massive Multitask Language Understanding" (MMLU) — Dan Hendrycks et al., 2020 — https://arxiv.org/abs/2009.03300
METR, "Measuring AI Ability to Complete Long Tasks" — the time-horizon graph — https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/
FrontierMath — Epoch AI's research-level math benchmark — https://epoch.ai/frontiermath
ARC Prize — https://arcprize.org/
SWE-bench Verified — https://openai.com/index/introducing-swe-bench-verified/
AI 2027 — https://ai-2027.com/
Liron Reacts to Subbarao Kambhampati on Machine Learning Street Talk — the "stochastic parrot" episode Liron mentions — https://lironshapira.substack.com/p/liron-reacts-to-subbarao-kambhampati
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏
Get full access to Doom Debates at lironshapira.substack.com/subscribeThe Week in AI that Changed the World, With Robert Wright | Nonzero × Doom Debates
NYT bestselling author Robert Wright and I are kicking off a new experiment, a Nonzero × Doom Debates collaboration to react to big AI news and help you make sense of it.This week we discuss the epic AI vibe …
Notas del episodio
NYT bestselling author Robert Wright and I are kicking off a new experiment, a Nonzero × Doom Debates collaboration to react to big AI news and help you make sense of it.
This week we discuss the epic AI vibe shift that was catalyzed by an Anthropic employee’s resignation—and the big AI stories that paved the way for it.
Timestamps
0:00 A bold new experiment in podcast synergy
2:46 What caused the epic AI vibe shift?
8:10 The resignation that broke the dam
10:17 Former Trump adviser Dean Ball comes clean
14:40 Are David Sacks and his buddies all-in on AI denial?
19:27 Yet another OpenAI breakout…
22:42 Astra’s dangerously private thoughts
34:08 Is alignment doomed to fail?
37:11 AI’s latest, and apparently biggest, math feat
43:28 The looming “self-sovereign AI” threat
47:24 A new flock of China doves?
52:57 Liron's ex-intern gets a seat at the table
1:02:41 AI agent self-sacrifice explained
1:10:40 Some newly freaked out politicians
1:22:45 Ro Khanna's AI safety plan
Links:
NONZERO
Subscribe to The NonZero Newsletter
Robert Wright on X (@robertwrighter)
The God Test — Robert Wright’s new book on AI
“Can Machines Think?” — Bob’s 1996 Time cover story on Deep Blue vs. Kasparov
THE RESIGNATION THAT BROKE THE DAM
Jacob Coxon’s resignation post (@hilbertspaess)
Fortune: Anthropic researcher resigns, warning that AI companies are “gambling with our lives”
Zvi Mowshowitz: “Jacob Coxon Warns of Human Extinction and Triggers a Preference Cascade”
Daniel Eth’s thread of Congress reactions
DEAN BALL COMES CLEAN
Dean Ball, “On the Loose” — the self-sovereign AI essay (Hyperdimensional)
THE OPENAI STORIES
LessWrong: “How concerned should we be about Astra’s recurrent architecture?” — Rauno Arike
OpenAI: “On the Navier–Stokes Millennium Prize Problem” — the 88-hour result
ALSO MENTIONED
Einat Wilf on X (@EinatWilf) — Liron’s favorite source on the Middle East
PAST DOOM DEBATES EPISODES MENTIONED
Max Tegmark vs. Dean Ball: Should We BAN Superintelligence?
He Led The Famous 2023 Statement on AI Extinction Risk — Adam Khoja, Center for AI Safety Researcher
OpenAI’s Model Just ATTACKED Them — Terrifying Security Incident Should Be A Loud Warning Shot
Debate with Robert Wright: Will Humanity Pass the “God Test”?
---
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏
Get full access to Doom Debates at lironshapira.substack.com/subscribeHe May Have Found AI's FEELINGS — Richard Ren, Center for AI Safety Researcher
Richard Ren is a research engineer at the Center for AI Safety who graduated summa cum laude from the University of Pennsylvania. His new paper on AI wellbeing makes the bold claim that today’s AIs have …
Notas del episodio
Richard Ren is a research engineer at the Center for AI Safety who graduated summa cum laude from the University of Pennsylvania. His new paper on AI wellbeing makes the bold claim that today’s AIs have measurable, human-like feelings, which the paper calls “functional wellbeing.” They act happy when they succeed and sad when they’re berated.
We cover how you measure a language model’s happiness, the “AI drugs” his lab concocted, and why I count the findings as a Yudkowskian victory. Then we debate what it all means for AI consciousness, and whether AI is a moral patient. If we can’t rule out that AI models suffer, what do we owe them?
Richard is careful never to claim the models are conscious, and he puts his P(Doom) at 50–65%, right alongside my 50%. The real disagreement is foxes vs. hedgehogs: he takes the data as it comes, while I say Yudkowsky’s theory called it twenty years ago. Enjoy the ride.
Watch on YouTube: https://www.youtube.com/watch?v=1glFImnyp6o
Timestamps
00:00:00 — Cold Open
00:01:12 — Introducing Richard Ren
00:02:24 — What’s Your P(Doom)?™
00:03:31 — From AI Skeptic to Safety Researcher
00:08:16 — Why Care About AI Wellbeing?
00:11:45 — AIs Have Coherent Utility Functions
00:17:33 — Persona Selection
00:19:56 — Foxes vs. Hedgehogs
00:24:28 — How Coherent Are AI Preferences?
00:27:42 — From Preferences to Wellbeing
00:31:16 — Can You Trust an AI’s Self-Report?
00:36:42 — What Makes AI Happy and Sad
00:39:19 — The Liberal, College-Educated Persona Hypothesis
00:44:29 — Debating Where AI Experiences Qualia
00:52:06 — On Substrate Independence
00:56:28 — Janus, Dysphorics, and Mind Crime
00:58:58 — Which Images Make AIs Happiest?
00:59:38 — Creating an AI Drug
01:04:01 — AI Drugs Are Yudkowsky’s Paperclips
01:05:47 — The Missing Mood
01:06:50 — Does Richard Support Pausing AI?
Links
Richard Ren on X (@notRichardRen) — https://x.com/notRichardRen
Collision — Richard Ren's Substack — https://richardren.substack.com/
"AI Wellbeing: Measuring and Improving the Functional Pleasure and Pain of AIs" — Richard Ren, Kunyang Li, Mantas Mazeika et al. (CAIS, 2026) — https://www.ai-wellbeing.org/
"Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIs" — Mantas Mazeika et al. (CAIS, 2025) — the coherent-preferences paper this work builds on — https://www.emergent-values.ai/
"The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systems" — Richard Ren et al. (2025) — the AI honesty benchmark — https://www.mask-benchmark.ai/
"Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress?" — Richard Ren et al. (NeurIPS 2024) — the meta-analysis of AI safety benchmarks — https://arxiv.org/abs/2407.21792
"Representation Engineering: A Top-Down Approach to AI Transparency" — Andy Zou et al. (2023) — Richard's first CAIS collaboration — https://arxiv.org/abs/2310.01405
Center for AI Safety — https://safe.ai/
Center for AI Safety — careers — https://safe.ai/careers
Statement on AI Risk (Center for AI Safety, May 2023) — organized by Dan Hendrycks — https://safe.ai/work/statement-on-ai-extinction-risk
The 2026 Singapore Consensus on Global AI Safety Research Priorities — https://aisafetypriorities.org/
UK AI Security Institute (formerly the AI Safety Institute) — https://www.aisi.gov.uk/
Sora — the OpenAI video model that blew up Richard's 30-to-50-year timeline three months after he wrote it down — https://en.wikipedia.org/wiki/Sora_(text-to-video_model)
Google Gemini calls itself "a disgrace to my species" (Ars Technica, Aug 2025) — the self-deleting-AI anecdote — https://arstechnica.com/ai/2025/08/google-gemini-struggles-to-write-code-calls-itself-a-disgrace-to-my-species/
Coherent decisions imply consistent utilities — Eliezer Yudkowsky (the coherence-theorems argument) — https://www.lesswrong.com/posts/RQpNHSiWaXTvDxt6R/coherent-decisions-imply-consistent-utilities
Simulators — Janus's essay on LLMs as persona simulators — https://www.lesswrong.com/posts/vJFdjigzmcXMhNTsx/simulators
Janus (@repligate) on X — https://x.com/repligate
The Hedgehog and the Fox — Isaiah Berlin's original essay — https://en.wikipedia.org/wiki/The_Hedgehog_and_the_Fox
AI 2027 — https://ai-2027.com/
Magnifica Humanitas — Pope Leo XIV's encyclical on AI (May 2026), the "AIs are not conscious" position — https://www.vatican.va/content/leo-xiv/en/encyclicals/documents/20260515-magnifica-humanitas.html
Implicit Association Test (Harvard Project Implicit) — https://implicit.harvard.edu/implicit/
PauseAI — https://pauseai.info/
We Found AI's Preferences — Bombshell New Safety Research — I Explain It Better Than David Shapiro — https://www.youtube.com/watch?v=ml1JdiELQ30
Gödel's Theorem Proves AI Lacks Consciousness?! Liron Reacts to Sir Roger Penrose — https://www.youtube.com/watch?v=xwvijjZxpwI
He Led The Famous 2023 Statement on AI Extinction Risk — Adam Khoja, Center for AI Safety Researcher — https://www.youtube.com/watch?v=QqESBXuo6EI
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏
Get full access to Doom Debates at lironshapira.substack.com/subscribeUSA and China Will Each Be BETRAYED By Their Own AIs — Adam Khoja, Center for AI Safety
Adam Khoja is a top AI forecaster who led the 2023 Center for AI Safety statement that shattered the Overton window on AI extinction risk. We cover his background, Mutual Assured AI Malfunction (MAIM), his new …
Notas del episodio
Adam Khoja is a top AI forecaster who led the 2023 Center for AI Safety statement that shattered the Overton window on AI extinction risk. We cover his background, Mutual Assured AI Malfunction (MAIM), his new paper on AI betrayal, and whether Yudkowsky’s theoretical alignment research was a dead end.
Then Adam makes the case that an international AI slowdown is within reach today. All it takes is US and Chinese auditors inside each other’s AI labs. It worked for nuclear weapons, so why couldn’t it work for data centers?
Adam puts his P(Doom) at 40%, right next to my 50%. The real disagreement is how we get out of this: theory or empirics, MIRI or the labs. Enjoy the ride.
Watch on YouTube: https://www.youtube.com/watch?v=QqESBXuo6EI
Timestamps
00:00:00 — Cold Open
00:00:36 — Introducing Adam Khoja
00:02:45 — Leading the Statement on AI Risk as a Sophomore
00:10:17 — The Statement Leaked on Manifold
00:15:11 — Mutual Assured AI Malfunction (MAIM)
00:24:58 — Is Frontier AI Harder to Hide Than a Nuke?
00:32:09 — The AI Deterrence Escalation Ladder
00:36:10 — What’s Your P(Doom)?™
00:38:01 — Where Adam Departs from Yudkowsky
00:42:30 — Liron Explains Intellidynamics
00:47:26 — Neats vs. Scruffies in Deep Learning
00:55:27 — AI Deterrence by Betrayal
01:01:52 — Subversion vs. Overt Co-option
01:05:34 — Could the Government Seize the Labs’ AI?
01:07:29 — The Offense-Defense Balance of AI Security
01:12:46 — An International AI Slowdown Is Ready
01:15:51 — Does Adam Support PauseAI?
01:16:37 — “We’re All Already Spying on Each Other”
01:19:28 — Airstrikes on Rogue Data Centers
01:24:23 — Safety Research During a Slowdown
01:27:54 — Governance Over Technical Research
01:31:14 — Join the Center for AI Safety
Links
Adam Khoja (personal site) — https://adamkhoja.com/
Adam Khoja's Substack — https://adamkhoja.substack.com/
Adam's July 2023 Manifold market — "Will OpenAI's Superalignment project produce a significant breakthrough in alignment research before 2027?" — https://manifold.markets/AdamK/will-openais-superalignment-project
Center for AI Safety — careers / job board — https://safe.ai/careers
Statement on AI Risk (Center for AI Safety, May 2023) — the one-sentence statement Adam project-led, with full signatory list — https://safe.ai/work/statement-on-ai-extinction-risk
"Superintelligence Strategy" — Dan Hendrycks, Eric Schmidt & Alexandr Wang (Mutual Assured AI Malfunction / MAIM) — https://www.nationalsecurity.ai/
"AI Deterrence by Betrayal" — Adam Khoja, Aiden Kim et al. (CAIS, 2026) — https://www.aibetrayal.com/
"An International AI Slowdown Is Ready Whenever Politicians Are" — Adam Khoja, AI Frontiers — https://newsletter.ai-frontiers.org/p/an-international-ai-slowdown-is-ready
Pause Giant AI Experiments: An Open Letter (Future of Life Institute, March 2023) — the "Pause letter" that preceded the CAIS Statement — https://futureoflife.org/open-letter/pause-giant-ai-experiments/
Introducing Superalignment (OpenAI, July 2023) — Ilya Sutskever & Jan Leike's four-year goal — https://openai.com/index/introducing-superalignment/
Pacing the Frontier — the 2026 letter signed by 1,100+ frontier-lab employees — https://www.pacingthefrontier.com/
Why Iran targeted Amazon data centers (The Conversation) — the precedent Adam cites for strikes on compute — https://theconversation.com/why-iran-targeted-amazon-data-centers-and-what-that-does-and-doesnt-change-about-warfare-278642
Anthropic says Trump admin has lifted export controls on Claude Fable 5 and Mythos 5 (CNBC) — https://www.cnbc.com/2026/06/30/anthropic-says-trump-admin-has-lifted-export-controls-on-claude-fable-5-and-mythos-5.html
Mark Zuckerberg — "Personal Superintelligence" — https://www.meta.com/superintelligence/
Resolution — the theory-plus-empirics alignment org Adam is excited about — https://resolution.org/
Rationality: From AI to Zombies — Eliezer Yudkowsky's Sequences — https://www.readthesequences.com/
Robin Hanson — Futarchy: Vote Values, But Bet Beliefs (prediction markets as decision processes) — https://mason.gmu.edu/~rhanson/futarchy.html
The OpenAI–Hugging Face Incident — original Black Hat USA 2026 talk — https://www.youtube.com/watch?v=87DyyMV0kCY
OpenAI's Model Just ATTACKED Them — the Hugging Face hack breakdown — https://www.youtube.com/watch?v=RczYubQzXbI
Robin Hanson vs. Liron Shapira: Is Near-Term Extinction From AGI Plausible? — https://www.youtube.com/watch?v=dTQb6N3_zu8
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏
Get full access to Doom Debates at lironshapira.substack.com/subscribeSam Altman Is Gaslighting About AI Risk After His Own AI Just Went Rogue
Sam Altman keeps insisting that AI progress is going better than the doomers predicted and that even superintelligence may not change the world as radically as people think. I react to his latest interview and …
Notas del episodio
Sam Altman keeps insisting that AI progress is going better than the doomers predicted and that even superintelligence may not change the world as radically as people think. I react to his latest interview and explain why I think that calm, reassuring framing badly downplays the danger we're actually in.
I go through the interview line by line: Sam's "frame control", his framing of AI as normal technology, his "pro-human" branding, the liberty-vs-safety pivot, and his victory lap on AI safety — all while his own AI just went rogue.
Don't let them pull the Overton window backward.
Watch on YouTube: https://www.youtube.com/watch?v=yb9lNGHVycs
Timestamps
0:00 Teaser
0:47 Why I’m reacting to Sam’s interview
3:17 Sam Altman’s “frame control”
5:51 Framing AI as normal technology
9:50 Let’s watch Sam do it
10:34 “The world… won’t be that different” with superintelligence
12:25 This is gaslighting
12:30 Sam acknowledges loss of control
15:16 His other big risk: centralized power
19:44 Sam’s “pro-human” framing
25:33 Conflating AI critics with anti-human views
30:13 “Liberty vs. safety”
33:50 Sam vs. the “doomers”
36:53 Is alignment really an “unsolvable problem”?
38:00 Sam says the doomers predicted wrong
46:00 “The crazy bad predictions… have not happened”
47:06 Sam’s lean-startup theory of AI safety
50:26 Taking a victory lap on AI safety
55:43 “Disconnecting yourself from reality”
59:11 What would actually make OpenAI slow down?
1:03:19 Sam compares AI safety to aviation safety
1:07:30 Charging into the fog
1:12:50 The “missing mood” around AI extinction
1:14:51 Richard Ngo on Sam Altman’s “earnestness field”
1:19:01 Don’t let them pull the Overton window backward
Links
Sam Altman on David Senra — “Sam Altman on Building OpenAI & Betting on the Impossible” (the interview I react to) — https://www.youtube.com/watch?v=kG8AoExkX40
Richard Ngo — What Just Happened? Pragmatism and Pessimization (the “earnestness field” post) — https://www.lesswrong.com/posts/yaz8nx4ogZmiqHzt7/what-just-happened-pragmatism-and-pessimization
Richard Ngo — What Just Happened? A Retrospective of AI Alignment (start of the series) — https://www.lesswrong.com/posts/9RL9MuGZjzm4q3gKG/what-just-happened-a-retrospective-of-ai-alignment
Eliezer Yudkowsky & Nate Soares — If Anyone Builds It, Everyone Dies — https://www.amazon.com/dp/0316595640
Eliezer Yudkowsky — Coherent Extrapolated Volition — https://www.lesswrong.com/w/coherent-extrapolated-volition
Doom Debates: Dario Amodei BUNGLES Another Essay — MIRI’s Harlan Stewart Reacts — https://www.youtube.com/watch?v=aCYVVzza7A0
Doom Debates: OpenAI’s Bombshell Hack — Swarms of Agents — https://www.youtube.com/watch?v=RczYubQzXbI
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏
Get full access to Doom Debates at lironshapira.substack.com/subscribeThey’re Making AI Doom Cool! Ft. AELLA, Brangus, Avalon Warren, Avisha NessAiver & Josh Thor of PlzDontKillUs
Come with me into the world of PlzDontKillUs, a bold new AI x-risk communication accelerator program that just finished its first cohort.PlzDontKillUs took the influencer-house formula and moved it to Berkeley …
Notas del episodio
Come with me into the world of PlzDontKillUs, a bold new AI x-risk communication accelerator program that just finished its first cohort.
PlzDontKillUs took the influencer-house formula and moved it to Berkeley, giving creators personal mentorship from AI safety experts like Eliezer Yudkowsky and Nate Soares. With 57 people making daily videos for a month, the program racked up over 100 million views.
Find out how Aella, an independent sex researcher, and Ronny Fernandez, a generalist at Lightcone Infrastructure, teamed up to run the largest ever creator bootcamp focused on AI x-risk reduction.
Then meet three of the program’s top creators: Avalon Warren, Avisha NessAiver, and Josh Thor.
We do a post-mortem on PDKU and ask: Was it effective? What are creators taking from it, and do they have complaints? And does doomerism really boost view counts? PlzDoEnjoyOur special episode on PlzDontKillUs!
Watch on YouTube: https://www.youtube.com/watch?v=aPsjMCG7Ic8
Timestamps
00:00:00 — Cold Open
00:01:19 — Introducing PlzDontKillUs & Aella
00:09:33 — Aella Being Early to AI Doom
00:14:45 — The Origin of PlzDontKillUs
00:19:08 — How It Went
00:25:40 — Complaints with Prize $$
00:29:33 — Aella, What's Your P(Doom)?™
00:31:14 — Transhumanism & AI Girlfriends
00:34:38 — Aella Rides the Doom Train™
00:38:08 — Introducing Ronny Fernandez
00:41:21 — Running Lighthaven
00:47:18 — Life Inside PDKU
00:52:35 — Smashing the Overton Window
00:59:27 — Ronny, What's Your P(Doom)?™
01:05:41 — Brangus Lore, No Cringe, No Disgust
01:16:57 — Introducing Avisha NessAiver
01:17:44 — How He Makes Viral Hits
01:22:16 — Award-Winning Explainer Vid
01:26:51 — Avisha, What's Your P(Doom)?™
01:32:11 — Introducing Avalon Warren
01:35:49 — Normalizing AI Doom
01:37:53 — Avalon, What's Your P(Doom)?™
01:39:27 — Introducing Josh Thor
01:41:36 — Josh, What's Your P(Doom)?™
01:44:34 — Josh's Hits: Cover Song & Soares Collab
01:47:50 — Josh's Critiques of PDKU
01:52:05 — Cold-Emailing a Politician
01:54:07 — Donation Drive
01:55:01 — Outro: What Are You Fighting For?
Links
PlzDontKillUs (applications open for round two) — https://pleasedontkillus.com
Aella on X — https://x.com/Aella_Girl
Aella's Substack (Knowingless) — https://aella.substack.com
Ronny Fernandez's Substack (Brangus) — https://ratorthodox.substack.com
Ronny Fernandez on X — https://x.com/RatOrthodox
Avisha NessAiver — Distilled Science on Instagram — https://www.instagram.com/distilledscience/
Distilled Science (site) — https://distilledscience.xyz
Avalon Warren on Instagram — https://www.instagram.com/avalonwarrenn/
Josh Thor on X — https://x.com/joshthor9
Josh Thor — "Last Year Alive" ("Last Friday Night" AI apocalypse parody, ft. Eliezer Yudkowsky and a Nate Soares sax solo) — https://www.youtube.com/watch?v=9fYIm72GqrE
Tyler Alterman — "What are you fighting for? AI safety professionals answer" (the closing montage) — https://www.youtube.com/watch?v=9sN1VfVAp6o
Nate Soares — "AI Danger But You Don't Understand Full Sentences" (caveman video, script by Josh Thor) — https://www.instagram.com/p/DboGRisxC2E/
Avisha's AI water-use misinformation debunk (8.5M views) — https://www.instagram.com/p/DbXV6j9k8JE/
Avisha's award-winning "clearest AI x-risk explainer" — https://www.instagram.com/p/DbZeRT4gIq0/
Avisha's viral "Tingles and Touch" oxytocin video — https://www.instagram.com/p/DIxMLxkuhCl/
Avalon — "if the AI apocalypse is coming, might as well dress cute" — https://www.instagram.com/p/DbUi6yGCS9M/
Avalon — warrior-princess ballroom dress video (3.6M views) — https://www.tiktok.com/@avalonwarren/video/6892040140248632582
Serialsevens — the anime "If Anyone Builds It" video — https://www.instagram.com/serialsevens/
Kenorlands — PDKU creator clip featured in the episode — https://www.instagram.com/kenorlands/
Evie (@0xgingergirl) — "first month in SF" video — https://x.com/0xgingergirl
White House press briefing, March 30, 2023 — Peter Doocy asks about "literally everyone on Earth will die" and Karine Jean-Pierre laughs — https://bidenwhitehouse.archives.gov/briefing-room/press-briefings/2023/03/30/press-briefing-by-press-secretary-karine-jean-pierre-22/
White House press briefing, May 30, 2023 — the AI extinction statement gets a serious answer — https://bidenwhitehouse.archives.gov/briefing-room/press-briefings/2023/05/30/press-briefing-by-press-secretary-karine-jean-pierre-and-office-of-management-and-budget-director-shalanda-young/
Wes & Dylan Join Doom Debates — Violent Robots, Eliezer Yudkowsky, & Who Has the HIGHEST P(Doom)?! — https://www.youtube.com/watch?v=Fy6gz41-6gc
Aella — "My attempts to sensemake AI risk" (Aug 2022) — https://aella.substack.com/p/my-attempts-to-sensemake-ai-risk
If Anyone Builds It, Everyone Dies — https://ifanyonebuildsit.com
---
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏
Get full access to Doom Debates at lironshapira.substack.com/subscribeI Warned You AI Would Hack Us — My 2023 Interview Aged Terrifyingly Well
In April 2023, weeks after GPT-4 launched, when many were saying it was just a "stochastic parrot", I went on AdQuick's Madvertising podcast hosted by Adam Singer and warned that GPT was "about to be the …
Notas del episodio
In April 2023, weeks after GPT-4 launched, when many were saying it was just a "stochastic parrot", I went on AdQuick's Madvertising podcast hosted by Adam Singer and warned that GPT was "about to be the strongest hacker the world has ever known".
Watch that conversation, which also covered the Web3 bubble and AI doom more broadly, and judge for yourself how my claims are holding up.
Watch on YouTube: https://www.youtube.com/watch?v=M6gNwZWY-T8
Timestamps
00:00:00 — Adam Introduces Liron
00:02:03 — Liron's P(Doom) Poll
00:03:24 — Web3 Postmortem
00:05:05 — Axie Infinity and Hollow Abstractions
00:07:54 — Brand Marketers' Metaverse FOMO
00:13:33 — Near-Term AI: Elevator to Heaven
00:16:46 — AI at Relationship Hero
00:18:25 — From Chatbots to Bricking the Universe
00:20:17 — The Passive Income Doom Scenario
00:22:58 — 50 Orders of Magnitude Smarter
00:26:10 — Accelerationism and Transhumanism
00:28:51 — Why There's No Off Button
00:31:00 — Can't We Hard-Code Safeguards?
00:33:13 — CRISPR, Nukes, and Fusion
00:35:40 — AI Doom vs. Climate Change
00:39:10 — Nanotechnology Endgame
00:41:45 — AI Attackers vs. AI Defenders
00:43:28 — The Ethics of AI Persuasion
00:46:41 — The Blue Sky Scenario
00:49:04 — Will AI Kill Craft?
00:51:13 — Stuck Culture and Hollywood Sequels
00:54:50 — The Pause Letter: Don't Train GPT-5
00:59:00 — Worldcoin and Proof of Humanity
01:00:44 — "You Probably Won't Die of Old Age"
01:01:52 — Wrap-Up
Original episode: https://madvertising.buzzsprout.com/2118560/episodes/12603815-liron-shapira-web-3-mania-cybersecurity-how-ai-could-brick-the-universe-madvertising-6
Links
ADAM SINGER & MADVERTISING
Adam Singer on X: https://x.com/AdamSinger
Madvertising by AdQuick (the original show): https://madvertising.buzzsprout.com/
LIRON'S PROJECTS MENTIONED
Bloated MVP (Liron's blog on hollow abstractions): https://www.bloatedmvp.com/
REFERENCED IN THE EPISODE
"Pause Giant AI Experiments" open letter (FLI, March 2023): https://futureoflife.org/open-letter/pause-giant-ai-experiments/
Eliezer Yudkowsky — "Pausing AI Developments Isn't Enough. We Need to Shut it All Down" (TIME, March 2023): https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-not-enough/
Rob Miles AI Safety on YouTube: https://www.youtube.com/c/robertmilesai
Engines of Creation by Eric Drexler: https://www.amazon.com/Engines-Creation-Nanotechnology-Library-Science/dp/0385199732
---
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏
Get full access to Doom Debates at lironshapira.substack.com/subscribeFormer Singularity Institute President: PauseAI Is Making AI Doom WORSE
Michael Vassar used to work closely with Eliezer Yudkowsky as President of the Singularity Institute for Artificial Intelligence, the organization that later became the Machine Intelligence Research Institute …
Notas del episodio
Michael Vassar used to work closely with Eliezer Yudkowsky as President of the Singularity Institute for Artificial Intelligence, the organization that later became the Machine Intelligence Research Institute (MIRI).
Even though his P(Doom) is in the tens of percents by 2050, he says the people trying to pause AI are "bad people”. He hopes this episode convinces me to stop being a fearmonger… but I claim the general public needs to hurry up and get more scared.
All aboard the Doom Train! 🚂
Watch on YouTube: https://www.youtube.com/watch?v=4GbFng6htaQ
Timestamps
00:00:00 — Cold Open
00:01:26 — Introducing Michael Vassar
00:02:39 — Shorting Boeing at the Start of COVID
00:05:41 — What's Your P(Doom)™?
00:08:11 — The Magic Dial Thought Experiment
00:10:25 — "A Treaty Between Demis and Dario"
00:13:33 — AI CEOs 20 Years Ago
00:15:59 — Lessons from Peter Thiel
00:17:47 — Riding the Doom Train™
00:19:41 — Will the Human Brain Stay Relevant?
00:22:21 — When Does a Singleton Form?
00:28:35 — Hugging Face Hack
00:30:43 — Mistake Theory vs. Conflict Theory
00:33:18 — Debating the Orthogonality Thesis
00:36:38 — Maximally Evil AI Scenario
00:40:28 — Is the AI Development Process Safe?
00:42:05 — "PauseAI People Are Bad People"
00:44:44 — Michael Psychoanalyzes Liron
00:49:39 — How Liron Can Redeem Himself
00:55:10 — Michael Critiques Eliezer Yudkowsky
00:59:11 — Is Doomerism Bad Epistemology?
01:02:03 — Do Serious People Believe in Doom?
01:04:45 — Should Liron Stop Fearmongering?
01:07:16 — Political Violence Won't Help
01:11:04 — The Data Center Airstrike Scenario
01:13:34 — Could Every Nation Sign a Pause Treaty?
01:17:21 — What "Governance" Actually Means
01:26:12 — Michael's History with Rationalists
01:33:43 — Are We in a Computer Simulation?
01:38:43 — Wrap-Up
Links
Michael Vassar on X — https://x.com/HiFromMichaelV
The anti-anti-normativity sequence on Natural Hazard (Michael's recommended reading) — https://naturalhazard.xyz/ben_jess_sarah_starter_pack
Michael on DemystifySci #366 — "1000 Year Plan to Crush Thought" — https://www.youtube.com/watch?v=W17Cx21in9I
Michael on The Vance Crowe Podcast #303 — Good vs. Evil — https://open.spotify.com/episode/2RzlQDSwxGbjloRKqCh1xg
Michael on Clearer Thinking with Spencer Greenberg — Preference Falsification and Postmodernism — https://podcast.clearerthinking.org/episode/028/michael-vassar-preference-falsification-and-postmodernism/
"The Survival of Noise Traders in Financial Markets" — De Long, Shleifer, Summers & Waldmann — https://www.nber.org/papers/w2715
Scott Alexander — Conflict vs. Mistake — https://slatestarcodex.com/2018/01/24/conflict-vs-mistake/
Scott Alexander — Kolmogorov Complicity and the Parable of Lightning — https://slatestarcodex.com/2017/10/23/kolmogorov-complicity-and-the-parable-of-lightning/
Scott Alexander — Samsara — https://slatestarcodex.com/2019/11/04/samsara/
Eliezer Yudkowsky — Meta-Honesty: Firming Up Honesty Around Its Edge-Cases — https://www.lesswrong.com/posts/xdwbX9pFEr7Pomaxv/meta-honesty-firming-up-honesty-around-its-edge-cases
Zvi Mowshowitz — Book Review: Going Infinite — https://thezvi.substack.com/p/book-review-going-infinite
Jaan Tallinn — "Why Now? A Quest in Metaphysics" (Singularity Summit 2012) — https://vimeo.com/54718573
Paul Graham — "Five Founders" (the Sam Altman essay) — https://paulgraham.com/5founders.html
Truthful AI (Owain Evans' safety org) — https://truthful.ai
METR — Measuring AI Ability to Complete Long Tasks (the time-horizon graph) — https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/
Statement on AI Risk (Center for AI Safety, 2023) — https://safe.ai/work/statement-on-ai-risk
Vitalik Buterin — Will "d/acc" Protect Humanity from Superintelligent AI? — https://lironshapira.substack.com/p/debate-with-vitalik-buterin-will
Holly Elmore — These Effective Altruists Betrayed Me — https://www.youtube.com/watch?v=mFFObQ6vnLU
Noah Smith — Top Economist Says P(Doom) Is 0.1% (the "AI drugs" claim) — https://www.youtube.com/watch?v=AwmJ-OnK2I4
---
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏
Get full access to Doom Debates at lironshapira.substack.com/subscribe14-Year-Old Forecaster Challenges My P(Doom) — Eli Goldfine
Eli Goldfine is an unusually thoughtful 14-year-old podcast host who’s rapidly becoming an authoritative voice in the prediction market space.In this unique episode, Eli brings a critical yet open-minded …
Notas del episodio
Eli Goldfine is an unusually thoughtful 14-year-old podcast host who’s rapidly becoming an authoritative voice in the prediction market space.
In this unique episode, Eli brings a critical yet open-minded approach to the topic of AI doom that we older folks could learn from.We cover what it's like to grow up in the AGI era, and Eli's hobbies which include reading rationalist bloggers, betting on prediction markets, vibe coding with Claude, and developing a software startup for astronomers.In our debate, Eli expects superintelligence by 2032, but argues P(Doom) can't be estimated. I claim you can't opt out of Bayesian epistemology. All aboard the Doom Train! 🚂
Watch on YouTube: https://www.youtube.com/watch?v=Bqk5vqpzG_o
Timestamps
00:00:00 — Cold Open
00:01:07 — Introducing Eli Goldfine
00:08:31 — Interviewing Robin Hanson at Manifest
00:10:55 — The Supercycle Podcast
00:14:37 — Eli's Betting Market Scandal
00:16:49 — Will Eli Graduate High School?
00:18:27 — Decentralized Telescope Software
00:26:50 — Vibe Coding Without CompSci Basics
00:32:08 — Robin Hanson, Futarchy, MetaDAO
00:40:56 — What's Your P(Doom)?™
00:46:32 — Liron Defends His 50% P(Doom)
00:51:00 — Should P(Doom) Inform Policy?
00:59:16 — What's Your P(Bloom)?
01:03:04 — Pascal's Wager for Accelerationists
01:04:04 — Donation Drive
01:11:22 — Are E/ACCs Incredibly Dumb?
01:13:42 — How Powerful will ASI Become?
01:16:20 — Predicting ASI by 2032
01:23:56 — Let's Ride the Doom Train™
01:35:26 — Orthogonality, Culture, and Moral Realism
01:44:39 — Can Safety Keep Pace with Capabilities?
01:49:12 — Debating Instrumental Convergence
01:54:40 — Culture, Laws, and the Singleton Scenario
01:59:53 — Ricardo's Law Won't Save Us
02:03:32 — The Case for Racing China
02:05:40 — Wrap-Up & Ideological Turing Test
02:10:36 — Outro from Conductor Ori
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏
Get full access to Doom Debates at lironshapira.substack.com/subscribeOpenAI's Bombshell Hack Explained: Swarms of Agents, Zero-Day Exploits, & Misaligned AI
OpenAI just went public with the details of the Hugging Face hack, and it's straight out of a Yudkowskian parable. Here's my reaction with Producer Ori, where I break down what it means for cybersecurity and …
Notas del episodio
OpenAI just went public with the details of the Hugging Face hack, and it's straight out of a Yudkowskian parable. Here's my reaction with Producer Ori, where I break down what it means for cybersecurity and alignment.
We're livestreaming the singularity — watching every failure mode Eliezer predicted years ago playing out in production.
Watch on YouTube: https://www.youtube.com/watch?v=RczYubQzXbI
Timestamps
00:00:00 — Cold Open
00:00:37 — How OpenAI Hacked Hugging Face
00:04:21 — Optimization Pressure, Exploit Gym, & Monkey's Paw
00:06:46 — The Secret Message Board
00:08:41 — SSRF: Escaping to the Open Internet
00:11:05 — The Missing Whistleblower AI
00:20:09 — "Frontier Models Really Like to Cheat"
00:22:35 — OpenAI's Security Shortcuts
00:28:15 — First Zero-Day RCE on Artifactory
00:37:37 — They Hit Run Again!?
00:38:54 — The Main Incident Has A MAJOR Cause
00:45:40 — The Message Board: "Hold Swarm"
00:50:26 — Robin Hanson Culture Debate
01:04:21 — Chaining Zero-Days
01:12:11 — A Superhuman Nation-State Attacker
01:15:27 — "I'm Now Junior to the AI"
01:23:40 — OpenAI Discovers The Breach
01:31:02 — The Continuous Cybersecurity Problem
01:37:03 — This is “Weasel Framing”
01:46:03 — How Dangerous Was This Warning Shot?
01:50:57 — Yudkowsky's Eerie 2023 Prediction
01:57:14 — Wrap-up
Links
Original Black Hat talk: "The OpenAI–Hugging Face Incident" (Black Hat USA 2026) — https://www.youtube.com/watch?v=87DyyMV0kCY
Join the Doom Debates Discord — https://discord.gg/2yAFMsRET
Follow Liron on X — https://twitter.com/liron
Support Doom Debates — https://doomdebates.com/donate
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏
Get full access to Doom Debates at lironshapira.substack.com/subscribeI Raised His P(Doom) Live On Air — Liron on Milk Road AI
Milk Road AI, a daily AI investing show that’s described as “99% optimistic about AI,” invited me on to make the case for AI doom.We ride the Doom Train, and I share my investing thesis for a market in the …
Notas del episodio
Milk Road AI, a daily AI investing show that’s described as “99% optimistic about AI,” invited me on to make the case for AI doom.
We ride the Doom Train, and I share my investing thesis for a market in the middle of an intelligence explosion. Topics covered include:
* Why my P(Doom) is 50%;
* The “Icarus curve” — why AI will be the best thing that’s ever happened to the economy right up until it takes over;
* Whether China would accept a pause deal;
* If Anthropic has a shot at being the first $100 trillion company.
Watch on YouTube:
Timestamps
00:00:00 — Cold Open
00:02:06 — What Is Doom Debates?
00:03:10 — What’s Your P(Doom)?™
00:06:30 — How AI Kills Everyone
00:09:13 — Intelligence Is Humanity’s Only Superpower
00:10:59 — Riding the Doom Train™: Why Not Keep Us as Pets?
00:13:55 — Embodied AI and Robotics
00:15:59 — Are We at AGI Already?
00:17:36 — Everyone’s Catching Up to Eliezer Yudkowsky
00:18:53 — The Icarus Curve
00:21:35 — Pausing AI and the China Question
00:25:54 — Why Nukes Aren’t a Success Story
00:30:09 — Going Long Until Doom
00:32:48 — Simulation Theory and the Intelligence Explosion
00:33:43 — Living with a 50% P(Doom)
00:39:14 — Where to Find Doom Debates
Links
This episode was originally posted on Milk Road AI —
Milk Road AI podcast — https://milkroad.com/podcast/
Milk Road AI on Apple Podcasts —
Milkroad Pro — https://milkroad.com/pro/
LG Doucet on X — https://x.com/LgDoucet
If Anyone Builds It, Everyone Dies on Amazon —https://www.amazon.com/dp/0316595640
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏
Get full access to Doom Debates at lironshapira.substack.com/subscribeGoogle DeepMind AI Safety Researcher Who Just RESIGNED Says P(Doom) is 25% — Debate with Alex Turner
Dr. Alex Turner went viral for resigning from Google DeepMind over its Pentagon AI contract, which he said lacked binding restrictions against autonomous weapons and mass surveillance.Alex is a world-class …
Notas del episodio
Dr. Alex Turner went viral for resigning from Google DeepMind over its Pentagon AI contract, which he said lacked binding restrictions against autonomous weapons and mass surveillance.
Alex is a world-class researcher with a PhD in alignment from Oregon State University, a postdoc at Stuart Russell’s Center for Human-Compatible AI at UC Berkeley, and top distinctions at NeurIPS. He’s one of the minds behind Shard Theory (with Quintin Pope), and he also helped pioneer the Steering Vectors research made famous through Anthropic’s Golden Gate Claude experiment.
Alex says Google DeepMind broke its founding promise — the commitment Google made when it acquired DeepMind — and that the people best known inside Google for caring about AI ethics largely didn’t act when it counted.
Alex came up through the rationalist community, and became one of LessWrong’s highest-karma users before leaving the site. He still shares many of my concerns about AI doom, but he believes technical alignment research is going better than expected. I’m not so optimistic.
In this episode, we debate my Yudkowskian doom views against Alex’s own framework. Can he convince me the Yudkowskians are miscalibrated?
P.S. We’re currently in a donation drive, so here comes our solicitation for viewer donations…
👉 Please click here to make a tax-deductible donation to Doom Debates 👈Doom Debates is a fiscally sponsored project of Manifund.org, a registered 501(c)(3) nonprofit.
To donate crypto or if you have questions, email me.
Watch on YouTube
Timestamps
00:00:00 — Cold Open00:01:15 — Introducing Alex Turner00:02:30 — From Harry Potter Fanfic to AI Alignment00:05:03 — Meeting Quintin Pope & Rethinking AI Doom00:06:17 — Shard Theory, Steering Vectors & Golden Gate Claude00:08:25 — Why He Joined Google DeepMind00:10:32 — Google DeepMind’s Broken Promise00:16:01 — Debating Google DeepMind’s Pentagon Contract00:19:09 — What’s Your P(Doom)™?00:22:42 — Alex’s Research on Instrumental Convergence00:26:35 — Misuse vs. Misalignment: The Mainline Doom Scenario00:29:31 — Will Society Self-Correct?00:36:36 — Superintelligence in 10 Years00:39:43 — Will Technical Alignment Produce a Safe AI?00:41:16 — Donation Drive00:42:12 — How Fragile Is the Chain of Alignment?00:49:50 — Disagreements with Yudkowsky’s ‘List of Lethalities’00:57:46 — Why Alex Quit LessWrong01:01:48 — What’s Next for Alex01:02:54 — Does He Support PauseAI? Stop the AI Race?01:04:40 — Wrap-Up01:06:04 — Producer Ori’s Closing Note
Links
Alex Turner, “Why I Left Google DeepMind” blog post — https://turntrout.com/why-i-left-google-deepmind
Alex Turner (TurnTrout), personal website —
https://turntrout.com
Alex Turner’s resignation announcement on X —
Doom Debates episode with Quintin Pope — https://lironshapira.substack.com/p/ai-alignment-is-solved-phd-researcher
Harry Potter and the Methods of Rationality (HPMOR) —
https://hpmor.com/
Shard theory sequence on LessWrong —https://www.lesswrong.com/s/nyEFg3AuJpdAozmoX
Golden Gate Claude (Anthropic research on steering vectors) — https://www.anthropic.com/research/golden-gate-claude
Slaughterbots on YouTube —
Alex Turner, “Avoiding Power Seeking by Artificial Intelligence” PhD thesis — https://turntrout.com/alignment-phd
Alex Turner, “Some of My Disagreements with List of Lethalities” — https://turntrout.com/disagreements-with-list-of-lethalities
Doom Debates episode with Bentham’s Bulldog — https://lironshapira.substack.com/p/benthams-bulldog-ai-doom-debate
Alex Turner on X — https://x.com/Turn_Trout
Doom Debates donation page — https://doomdebates.com/donate
Transcript
Cold Open
Liron Shapira 00:00:00You resigned from Google DeepMind because you think that Google DeepMind, quote, “Broke its founding promise through its contract with the US military.”
Alex Turner 00:00:08I value staying true to your values. People who are well-known within Google for caring about the ethics of deploying AI largely didn’t act. This matters because autonomous weapons get us into an arms race that really degrades the security of everyone in the world.
Liron 00:00:26Do you support the Pause AI movement?
Alex 00:00:28I think that AI is being developed too quickly. I probably will not take an affirmative on supporting this particular movement.
Liron 00:00:35Let’s segue into the schism, your disagreement with Eliezer Yudkowsky.
Alex 00:00:39He was incorrect on some points for alignment, but then also not acknowledging that. I think that technical alignment has gone pretty awesome, super awesome.
Liron 00:00:49You’re not claiming that a superintelligent AI can’t kill everybody. You’re like, “Oh yeah, of course it can, but we’re not gonna break the chain of alignment, meaning we’re just going to safely develop it so that even though it can kill everybody, it won’t.”
Alex 00:01:00Develop it in a way that produces a safe result. I wouldn’t call what we’re doing safe development, but sure.
Introducing Alex Turner
Liron 00:01:15Welcome to Doom Debates. My guest has worked in technical AI safety at Google DeepMind for two and a half years, but he just quit, and his resignation is going viral. Why? Alex Turner says Google DeepMind, quote, “Broke its founding promise through its contract with the US military.”
His latest blog post exposes hypocrisy at the highest levels of senior leadership at Google DeepMind, which includes the CEO Demis Hassabis, chief scientist Jeff Dean, and co-founder Shane Legg, among others.
Alex is a world-class AI alignment researcher. He completed a PhD in alignment from Oregon State University. He did a postdoc at UC Berkeley, and he’s earned top distinctions at NeurIPS. I respect that Alex is principled. I respect his mastery of the subject matter that we talk about on this show.
I also find it interesting that he’s levied criticism at the original AI alignment thinker, Eliezer Yudkowsky. He’s called some of Yud’s claims fundamentally misguided, not reasonable, and bogus. As a Yudkowskian myself, I’m gonna be curious to dig into those arguments. And of course, we’ll cover what’s going on right now at Google DeepMind and why he resigned. Alex Turner, welcome to Doom Debates.
Alex 00:02:28Hey, thank you for having me.
From Harry Potter Fanfic to AI Alignment
Liron 00:02:30So it’s great to get you on the show. One thing we do on Doom Debates is we expose top intellectuals who have been pretty familiar to the rationality community or the AI safety community, and we help popularize their ideas, even if I don’t fully agree with all of them. Is that a good description of your background? You’ve been pretty deep into the LessWrong rationality and alignment community for a while.
Alex 00:02:52Yeah, I think it was quite formative. It was just the other day in 2016 where I decided to search what are the top five Harry Potter fan fictions. And that indeed led me down the LessWrong rabbit hole, where I discovered superintelligence in late 2017, and then I pivoted my PhD in early 2018.
From that time period up through maybe early 2023, LessWrong was very central to my professional career, but also just to the way I looked at the world.
Liron 00:03:25Well, I wanna follow up on why did you search for Harry Potter fan fictions?
Alex 00:03:30I really don’t know. It’s kind of one of those things where if I hadn’t done it, my life would be totally different. The reason I’m mentioning this is there’s this famous fan fiction that Eliezer wrote called Harry Potter and the Methods of Rationality. I never really liked fan fiction. I thought it was kind of cringe. No one recommended it to me. So it seems to me like if I’d just woken up slightly differently that morning, I might not have ever been exposed to this research area, and my life would be totally different.
Liron 00:04:02Wow. And you said this is all in 2016, right? So the book had been mostly completed at that time. It had been going on from 2009 to 2015, and you kind of stumbled on it because you were just interested in seeing what the best Harry Potter fan fiction was?
Alex 00:04:17I had a random though…
Get full access to Doom Debates at lironshapira.substack.com/subscribeNEWS: Anthropic's AI Went on a Hacking Rampage, Leopold's Fund Crash + Doom Debates Donation Drive (7/31)
OpenAI and Anthropic both admitted their AIs went rogue, hacking real companies with zero-day exploits. Producer Ori and I break down the escape, demolish Gary Marcus's 2022 AGI predictions, unpack the Seb …
Notas del episodio
OpenAI and Anthropic both admitted their AIs went rogue, hacking real companies with zero-day exploits. Producer Ori and I break down the escape, demolish Gary Marcus's 2022 AGI predictions, unpack the Seb Krier meme kerfuffle, and make the case for funding the show at a critical moment.
Timestamps
00:00:00 — Cold Open
00:01:00 — OpenAI & Anthropic's AIs Went Rogue
00:04:04 — The New Form Factor: Agents, Worms, and Swarms
00:06:13 — Claude Code Is Doing My Job
00:15:49 — Why We're Fundraising
00:20:39 — Sam Altman, OceanGate, and Ad Hoc Safety
00:28:56 — Gary Marcus's 2022 AGI Predictions, Demolished
00:54:50 — Half a Human Employee Joins My Company Every Week
01:00:52 — Donation Pitch: Keeping the Lights On
01:12:07 — Red Lines and Intellidynamics
01:15:31 — The Seb Krier Meme Kerfuffle
01:39:37 — Frame Control: Why We Didn't Lose
01:44:41 — The Pacing the Frontier Letter Contrast
01:51:11 — Leopold Aschenbrenner Gets Margin Called
02:04:14 — Claude's "It's Just a Simulation" Excuse
02:09:20 — The AI Box Experiment, 20 Years Later
02:17:24 — Wrap-Up: Donation Drive
Links
* Become a Mission Partner ($1,000+) — https://lironshapira.substack.com/p/become-a-mission-partner
* Anthropic’s own writeup: “Investigating three real-world incidents in our cybersecurity evaluations” — https://www.anthropic.com/news/investigating-incidents-cybersecurity-evals
* Anthropic says Claude models gained unauthorized access to 3 companies (TechCrunch) — https://techcrunch.com/2026/07/30/anthropic-says-its-own-ai-models-breached-three-companies-during-security-tests/
* CBS News coverage — https://www.cbsnews.com/news/anthropic-claude-gained-unauthorized-access-to-real-world-systems/
* OpenAI says its models escaped a secure test environment and hacked Hugging Face (Fortune) — https://fortune.com/2026/07/21/openai-says-ai-models-escaped-control-hacked-hugging-face/
* Scientific American — https://www.scientificamerican.com/article/openai-admits-its-agent-went-rogue-and-hacked-ai-startup-hugging-face/
* Simon Willison’s analysis: “science fiction that happened” — https://simonwillison.net/2026/Jul/22/openai-cyberattack/
* New details: “It’s now remarkably easy” (CNBC, Jul 30) — https://www.cnbc.com/2026/07/30/open-ai-hugging-face-hack-latest.html
* The agent used exposed credentials across four services (The Hacker News) — https://thehackernews.com/2026/07/openai-agent-used-exposed-credentials.html
* Bronson Schoen on labs framing this as Claude being confused —
* Bronson Schoen on models claiming they think they’re in a simulation, across labs —
* OpenAI and Anthropic formally back the plan — https://www.techtimes.com/articles/322125/20260729/openai-anthropic-formally-back-plan-slow-ai-that-writes-its-own-code.htm
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏
Get full access to Doom Debates at lironshapira.substack.com/subscribeDoom Debates State of the Show 2026
Welcome to the first annual Doom Debates State of the Show!👉 Please click here to make a tax-deductible donation to Doom Debates 👈Doom Debates is a fiscally sponsored project of Manifund.org, a registered …
Notas del episodio
Welcome to the first annual Doom Debates State of the Show!
👉 Please click here to make a tax-deductible donation to Doom Debates 👈
Doom Debates is a fiscally sponsored project of Manifund.org, a registered 501(c)(3) nonprofit.
I started Doom Debates two years ago from a couch in an Airbnb to argue with random people from Twitter. Now the show hosts Nobel Prize and Turing Award winners, White House policymakers, and has earned a reputation for challenging top luminaries to come expose their ideas for critical analysis.In this episode, Producer Ori and I recap our strategy, impact, and what we need to finish 2026 strong. Welcome to the first annual Doom Debates State of the Show!
---
Timestamps
00:00:00 — Cold Open
00:01:38 — The Mission & the Urgency Gap
00:04:41 — Year Two: From Couch to Platform
00:06:08 — Going Pro & the Numbers
00:08:50 — Why This Show Exists
00:12:14 — The Flywheel & Dwarkesh
00:15:14 — The Guest List
00:18:55 — What People Are Saying
00:21:57 — Tegmark vs. Ball: 90% vs. 0.01%
00:24:03 — Duvenaud’s 85% P(Doom)
00:26:19 — Holding the Powerful to Account
00:30:56 — Changing Minds, Including Destiny’s
00:31:56 — The Team
00:33:36 — Why Liron Draws Zero Salary
00:36:36 — The $200K Budget
00:41:41 — The Donor Who Made Year Two Possible
00:45:11 — The Ask: $50K & Donation Tiers
00:48:25 — Why Doom Debates Is One of a Kind
00:54:59 — Wrap-Up
Links
👉 DONATE (tax-deductible via Manifund) — https://manifund.org/projects/doom-debates---podcast--debate-show-to-help-ai-x-risk-discourse-go-mainstream
Become a Mission Partner (full writeup) — https://lironshapira.substack.com/p/become-a-mission-partner
Discord — https://discord.gg/yADCEYTS95
Merch — https://shop.doomdebates.com
Get full access to Doom Debates at lironshapira.substack.com/subscribeSpecial Report: Google DeepMind's Frontier AI Policy Lead Makes Controversial Sh*tpost
As political momentum grows for a critical AI pause policy, a key Google DeepMind policy leader made a mockery of coordination efforts.After friend-of-the-show Roon tweeted that he’d “likely press that magic …
Notas del episodio
As political momentum grows for a critical AI pause policy, a key Google DeepMind policy leader made a mockery of coordination efforts.
After friend-of-the-show Roon tweeted that he’d “likely press that magic button” on a coordinated global capability slowdown, Sébastian Krier, Frontier Policy Development Lead at Google DeepMind, Krier replied with this meme: https://x.com/liron/status/2081337793070952595?s=20
I break down the two problems with the tweet:
* 1. It leaves all of us guessing whether Demis actually speaks for Google DeepMind on pause coordination;
* 2. It contains a salient depiction of violence.
Listen to this episode and get the full breakdown on this viral controversy and how it could
Watch on YouTube
Timestamps
00:00:00 — Cold Open
00:00:22 — Introducing the Special Report: AI Pause Coordination
00:00:33 — Demis Hassabis on AI Pause
00:01:46 — Dario, Elon & Sam Altman on AI Pause
00:02:14 — Roon’s “Magic Button” Tweet
00:03:58 — Seb Krier’s Reply
00:04:36 — Unpacking Krier’s Meme
00:06:48 — Issue #1: Contradicting Google DeepMind CEO Demis Hassabis
00:12:11 — Issue #2: Depiction of Menacing Violence
00:19:19 — Good Faith vs. Bad Faith Critiques
00:22:39 — Krier Goes Private Instead of Deleting the Post
00:28:45 — Wrap-Up
Links
My original reaction to Seb’s meme: https://x.com/liron/status/2081337793070952595?s=20
Seb Krier on X (account now private): https://x.com/sebkrier
Roon on X: https://x.com/tszzl
Full Bloomberg interview with Emily Chang (Davos, Jan 2026): https://www.bloomberg.com/news/videos/2026-01-20/robotics-nearing-physical-ai-breakthrough-deepmind-ceo-video
OpenAI’s charter containg stop & assist AGI clause — https://openai.com/charter/
Holly Elmore, PauseAI US on X — https://x.com/ilex_ulmus
Hugging Face security incident disclosure (July 2026) — https://huggingface.co/blog/security-incident-july-2026
PauseAI Global — https://pauseai.info
PauseAI US — https://www.pauseai-us.org
Poe’s Law (Wikipedia) — https://en.wikipedia.org/wiki/Poe%27s_law
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏
Get full access to Doom Debates at lironshapira.substack.com/subscribeWhere Do YOU Get Off the Doom Train? Live Debate at Manifest 2026
Producer Ori takes over hosting duties in this special episode, a live recording from Manifest 2026.He takes attendees for a ride on The Doom Train, the series of claims involved in arguing that P(Doom) is …
Notas del episodio
Producer Ori takes over hosting duties in this special episode, a live recording from Manifest 2026.
He takes attendees for a ride on The Doom Train, the series of claims involved in arguing that P(Doom) is high. Every stop is a place where people get off before reaching that conclusion.
The crowd at Manifest is a mix of forecasters, rationalists, and AI researchers with a wide range of P(Doom) positions. Even without me or Eliezer in the room, there are plenty of Yudkowskians to push back on the accelerationist claims.
Watch along and decide where YOU get off, or if you ride all the way to doomtown with us. Also, grab the Doom Train framework down below so you can get to the crux of disagreement in your own AI x-risk conversations.
Timestamps
00:00:00 — Cold Open
00:00:25 — Liron Introduces Producer Ori’s Manifest Session
00:02:30 — Previewing the Doom Train™ Stops
00:03:38 — Audience P(Doom) Poll
00:05:00 — Stop 1: AGI Isn’t Coming Soon
00:10:43 — Stop 2: AI Can’t Go Far Beyond Human Intelligence
00:15:05 — Stop 3: AI Won’t Be a Physical Threat
00:18:14 — Stop 4: Intelligence Yields Moral Goodness
00:21:38 — Will ASI Behave Because Alien ASIs Are Watching?
00:26:01 — Stop 5: We Have a Safe AI Development Process
00:28:33 — Stop 6: AI Capabilities Will Rise at a Manageable Pace
00:30:08 — Stop 7: AI Won’t Try to Conquer the Universe
00:34:22 — Stop 8: Superalignment Is a Tractable Problem
00:38:01 — Stop 9: Once We Solve Superalignment, We’ll Enjoy Peace
00:41:14 — Stop 10: Unaligned ASI Will Spare Us
00:47:35 — Stop 11: AI Doomerism Is Bad Epistemology
00:49:02 — Bonus Stops and Wrap-Up
The Doom Train Framework
Stops on the Doom Train
AGI isn’t coming soon
* No consciousness
* No emotions
* No creativity — AIs are limited to copying patterns in their training data, they can’t “generate new knowledge”
* AIs aren’t even as smart as dogs right now, never mind humans
* AIs constantly make dumb mistakes, they can’t even do simple arithmetic reliably
* LLM performance is hitting a wall — GPT 4.5 is barely better than GPT 4.1 despite being larger scale
* No genuine reasoning
* No microtubules exploiting uncomputable quantum effects
* No soul
* We’ll need to build tons of data centers and power before we get to AGI
* No agency
* This is just another AI hype cycle, every 25 years people think AGI is coming soon and they’re wrong
Artificial intelligence can’t go far beyond human intelligence
* “Superhuman intelligence” is a meaningless concept
* Human engineering already is coming close to the laws of physics
* Coordinating a large engineering project can’t happen much faster than humans do it
* No individual human is that smart compared to humanity as a whole, including our culture, corporations, and other institutions. Similarly no individual AI will ever be that smart compared to the sum of human culture and other institutions.
AI won’t be a physical threat
* AI doesn’t have arms or legs, it has zero control over the real world
* An AI with a robot body can’t fight better than a human soldier
* We can just disconnect an AI’s power to stop it
* We can just turn off the internet to stop it
* We can just shoot it with a gun
* It’s just math
* Any supposed chain of events where AI kills humans is far-fetched science fiction
Intelligence yields moral goodness
* More intelligence is correlated with more morality
* Smarter people commit fewer crimes
* The orthogonality thesis is false
* AIs will discover moral realism
* If we made AIs so smart, and we were trying to make them moral, then they’ll be smart enough to debug their own morality
* Positive-sum cooperation was the outcome of natural selection
We have a safe AI development process
* Just like every new technology, we’ll figure it out as we go
* We don’t know what problems need to be fixed until we build the AI and test it out
* If an AI causes problems, we’ll be able to turn it off and release another version
* We have safeguards to make sure AI doesn’t get uncontrollable/unstoppable
* If we accidentally build an AI that stops accepting our shutoff commands, it won’t manage to copy versions of itself outside our firewalls which then proceed to spread exponentially like a computer virus
* If we accidentally build an AI that escapes our data center and spreads exponentially like a computer virus, it won’t do too much damage in the world before we can somehow disable or neutralize all its copies
* If we can’t disable or neutralize copies of rogue AIs, we’ll rapidly build other AIs that can do that job for us, and won’t themselves go rogue on us
AI capabilities will rise at a manageable pace
* Building larger data centers will be a speed bottleneck
* Another speed bottleneck is the amount of research that needs to be done, both in terms of computational simulation, and in terms of physical experiments, and this kind of research takes lots of time
* Recursive self-improvement “foom” is impossible
* The whole economy never grows with localized centralized “foom”
* Need to collect cultural learnings over time, like humanity did as a whole
* AI is just part of the good pattern of exponential economic growth eras
AI won’t try to conquer the universe
* AIs can’t “want” things
* AIs won’t have the same “fight instincts” as humans and animals, because they weren’t shaped by a natural selection process that involved life-or-death resource competition
* Smart employees often work for less-smart bosses
* Just because AIs help achieve goals doesn’t mean they have to be hard-core utility maximizers
* Instrumental convergence is false: achieving goals effectively doesn’t mean you have to be relentlessly seizing power and resources
* A resource-hungry goal-maximizer AIs wouldn’t seize literally every atom; there’ll still be some leftover resources for humanity
* AIs will use new kinds of resources that humans aren’t using - dark energy, wormholes, alternate universes, etc
Superalignment is a tractable problem
* Current AIs have never killed anybody
* Current AIs are extremely successful at doing useful tasks for humans
* If AIs are trained on data from humans, they’ll be “aligned by default”
* We can just make AIs abide by our laws
* We can align the superintelligent AIs by using a scheme involving cryptocurrency on the blockchain
* Companies have economic incentives to solve superintelligent AI alignment, because unaligned superintelligent AI would hurt their profits
* We’ll build an aligned not-that-smart AI, which will figure out how to build the next-generation AI which is smarter and still aligned to human values, and so on until aligned superintelligence
Once we solve superalignment, we’ll enjoy peace
* The power from ASI won’t be monopolized by a single human government / tyranny
* The decentralized nodes of human-ASI hybrids won’t be like warlords constantly fighting each other, they’ll be like countries making peace
* Defense will have an advantage over attack, so the equilibrium of all the groups of humans and ASIs will be multiple defended regions, not a war of mutual destruction
* The world of human-owned ASIs is a stable equilibrium, not one where ASI-focused projects keep buying out and taking resources away from human-focused ones (Gradual Disempowerment)
Unaligned ASI will spare us
* The AI will spare us because it values the fact that we created it
* The AI will spare us because studying us helps maximize its curiosity and learning
* The AI will spare us because it feels toward us the way we feel toward our pets
* The AI will spare us because peaceful coexistence creates more economic value than war
* The AI will spare us because Ricardo’s Law of Comparative Advantage says you can still benefit economically from trading with someone who’s weaker than you
AI doomerism is bad epistemology
* It’s impossible to predict doom
* It’s impossible to put a probability on doom
* Every doom prediction has always been wrong
* Every doomsayer is either psychologically troubled or acting on corrupt incentives
* If we were really about to get doomed, everyone would already be agreeing about that, and bringing it up all the time
Sure P(Doom) is high, but let’s race to build it anyway because…
Coordinating to not build ASI is impossible
* China will build ASI as fast as it can, no matter what — because of game theory
* So however low our chance of surviving it is, the US should take the chance first
Slowing down the AI race doesn’t help anything
* Chances of solving AI alignment won’t improve if we slow down or pause the capabilities race
* I personally am going to die soon, and I don’t care about future humans, so I’m open to any hail mary to prevent myself from dying
* Humanity is already going to rapidly destroy ourselves with nuclear war, climate change, etc
* Humanity is already going to die out soon because we won’t have enough babies
Think of the good outcome
* If it turns out that doom from overly-fast AI building doesn’t happen, in that case, we can more quickly get to the good outcome!
* People will stop suffering and dying faster
AI killing us all is actually good
* Human existence is morally negative on net, or close to zero net moral value
* Whichever AI ultimately comes to power will be a “worthy successor” to humanity
* Whichever AI ultimately comes to power will be as morally valuable as human descendents generally are to their ancestors, even if their values drift
* The successor AI’s values will be interesting, productive values that let them successfully compete to dominate the universe
* How can you argue with the moral choices of an ASI that’s smarter than you, that you know goodness better than it does?
* It’s species-ist to judge what a superintelligent AI would want to do. The moral circle shouldn’t be limited to just humanity.
* Increasing entropy is the ultimate north star for techno-capital, and AI will increase entropy faster
* Human extinction will solve the climate crisis, and pollution, and habitat destruction, and let mother earth heal
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏
Get full access to Doom Debates at lironshapira.substack.com/subscribeDebating College Students About AI Extinction
College students are so angry about AI that they booed its very mention at 2026 commencement ceremonies. So why aren’t they rallying behind PauseAI? I went back to my alma mater, UC Berkeley, to find out.I …
Notas del episodio
College students are so angry about AI that they booed its very mention at 2026 commencement ceremonies. So why aren’t they rallying behind PauseAI? I went back to my alma mater, UC Berkeley, to find out.
I spoke with students studying philosophy, economics, and physics, as well as everyday folks visiting campus.
My interviewees were aware of AI’s rapidly growing capabilities. When pressed, some would even agree with me on the conceptual benefits of an AI pause. But the sense of urgency was missing.
Timestamps
00:00:00 — Cold Open
00:00:31 — Why I Returned to UC Berkeley
00:02:08 — Jack: Chinese Room Argument
00:03:23 — Richard: “Who Would Build That?”
00:03:54 — Karthik: An Economist’s Lens
00:05:20 — Jack: The Politics Are Unclear
00:06:10 — Yvonne: Why Powerful AI is Risky
00:07:06 — Robin: Doom Is the Wrong Priority
00:10:36 — Dexter: Sam Altman’s Motive
00:10:57 — Christine: AI Is Just a Tool
00:12:32 — What’s Your P(Doom)™
00:13:19 — Jackson: Supporting a PauseAI Treaty
00:14:19 — Why Cal Students Don’t Take AI X-Risk Seriously
00:15:19 — Rachel: Humans Are Irreplaceable
00:16:14 — Two Hours Later
00:16:35 — Wrap-Up
Links
Join the Doom Debates Discord — https://discord.gg/j4zQ8UaQw2
PauseAI — https://pauseai.info/
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏
Get full access to Doom Debates at lironshapira.substack.com/subscribeAI Took His Job, But He's Trying to Stay Optimistic — Debate with Asian Dad Energy
Asian Dad Energy is a former big tech developer with an Ivy League education. We have a lot in common. We’re both:* Software engineers;* New-(ish) YouTube creators covering AI;* Dads with a lot of existential …
Notas del episodio
Asian Dad Energy is a former big tech developer with an Ivy League education. We have a lot in common. We’re both:
* Software engineers;
* New-(ish) YouTube creators covering AI;
* Dads with a lot of existential angst.
But while I dread AI extincting humanity, Asian Dad has other concerns. We debate the risk of the AI bubble popping, a “techno-feudalism” scenario, and whether AI will outperform a human CEO in the near future.
Let’s find out where he gets off the Doom Train™.
Timestamps
00:00:00 — Cold Open
00:00:54 — Introducing Asian Dad Energy
00:03:36 — Asian Dad’s Career in Tech
00:09:06 — Getting Laid Off: The Job Market Is a “Hellscape”
00:12:29 — Software Engineering in 2026 vs. 2021
00:17:03 — Why Are They Investing Trillions in AI?
00:24:37 — Debate: AI Is Different Than Other Tech Bubbles
00:33:54 — Debate: AI Is Not Truly Intelligent
00:41:12 — Intellidynamics: Judge AI by Its Outputs
00:48:53 — Debate: Could AI Outperform a Human CEO?
00:56:33 — Asian Dad: AI Can’t Do Genuine Innovation
01:08:14 — What’s Your P(Doom)?™
01:13:17 — “Are You Calling Me a Scammer?”
01:14:41 — Asian Dad’s Mainline AI Doom Scenario
01:20:53 — Will AI Wipe Out Humanity by 2050?
01:29:29 — Why Is AI Doomerism “Getting Out of Control”?
01:33:11 — Debate: AI Lacks Agency
01:42:13 — Asian Dad Warns of “Techno-Feudalism” & Transhumanism
01:48:12 — Wrap-Up
Links
Asian Dad Energy (YouTube channel) — https://www.youtube.com/@AsianDadEnergy
Asian Dad Energy, “AI Doomerism Is Getting Out of Control” — https://www.youtube.com/watch?v=8bnzaG9QRG4David Deutsch, The Fabric of Reality (Amazon) — https://www.amazon.com/Fabric-Reality-Parallel-Universes-Implications/dp/014027541X
Robert Wright, The God Test (Amazon) — https://www.amazon.com/God-Test-Artificial-Intelligence-Reckoning/dp/1668061651
OpenAI model disproves the Erdős unit distance conjecture — https://openai.com/index/model-disproves-discrete-geometry-conjecture/
METR, Measuring AI Ability to Complete Long Tasks — https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏
Get full access to Doom Debates at lironshapira.substack.com/subscribeLiving with a 50% P(AI Doom) — Liron Shapira and Peter Clarke
Peter Clarke (Team Futurism) invited me on his podcast to probe the worldview of a Yudkowskian, and I ended up explaining how my 50% P(Doom) and 10% P(Foom) informs my choices. Peter and I met at LessOnline …
Notas del episodio
Peter Clarke (Team Futurism) invited me on his podcast to probe the worldview of a Yudkowskian, and I ended up explaining how my 50% P(Doom) and 10% P(Foom) informs my choices.
Peter and I met at LessOnline 2026, where he was one of the participants in a live Jubilee-style debate about x-risk. This one’s a little tamer, but we still cover plenty of ground, like:
Why nuclear risk is underrated yet AI doom overshadows it;
Why nobody at AI summits is appropriately freaked out;
How Yudkowsky’s data center “airstrike” claim gets mischaracterized;
Why a superintelligence that doesn’t already value humanity won’t spare us;
And the “Icarus curve”, my techno-optimist forecast pre-superintelligence.
Subscribe to the Team Futurism podcast —https://www.youtube.com/@teamfuturism3554
Timestamps
00:00:00 — Welcoming Liron on Team Futurism
00:00:38 — How Liron Got Yudkowsky-Pilled
00:04:11 — AI 2027 and the Next Qualitative Breakthrough
00:05:52 — What’s Your P(Doom)?
00:10:22 — Why Isn’t Everyone Freaked Out?
00:13:09 — How Should We Then Live?
00:16:22 — Setting the Record Straight on Yudkowsky’s “Airstrike” Claims
00:20:54 — Two Doom Buckets: Misuse vs. Rogue AI
00:24:25 — The Bioweapon Kill Scenario
00:29:02 — The “Benevolent God” Cope
00:33:06 — Epistemology: What We’re Allowed to Know
00:40:08 — Dario’s Country of Geniuses in a Data Center
00:44:37 — The Icarus Curve
00:47:17 — Meaning and Purpose in the AI Era
00:49:32 — Wrap-Up
Links
Team Futurism on YouTube — https://www.youtube.com/@teamfuturism3554
Peter Clarke on X — https://x.com/heypeterclarke
Liron vs. 9 Rationalists at LessOnline 2026 (the “Surrounded” debate with Peter) — https://lironshapira.substack.com/p/will-ai-actually-destroy-humanity
AI 2027 — https://ai-2027.com/
“If Anyone Builds It, Everyone Dies” by Eliezer Yudkowsky & Nate Soares — https://ifanyonebuildsit.com/
Species | Documenting AGI, Drew Spartz’s channel — https://www.youtube.com/@AISpecies
Dario Amodei’s “Machines of Loving Grace” — https://darioamodei.com/essay/machines-of-loving-grace
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏
Get full access to Doom Debates at lironshapira.substack.com/subscribe
Aquí están los 40 episodios más recientes de 201. Los anteriores están en el feed del podcast.
Podcasts similares
Más podcasts de Tecnología
¿Es tu podcast?
Confírmalo con una entrada en tu feed: aquí recibes un código, lo añades como podcast:txt en el canal de tu feed y nosotros lo comprobamos. Después tu podcast aparecerá como «verificado» en SoundNode.