This Harvard Professor Says AI Alignment Will BACKFIRE - Dr. Stephen Casper
Show notes
Stephen Casper, PhD, is an incoming tenure-track professor of public policy at the Harvard Kennedy School.
Prof. Casper is relatively unworried about the threat of human extinction from rogue superintelligence, but he is concerned about a handful of companies wielding extremely powerful AI and enfeebling the public. That’s why he’s an AI governance hawk who advocates for taxes and regulation to keep frontier labs in check.
We agree that slowing down AI development would make the world safer. But you know his position is unique when he says he’d prefer to have less research on AI alignment!
Timestamps
00:00:00 — Cold Open
00:00:50 — Introducing Stephen “Cas” Casper
00:04:21 — Cas’s MIT Research: Model Tampering Attacks
00:07:14 — What’s Your P(Doom)?™
00:08:34 — Why a 5–10% P(Doom) Feels Patronizing
00:13:07 — Crux: “The Intelligence Ceiling Might Not Be That High”
00:17:18 — Power Structures vs. the Shortcut-Finding AI
00:26:03 — Thought Experiment: A Data Center From the Year 2126
00:28:25 — Is a Data Center as Dangerous as a Nuke?
00:33:41 — Cas’s Mainline Scenario: Slow-Burn Gradual Disempowerment
00:40:49 — A Multipolar Powder Keg: 10 Companies, Thousands of Open Models
00:43:48 — Boarding the Doom Train at a Later Station
00:45:49 — Why Cas Is an “Anti-Timelines” Person
00:52:26 — Sycophancy, MechaHitler & Nudification: Foreseeable Failures
00:57:18 — Crux: Does Humanity Get Retries?
01:02:39 — The Scariest Superintelligent Optimizers Are Companies
01:06:08 — Lessons From the Deepfake Ecosystem: DALL-E 2 vs. Stable Diffusion
01:12:42 — The Case for a Pause Treaty
01:16:35 — Whac-A-Mole Forever: What Winning Looks Like
01:20:51 — Enshittification and the 84% Who Don’t Use AI
01:30:30 — Alignment Research as Safety Washing
01:34:25 — The Jevons Paradox of AI Safety
01:36:48 — Cas Would Press the Button to Halt Superalignment
01:39:21 — Wrap-Up: The Unsexy Path to Lowering P(Doom)
Links
Cas’s links
* Cas on X
Things referenced
* Superintelligence by Nick Bostrom
* Gradual Disempowerment (Duvenaud et al.)
* If Anyone Builds It, Everyone Dies
* AI 2027
* Center for Human-Compatible AI (Berkeley)
Doom Debates episodes mentioned
* Mike Israetel Returns — AI’s Gonna Kill Everyone vs. AI Will Make Everything Awesome
* Top AI Professor Has 85% P(Doom) — David Duvenaud
* He Leads a Top AI Research Program, But He’d Hit the PAUSE Button — Kevin Zhu
* Alignment is EASY and Roko’s Basilisk is GOOD?! — Roko Mijic
* Andrew Critch vs. Liron Shapira: Will AI Extinction Be Fast Or Slow?
* AI Could Give Humans MORE Control — Ozzie Gooen
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏
Get full access to Doom Debates at lironshapira.substack.com/subscribe