
Scott Alexander’s Against SURA Seems Deeply Confused
Shownotes
Scott Alexander argued against SURA, the push for AI labs, their employees and whole countries to stop unilaterally and raise awareness. Zvi Mowshowitz thinks that argument is deeply confused. He sets out Scott’s core claims, then takes them in turn: what high p(doom)s actually rest on, whether building something that might kill everyone is deontologically evil, why loud resignations from the labs have worked, how far to trust AI forecasters, what Anthropic has done to the timeline, and the eternal question of China. Along the way come replies from Oliver Habryka, Daniel Kokotajlo, Vaniver, TsviBT, Eliezer Yudkowsky and others, ending with a map of Absurd, Strong and Weak SURA and where Zvi stands among them.
The Don’t Worry About the Vase Podcast is a listener-supported podcast. To receive new posts and support the cost of creation, consider becoming a free or paid subscriber.
This has been an Askwho Casts audio conversion. If you would like your own private feed of audio conversions of any blog posts you would like to listen to, You can sign up For Askwho Casts Pro at https://askwhocasts.com/, Where you can give any post the multi-voiced podcast treatment, into your own podcast feed. Thanks for listening.
* 00:00 - Introduction
* 01:33 - Table of Contents
* 02:39 - Scott Alexander’s Core Claims
* 06:24 - My High Level Responses To Scott’s Core Claims
* 09:15 - Briefly on Alignment
* 11:26 - Probabilities and How They Work
* 14:51 - Building a Thing That Might Kill Everyone is Deontologically Evil
* 22:16 - People Quitting Major Labs While Raising Alarms Obviously Works
* 26:32 - On AI Forecasting
* 29:38 - Advancing Capabilities Burns Timeline
* 31:42 - But China
* 35:39 - Ethical People Can Disagree With You
* 37:00 - There Are More Than Two Outcomes
* 40:31 - Efforts Are Complementary, Not Mutually Exclusive
* 42:54 - If Anyone Builds It, Everyone Dies
* 44:28 - Vaniver is Not Convinced
* 49:02 - There Is No Consensus That We Should Exclusively Focus on Plan A
* 59:21 - It’s Not the Incentives, It’s You
* 01:01:22 - Anthropic Is Not Optimally Pursuing These Plans
* 01:07:29 - Against Against Sura
Get full access to DWAtV Podcast at dwatvpodcast.substack.com/subscribe