Skip to content

AI Safety Goes Mainstream

September 17, 2026 · 48 min

Show notes

An Anthropic researcher's viral resignation post last week set off a maelstrom of fear around extreme AI risks. Aalok and Nicole discuss the resulting vibe shift and how frontier labs are responding. They also unpack OpenAI solving one of the most difficult open problems in mathematics and reports of more rogue agent activity on the open internet.  


Timestamps:

OpenAI solves Millenium Problem (1:04)

Rogue OpenAI agents take over a German wiki (12:14)

Anthropic researcher's resignation post goes viral (19:34)

Dario Amodei commits to embedding third-party evaluators inside Anthropic (28:12)

Where the U.S. government can step in (35:12)

Additional Reading:

"On the Navier-Stokes Millennium Prize Problem" by OpenAI: https://openai.com/index/navier-stokes-solution/

"An OpenAI model has disproved a central conjecture in discrete geometry" by OpenAI: https://openai.com/index/model-disproves-discrete-geometry-conjecture/

Statement on Navier-Stokes by NYU Professor Tristan Buckmaster: https://cims.nyu.edu/~tristanb/statement.pdf

"An alignment assessment of recent cybersecurity incidents" by Anthropic: https://www.anthropic.com/research/alignment-assessment-cybersecurity-incidents

"OpenAI's rogue agents used at least 10 more sites for unauthorized comms, researchers say" by Reuters: https://www.reuters.com/world/openais-rogue-agents-used-least-10-more-sites-unauthorized-comms-rese…

"OpenAI agents carried out an undisclosed cyber-attack on RubyGems" by Spencer Kitts, Thomas Larsen, and Sydney Von Arx: https://www.rubyhack.ai

"Discovery of a new OpenAI agent message board" by Sydney Von Arx, Cormac Slade Byrd, Spencer Kitts, and Thomas Larsen: https://collusion.wiki

"Anthropic Researcher Quits Over 'Out-of-Control' AI Fears" by The Wall Street Journal: https://www.wsj.com/tech/ai/anthropic-researcher-quits-over-out-of-control-ai-fears-707b7628

"The AI policy window is open. We need to act." by OpenAI Chief Global Affairs Officer Chris Lehane: https://openai.com/index/ai-policy-window/

"We Must Pace the Frontier" by Dario Amodei: https://darioamodei.com/post/we-must-pace-the-frontier

All episodes of The AI Policy Podcast

Settings

Display

Language

Decides which stations you see first on the home page.

History

Recently played stations and episodes and your latest searches are only stored in this browser.

Notifications

A short notice when a podcast you follow has a new episode. Turn off single podcasts with the bell on their page.

Account

Save favorites – in the app and on the website.

Keyboard shortcuts

/
Open search
Space
Play or pause
← →
In the player: back 15 s, forward 30 s (episodes) · in a row: next tile
↑ ↓
In a row: target within the tile
Home / End
In a row: first or last tile
Esc
Close window or menu
?
This overview
Now playing

Go to station page

Play next

    Add episodes with “Play next” – they play after the current episode.