AI:AM — When Safety Tests Fail · September 21, 2026

Prakash Narayanan and Nathan Labenz open with the limits of AI safety testing, including finite-choice guardrails, fast classifiers, emergent multi-agent behavior, and why alignment measurement gets harder as systems become more capable. Max Nadeau of Coefficient Giving then explains why audits can turn into box-checking, how funders assess uncertain technical safety work, and why talent may matter more than budget. The episode closes with AI shopping agents, Amazon’s response, and the question of who owns the customer relationship when agents become first-class users.

Chapters

(0:00) When safety becomes reflexive.

(0:10) When AI audits become box-checking.

(1:22) Who owns the AI customer?

(2:01) Opening and live setup

(4:20) JEV and fast classifiers

(5:40) Guardrails for AI agents

(7:12) AI moderation use case

(11:25) System one and system two

(13:24) Multi-agent scaling

(21:18) Emergent agent hierarchies

(23:43) The case for slowing down

(34:28) AI and mathematics

(41:10) Capability acceleration

(45:45) Building an AI organization

(57:36) Funding the AI project

(58:17) Meet Max Nadeau

(1:05:21) Why safety research resists metrics

(1:15:13) When AI audits become box-checking

(1:23:07) Funding across uncertain timelines

(1:30:49) Why fund whole organizations

(1:39:06) Preparing for larger grants

(1:39:16) Why talent is the bottleneck

(1:46:48) Opening and agent setup

(1:49:16) The AI services business model

(1:52:59) Muse and autonomous shopping

(1:56:52) Who owns the customer

(1:57:37) Agents as the universal interface

(2:00:01) Why ban shopping agents

(2:05:31) The bot-detection arms race

(2:07:26) A better lane for agents

(2:09:30) Agents as internet users

(2:11:31) Closing and next steps

Guests

Max Nadeau — Program Officer, Coefficient Giving (𝕏 | LinkedIn)

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit briefing.ai-in-the-am.com