Cheap cognition, supercharged surveillance, and AI risks, with Garrison Lovely

Patrick McKenzie (patio11) is joined by Garrison Lovely, journalist and author of Obsolete: The AI Industry's Trillion-Dollar Race to Replace Us and How to Stop It, to map the three-sided debate over AI risk and why the arguments keep talking past each other.

They then turn to what cheap cognition does to surveillance that already exists: FinCEN receives roughly 4 million suspicious activity reports a year and reads almost none of them, ICE agents run about a million queries against that database annually, and every podcast ever recorded is now transcribable for approximately nothing. The conversation covers capabilities denialism, ablated open-weights models, the fraud supply chain, and why AI is also unusually good at writing the Regulation E letter that gets your bank to fix your problem.

–
Full transcript available here: https://www.complexsystemspodcast.com/cheap-cognition-and-the-end-of-practical-obscurity-with-garrison-lovely/

–
Presenting Sponsors: Mercury, MongoDB & Chainguard

Complex Systems is presented by Mercury—radically better banking for founders. Mercury's new feature Command brings an LLM directly into your banking interface, so checking balances, finding invoices, or sending a wire is as easy as asking. Apply online in minutes at https://mercury.com/.

What's the point of building faster with AI if your database can't keep up? MongoDB's native data model mirrors the language LLMs already speak. Ship at the speed of AI while staying ACID compliant at Fortune 500 scale. Start building at https://mongodb.com/ai.

If attackers are using AI to weaponize code faster than any team can review it, your scanners won't save you. Chainguard builds libraries and container images from source, verified all the way down, with near-zero CVEs and zero malware. Build safely at https://www.chainguard.dev/.

–

Links:

• Obsolete: The AI Industry’s Trillion Dollar Race to Replace Us―and How to Stop It: https://www.amazon.com/Obsolete-Power-Profit-Machine-Superintelligence/dp/1682196305

–

Timestamps:
(00:00) Preview
(00:43) Intro
(01:51) The three-sided debate over AI risk
(05:32) Power, politics, and the tech backlash
(09:49) Capabilities denialism and AI tells
(14:54) The obsoleting machine
(17:09) The Turing test is dead
(18:56) Languages for free, then software engineering
(22:37) Sponsors: Mercury | MongoDB
(25:09) How high up the stack do the models decide?
(29:11) Surveillance and cheap cognition
(32:38) Podcasts, FinCEN, and the end of practical obscurity
(38:35) Section 702 and the data broker loophole
(39:35) Sponsor: Chainguard
(40:55) Section 702 and the data broker loophole (cont’d)
(47:23) Institutional friction and a million ICE queries
(53:29) Security through obscurity no longer works
(54:54) Scams, fraud, and ablated models
(1:00:20) Personal utility versus societal backlash
(1:02:16) AI as a tool for redress
(1:06:34) State capacity and regulating what you understand
(1:12:37) Tobacco, nuclear, and AlphaFold: strangle it or steer it
(1:18:37) Where to find Garrison and the book
(1:20:32) Wrap