Saltar al contenido

#42 - Anthony Aguirre and Malo Bourgon - Why Superintelligent AI Should Not Be Built

21 de mayo de 2025 · 1 h 8 min

Notas del episodio

Humanity is in a race to build superintelligent AI gods. What happens if we create beings smarter than us—and can no longer steer them?

In this special Win-Win IRL episode, Liv Boeree sits down with Anthony Aguirre (CEO of Future of Life Institute) and Malo Bourgon (CEO of MIRI) for a no-holds-barred conversation on finding alignment, avoiding power concentration and surveillance states, and the geopolitical implications of the race to AGI.

Questions include: Why can’t the market be trusted to self-regulate AI? Why does power concentration in AI pose such a massive threat to democracy? And what happens when even well-meaning developers lose control of the systems they’ve built?

We also explore win-win solutions—from compute governance and open-source safety measures, to collaborative international treaties and new models for responsible innovation. Also keep an eye out So buckle up for a provocative episode that challenges assumptions and asks what it will really take to keep the future human.


Chapters:

0:00 - Intro

4:08 - Different Categories of Risks

10:51 - Why Can't the Market Manage AI Risks Sufficiently?

15:33 - The Dangers of Power Concentration

22:52 - What Solution Space Exists?

32:55 - Should Compute be Regulated?

38:38 - Taylor Capito - Are we at an Evolutionary Nexus

42:07 - JD Ross - Decentralization vs Centralization

46:46 - Isabelle Boemeke - Nuclear Power & AI

51:06 - Peter Wang - Wrong Stories Attract the Wrong People

57:39 - Aurora Quinn-Elmore & Eliezer Yudkowsky - Coherent Extrapolated Volition

1:05:59 - Can Win-Win be embedded into AI?


Links:

♾️ Keep the Future Human Paper: https://keepthefuturehuman.ai

♾️ Future of Life Institute: https://futureoflife.org

♾️ MIRI: https://intelligence.org

♾️ Superintelligence Book by Nick Bostrom - https://www.amazon.com/Superintelligence-Dangers-Strategies-Nick-Bostrom/dp/0198739834


Credits:

♾️ Hosted by Liv Boeree

♾️ Produced by Luca de Vico


The Win-Win Podcast:


Poker champion Liv Boeree takes to the interview chair to tease apart the complexities of one of the most fundamental parts of human nature: competition. Liv is joined by top philosophers, gamers, artists, technologists, CEOs, scientists, athletes and more to understand how competition manifests in their world, and how to change seemingly win-lose games into Win-Wins.


Podcast links:

♾️ Youtube: https://www.youtube.com/playlist?list=PLWgq0OZMtwtOIyMsVM_vksqdfWcM-b68S

♾️ Spotify: https://open.spotify.com/show/03bGVUaFZmJUmEvSHNDPdI?si=64379cc23696454f

♾️ Apple Podcasts: https://podcasts.apple.com/us/podcast/win-win-with-liv-boeree/id1724791350

♾️ Pocketcast: https://play.pocketcasts.com/podcasts/7f708340-d17c-013b-f46e-0acc26574db2

#AGI #podcast #aialignment

Todos los episodios de Win-Win with Liv Boeree

Ajustes

Visualización

Idioma

Determina qué emisoras ves primero en la página de inicio.

Historial

Las emisoras y episodios escuchados recientemente y tus últimas búsquedas solo se guardan en este navegador.

Avisos

Un breve aviso cuando un pódcast que sigues tiene un episodio nuevo. Puedes desactivar pódcast concretos con la campana de su página.

Cuenta

Guarda tus favoritos: en la app y en la web.

Atajos de teclado

/
Abrir búsqueda
Espacio
Reproducir o pausar
← →
En el reproductor: 15 s atrás, 30 s adelante (episodios) · en una fila: siguiente tarjeta
↑ ↓
En una fila: destino dentro de la tarjeta
Inicio / Fin
En una fila: primera o última tarjeta
Esc
Cerrar ventana o menú
?
Este resumen
Reproduciendo ahora

Ir a la emisora

A continuación

    Añade episodios con «A continuación»: sonarán después del episodio actual.