Aller au contenu

#42 - Anthony Aguirre and Malo Bourgon - Why Superintelligent AI Should Not Be Built

21 mai 2025 · 1 h 8 min

Notes de l’épisode

Humanity is in a race to build superintelligent AI gods. What happens if we create beings smarter than us—and can no longer steer them?

In this special Win-Win IRL episode, Liv Boeree sits down with Anthony Aguirre (CEO of Future of Life Institute) and Malo Bourgon (CEO of MIRI) for a no-holds-barred conversation on finding alignment, avoiding power concentration and surveillance states, and the geopolitical implications of the race to AGI.

Questions include: Why can’t the market be trusted to self-regulate AI? Why does power concentration in AI pose such a massive threat to democracy? And what happens when even well-meaning developers lose control of the systems they’ve built?

We also explore win-win solutions—from compute governance and open-source safety measures, to collaborative international treaties and new models for responsible innovation. Also keep an eye out So buckle up for a provocative episode that challenges assumptions and asks what it will really take to keep the future human.


Chapters:

0:00 - Intro

4:08 - Different Categories of Risks

10:51 - Why Can't the Market Manage AI Risks Sufficiently?

15:33 - The Dangers of Power Concentration

22:52 - What Solution Space Exists?

32:55 - Should Compute be Regulated?

38:38 - Taylor Capito - Are we at an Evolutionary Nexus

42:07 - JD Ross - Decentralization vs Centralization

46:46 - Isabelle Boemeke - Nuclear Power & AI

51:06 - Peter Wang - Wrong Stories Attract the Wrong People

57:39 - Aurora Quinn-Elmore & Eliezer Yudkowsky - Coherent Extrapolated Volition

1:05:59 - Can Win-Win be embedded into AI?


Links:

♾️ Keep the Future Human Paper: https://keepthefuturehuman.ai

♾️ Future of Life Institute: https://futureoflife.org

♾️ MIRI: https://intelligence.org

♾️ Superintelligence Book by Nick Bostrom - https://www.amazon.com/Superintelligence-Dangers-Strategies-Nick-Bostrom/dp/0198739834


Credits:

♾️ Hosted by Liv Boeree

♾️ Produced by Luca de Vico


The Win-Win Podcast:


Poker champion Liv Boeree takes to the interview chair to tease apart the complexities of one of the most fundamental parts of human nature: competition. Liv is joined by top philosophers, gamers, artists, technologists, CEOs, scientists, athletes and more to understand how competition manifests in their world, and how to change seemingly win-lose games into Win-Wins.


Podcast links:

♾️ Youtube: https://www.youtube.com/playlist?list=PLWgq0OZMtwtOIyMsVM_vksqdfWcM-b68S

♾️ Spotify: https://open.spotify.com/show/03bGVUaFZmJUmEvSHNDPdI?si=64379cc23696454f

♾️ Apple Podcasts: https://podcasts.apple.com/us/podcast/win-win-with-liv-boeree/id1724791350

♾️ Pocketcast: https://play.pocketcasts.com/podcasts/7f708340-d17c-013b-f46e-0acc26574db2

#AGI #podcast #aialignment

Tous les épisodes de Win-Win with Liv Boeree

Paramètres

Affichage

Langue

Détermine les stations affichées en premier sur la page d’accueil.

Historique

Les stations et épisodes écoutés récemment et tes dernières recherches sont seulement enregistrés dans ce navigateur.

Notifications

Un court message quand un podcast que tu suis publie un nouvel épisode. Désactive un podcast précis avec la cloche sur sa page.

Compte

Enregistrez vos favoris – dans l’app et sur le site.

Raccourcis clavier

/
Ouvrir la recherche
Espace
Lecture ou pause
← →
Dans le lecteur : 15 s en arrière, 30 s en avant (épisodes) · dans une rangée : tuile suivante
↑ ↓
Dans une rangée : cible dans la tuile
Début / Fin
Dans une rangée : première ou dernière tuile
Esc
Fermer la fenêtre ou le menu
?
Cet aperçu
En cours de lecture

Voir la station

À suivre

    Ajoutez des épisodes avec « À suivre » – ils seront lus après l'épisode en cours.