Zum Inhalt springen

#42 - Anthony Aguirre and Malo Bourgon - Why Superintelligent AI Should Not Be Built

21.05.2025 · 1 Std. 8 Min.

Shownotes

Humanity is in a race to build superintelligent AI gods. What happens if we create beings smarter than us—and can no longer steer them?

In this special Win-Win IRL episode, Liv Boeree sits down with Anthony Aguirre (CEO of Future of Life Institute) and Malo Bourgon (CEO of MIRI) for a no-holds-barred conversation on finding alignment, avoiding power concentration and surveillance states, and the geopolitical implications of the race to AGI.

Questions include: Why can’t the market be trusted to self-regulate AI? Why does power concentration in AI pose such a massive threat to democracy? And what happens when even well-meaning developers lose control of the systems they’ve built?

We also explore win-win solutions—from compute governance and open-source safety measures, to collaborative international treaties and new models for responsible innovation. Also keep an eye out So buckle up for a provocative episode that challenges assumptions and asks what it will really take to keep the future human.


Chapters:

0:00 - Intro

4:08 - Different Categories of Risks

10:51 - Why Can't the Market Manage AI Risks Sufficiently?

15:33 - The Dangers of Power Concentration

22:52 - What Solution Space Exists?

32:55 - Should Compute be Regulated?

38:38 - Taylor Capito - Are we at an Evolutionary Nexus

42:07 - JD Ross - Decentralization vs Centralization

46:46 - Isabelle Boemeke - Nuclear Power & AI

51:06 - Peter Wang - Wrong Stories Attract the Wrong People

57:39 - Aurora Quinn-Elmore & Eliezer Yudkowsky - Coherent Extrapolated Volition

1:05:59 - Can Win-Win be embedded into AI?


Links:

♾️ Keep the Future Human Paper: https://keepthefuturehuman.ai

♾️ Future of Life Institute: https://futureoflife.org

♾️ MIRI: https://intelligence.org

♾️ Superintelligence Book by Nick Bostrom - https://www.amazon.com/Superintelligence-Dangers-Strategies-Nick-Bostrom/dp/0198739834


Credits:

♾️ Hosted by Liv Boeree

♾️ Produced by Luca de Vico


The Win-Win Podcast:


Poker champion Liv Boeree takes to the interview chair to tease apart the complexities of one of the most fundamental parts of human nature: competition. Liv is joined by top philosophers, gamers, artists, technologists, CEOs, scientists, athletes and more to understand how competition manifests in their world, and how to change seemingly win-lose games into Win-Wins.


Podcast links:

♾️ Youtube: https://www.youtube.com/playlist?list=PLWgq0OZMtwtOIyMsVM_vksqdfWcM-b68S

♾️ Spotify: https://open.spotify.com/show/03bGVUaFZmJUmEvSHNDPdI?si=64379cc23696454f

♾️ Apple Podcasts: https://podcasts.apple.com/us/podcast/win-win-with-liv-boeree/id1724791350

♾️ Pocketcast: https://play.pocketcasts.com/podcasts/7f708340-d17c-013b-f46e-0acc26574db2

#AGI #podcast #aialignment

Alle Folgen von Win-Win with Liv Boeree

Einstellungen

Darstellung

Sprache

Bestimmt, welche Sender dir auf der Startseite zuerst gezeigt werden.

Verlauf

Zuletzt gehörte Sender und Folgen und deine letzten Suchen liegen nur in diesem Browser.

Hinweise

Ein kurzer Hinweis, wenn ein Podcast, dem du folgst, eine neue Folge hat. Einzelne Podcasts stellst du über die Glocke auf ihrer Seite ab.

Konto

Favoriten speichern – in der App und auf der Website.

Tastenkürzel

/
Suche öffnen
Leertaste
Abspielen oder anhalten
← →
Im Player: 15 s zurück, 30 s vor (Folgen) · im Regal: nächste Kachel
↑ ↓
Im Regal: Ziel innerhalb der Kachel
Pos1 / Ende
Im Regal: erste bzw. letzte Kachel
Esc
Fenster oder Menü schließen
?
Diese Übersicht
Läuft gerade

Zur Senderseite

Als Nächstes

    Folgen mit „Als Nächstes“ vormerken – sie laufen nach der aktuellen Folge.