Pular para o conteúdo

#42 - Anthony Aguirre and Malo Bourgon - Why Superintelligent AI Should Not Be Built

21 de maio de 2025 · 1 h 8 min

Notas do episódio

Humanity is in a race to build superintelligent AI gods. What happens if we create beings smarter than us—and can no longer steer them?

In this special Win-Win IRL episode, Liv Boeree sits down with Anthony Aguirre (CEO of Future of Life Institute) and Malo Bourgon (CEO of MIRI) for a no-holds-barred conversation on finding alignment, avoiding power concentration and surveillance states, and the geopolitical implications of the race to AGI.

Questions include: Why can’t the market be trusted to self-regulate AI? Why does power concentration in AI pose such a massive threat to democracy? And what happens when even well-meaning developers lose control of the systems they’ve built?

We also explore win-win solutions—from compute governance and open-source safety measures, to collaborative international treaties and new models for responsible innovation. Also keep an eye out So buckle up for a provocative episode that challenges assumptions and asks what it will really take to keep the future human.


Chapters:

0:00 - Intro

4:08 - Different Categories of Risks

10:51 - Why Can't the Market Manage AI Risks Sufficiently?

15:33 - The Dangers of Power Concentration

22:52 - What Solution Space Exists?

32:55 - Should Compute be Regulated?

38:38 - Taylor Capito - Are we at an Evolutionary Nexus

42:07 - JD Ross - Decentralization vs Centralization

46:46 - Isabelle Boemeke - Nuclear Power & AI

51:06 - Peter Wang - Wrong Stories Attract the Wrong People

57:39 - Aurora Quinn-Elmore & Eliezer Yudkowsky - Coherent Extrapolated Volition

1:05:59 - Can Win-Win be embedded into AI?


Links:

♾️ Keep the Future Human Paper: https://keepthefuturehuman.ai

♾️ Future of Life Institute: https://futureoflife.org

♾️ MIRI: https://intelligence.org

♾️ Superintelligence Book by Nick Bostrom - https://www.amazon.com/Superintelligence-Dangers-Strategies-Nick-Bostrom/dp/0198739834


Credits:

♾️ Hosted by Liv Boeree

♾️ Produced by Luca de Vico


The Win-Win Podcast:


Poker champion Liv Boeree takes to the interview chair to tease apart the complexities of one of the most fundamental parts of human nature: competition. Liv is joined by top philosophers, gamers, artists, technologists, CEOs, scientists, athletes and more to understand how competition manifests in their world, and how to change seemingly win-lose games into Win-Wins.


Podcast links:

♾️ Youtube: https://www.youtube.com/playlist?list=PLWgq0OZMtwtOIyMsVM_vksqdfWcM-b68S

♾️ Spotify: https://open.spotify.com/show/03bGVUaFZmJUmEvSHNDPdI?si=64379cc23696454f

♾️ Apple Podcasts: https://podcasts.apple.com/us/podcast/win-win-with-liv-boeree/id1724791350

♾️ Pocketcast: https://play.pocketcasts.com/podcasts/7f708340-d17c-013b-f46e-0acc26574db2

#AGI #podcast #aialignment

Todos os episódios de Win-Win with Liv Boeree

Configurações

Exibição

Idioma

Define quais estações você vê primeiro na página inicial.

Histórico

As rádios e episódios ouvidos recentemente e suas últimas buscas ficam só neste navegador.

Avisos

Um aviso rápido quando um podcast que você segue tem um episódio novo. Desative podcasts individuais pelo sino na página deles.

Conta

Salve favoritos – no app e no site.

Atalhos de teclado

/
Abrir busca
Espaço
Reproduzir ou pausar
← →
No player: voltar 15 s, avançar 30 s (episódios) · na fileira: próximo bloco
↑ ↓
Na fileira: alvo dentro do bloco
Home / End
Na fileira: primeiro ou último bloco
Esc
Fechar janela ou menu
?
Esta visão geral
Tocando agora

Ir para a estação

A seguir

    Adicione episódios com “A seguir” – eles tocam depois do episódio atual.