Aller au contenu

AI Is Learning at the Wrong Level of Abstraction — Matthieu Wyart

11 août 2026 · 1 h 19 min

Notes de l’épisode

This episode is sponsored by Notion. Learn more about Notion's Developer Platform today at https://notion.com/mlstWhy can deep networks discover abstractions that shallow models miss? Statistical physicist Matthieu Wyart joins Tim Scarfe to argue that the answer lies in the hidden hierarchy of data. Language and images are built from parts within parts; depth lets a network recover those coarse-grained variables and escape the curse of dimensionality.The conversation moves from jamming transitions and rough loss surfaces to Chomsky, context-free grammars and machine creativity. Wyart explains why next-token prediction can still recover compositional structure, where current systems fall short of genuine scientific invention, and why predicting latent representations rather than raw tokens could make learning far more sample-efficient.They also examine diffusion models, neural scaling laws and the limits of physics-inspired theory. The final question is on a personal note: if mistakes are the price of leaving the beaten path, how much scientific risk is worth taking?---TIMESTAMPS:00:00:00 Can machines learn abstractions from data?00:02:00 Notion agentic workspace00:02:49 From statistical physics to machine learning00:06:40 What physics can explain about learning00:16:37 From Carnot to Chomsky bulldozer00:21:21 How deep networks recover hidden hierarchies00:32:43 Where machine creativity still falls short00:40:48 How deep nets escape the curse of dimensionality00:52:19 Why predict latents instead of tokens01:02:49 The sample-efficiency case for latent prediction01:08:31 Diffusion, scaling laws and text entropy01:16:40 The scientists we learn from and the mistakes we make---REFERENCES:person:[00:00:43] Noam Chomskyhttps://linguistics.mit.edu/user/chomsky/tool:[00:02:08] Notion Developer Platformhttps://www.notion.com/en-gb/blog/introducing-developer-platformpaper:[00:04:43] Mastering the game of Go with deep neural networks and tree searchhttps://www.nature.com/articles/nature16961[00:05:52] Reconciling modern machine-learning practice and the bias-variance trade-offhttps://arxiv.org/abs/1812.11118[00:25:54] How Deep Neural Networks Learn Compositional Data: The Random Hierarchy Modelhttps://arxiv.org/abs/2307.02129[00:42:12] Efficient Estimation of Word Representations in Vector Spacehttps://arxiv.org/abs/1301.3781[00:52:46] Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturehttps://arxiv.org/abs/2301.08243[00:52:54] Learn from your own latents and not from tokens: A sample-complexity theoryhttps://arxiv.org/abs/2605.27734[01:08:31] A Phase Transition in Diffusion Models Reveals the Hierarchical Nature of Datahttps://arxiv.org/abs/2402.16991[01:11:39] Scaling Laws for Neural Language Modelshttps://arxiv.org/abs/2001.08361[01:12:17] Deriving Neural Scaling Laws from the statistics of natural languagehttps://arxiv.org/abs/2602.07488[01:13:34] Prediction and Entropy of Printed Englishhttps://ieeexplore.ieee.org/document/6773263---LINKS:Download PDF transcript: https://app.rescript.info/share/f7644cdaa86c5cc1e41e484e290f2bd4

Tous les épisodes de Machine Learning Street Talk (MLST)

Paramètres

Affichage

Langue

Détermine les stations affichées en premier sur la page d’accueil.

Historique

Les stations et épisodes écoutés récemment et tes dernières recherches sont seulement enregistrés dans ce navigateur.

Notifications

Un court message quand un podcast que tu suis publie un nouvel épisode. Désactive un podcast précis avec la cloche sur sa page.

Compte

Enregistrez vos favoris – dans l’app et sur le site.

Raccourcis clavier

/
Ouvrir la recherche
Espace
Lecture ou pause
← →
Dans le lecteur : 15 s en arrière, 30 s en avant (épisodes) · dans une rangée : tuile suivante
↑ ↓
Dans une rangée : cible dans la tuile
Début / Fin
Dans une rangée : première ou dernière tuile
Esc
Fermer la fenêtre ou le menu
?
Cet aperçu
En cours de lecture

Voir la station

À suivre

    Ajoutez des épisodes avec « À suivre » – ils seront lus après l'épisode en cours.