1025: Word Gravity: How Transformers Bend Space, with Dr. Luis Serrano
Notes de l’épisode
In Episode #1025, Dr. Luis Serrano (Founder of Serrano Academy) joins Jon Krohn to explain the paper he co-authored on the curved spacetime of transformer architectures, in which attention stops being a lookup table and becomes something closer to gravity: words bend the space around them, and the embedding of "bank" visibly curves toward "river" as it travels through the layers of the network. In this episode, he recreates Eddington’s 1919 eclipse experiment inside a transformer, draws the line between an LLM workflow and an actual agent, explains why agent evaluation is a step harder than evaluating an essay, and gives the cleanest account of GRPO you will hear.
Additional materials: https://www.superdatascience.com/1025
Interested in sponsoring a SuperDataScience Podcast episode? Email [email protected] for sponsorship information.
In this episode you will learn:
- (00:10:53) What changed, and what survived, between the two editions of Grokking Machine Learning
- (00:27:48) Word gravity: how attention pulls "bank" toward "river"
- (00:42:12) Why RAG is an LLM workflow rather than an agent
- (00:51:23) The two-by-two that explains why GRPO powers reasoning models
Tous les épisodes de Super Data Science: ML & AI Podcast with Jon Krohn