Machine Learning Street Talk (MLST)
- Épisodes
- 273
- Parution
- plusieurs fois par semaine
- Dernier épisode
- 2 Nov 2026
À propos du podcast
Welcome! We engage in fascinating discussions with pre-eminent figures in the AI field. Our flagship show covers current affairs in AI, cognitive science, neuroscience and philosophy of mind with in-depth analysis. Our approach is unrivalled in terms of scope and rigour – we believe in intellectual diversity in AI, and we touch on all of the main ideas in the field with the hype surgically removed. MLST is run by Tim Scarfe, Ph.D (https://www.linkedin.com/in/ecsquizor/) and features regular appearances from MIT Doctor of Philosophy Keith Duggar (https://www.linkedin.com/in/dr-keith-duggar/).
Épisodes 273
Why Leibniz Couldn't Find the Mind Inside the Mill (Joscha Bach)
Joscha Bach is a cognitive scientist and the founder of the California Institute for Machine Consciousness.SPONSOR:---Parallel is a single platform for all your agentic research: web search built for AI agents …
Notes de l’épisode
Joscha Bach is a cognitive scientist and the founder of the California Institute for Machine Consciousness.
SPONSOR:
---
Parallel is a single platform for all your agentic research: web search built for AI agents, with Pareto-optimal quality, cost and speed.
Get started: https://parallel.ai/mlst
---
Bach's null hypothesis is that consciousness is easy: an algorithm not much more complicated than the transformer, simple enough for an infant's brain to discover by itself. In this conversation with Tim, he lays out the Machine Consciousness Hypothesis in two parts. The first is that consciousness is a biological learning algorithm, the training algorithm that makes a brain like ours coherent. The second is that machines learning in self-organising ways might discover it in the same way. Either part can fail on its own.
From there they ask why there will never be a Turing test for consciousness, why a chess program is already a zombie, and why faking a mind can sometimes be harder than building one. The conversation runs through Leibniz's mill, the explanatory gap from John Searle to Thomas Nagel, Anil Seth's biological naturalism, integrated information theory and the letter that called it pseudoscience, Gödel and the limits of formal systems, and Mike Levin's idea that morphogenesis and the mind may run the same coherence algorithm. A tea-room interlude takes in Picbreeder and fractured, entangled representations. The episode closes on what it would take to convince us that a machine is conscious.
---
TIMESTAMPS:
0:00 Cold open: Could consciousness be a learning algorithm?
1:51 Introduction
3:37 What makes an agent?
6:16 Why consciousness needs explaining
14:57 Sponsor: Parallel
15:51 Leibniz's mill and causal patterns
24:21 Perceiving that you perceive
32:04 Two hypotheses, two ways to fail
40:42 Do minds converge?
49:23 Does consciousness converge too?
56:20 When language models reshape our thinking
59:16 Can simulated water make you wet?
1:09:21 Computation as construction
1:16:03 Error-correcting patterns and the Ruliad
1:22:28 Conscious, or simulating consciousness?
1:28:23 What would count as evidence?
1:35:49 The research programme
REFERENCES:
Joscha Bach, The Machine Consciousness Hypothesis (California Institute for Machine Consciousness)
Kumar, Clune, Lehman & Stanley, "Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesis"
Giulio Tononi, integrated information theory
Fleming et al. 2023, "The Integrated Information Theory of Consciousness as Pseudoscience"
John Searle 1980, "Minds, brains, and programs", Behavioral and Brain Sciences
Kurt Gödel 1931, "On Formally Undecidable Propositions of Principia Mathematica and Related Systems"
Alan Turing 1950, "Computing Machinery and Intelligence"
Whitehead & Russell, Principia Mathematica
Paul Thagard 2022, "Energy requirements undermine substrate independence and mind-body functionalism"
Richard Sutton 2019, "The Bitter Lesson"
Michael Graziano, attention schema theory
Bernard Baars and Stanislas Dehaene, global workspace theory
Michael Levin 2022, "Technological Approach to Mind Everywhere", Frontiers in Systems Neuroscience
Chris Olah et al. 2020, "Zoom In: An Introduction to Circuits", Distill
Adele Lopez 2025, "The Rise of Parasitic AI", LessWrong
RESCRIPT:https://app.rescript.info/share/fc7088df0dc8b84200aa6e01c9a9ab6a
Mathematics Just Changed Forever — Kevin Hartnett
Kevin Hartnett is the author of *The Proof in the Code*.SPONSOR:---Cyber Fund built the Monastery to help founders ship products that were impossible a year ago. Applications for Batch 1 are now open.Apply now …
Notes de l’épisode
Kevin Hartnett is the author of *The Proof in the Code*.SPONSOR:---Cyber Fund built the Monastery to help founders ship products that were impossible a year ago. Applications for Batch 1 are now open.Apply now: https://cyber.fund---In 1998 Thomas Hales finished a proof of the Kepler conjecture that leaned so heavily on computer code that the Annals of Mathematics could not say whether it was true. That impasse opens Kevin Hartnett's book *The Proof in the Code*, and this documentary.Built around Tim's conversation with Hartnett, the film follows Lean from a side project at Microsoft Research to the system mathematicians now use to check their own field. Leonardo de Moura, Lean's creator, explains why everything rests on a small trusted kernel. Jeremy Avigad recalls the argument over foundations that changed Lean. Kevin Buzzard describes his fear that his corner of mathematics was becoming a house of cards, and Paul Lessard separates a correct proof from a useful one. Mathlib, Peter Scholze's Liquid Tensor challenge and Terence Tao's Equational Theories project all pass through.The second half turns to AI. AlphaProof, natural-language olympiad proofs and autoformalisation sharpen the question the book keeps asking: when a machine produces a proof nobody can read, what has actually been checked? The answer lies in the formalisation gap, the distance between the mathematics people mean and the statement a checker certifies. The film closes on the human story behind the software.---TIMESTAMPS:0:00 Cold open: proof, code and the wrong problem1:50 Kepler's oranges and the proof nobody could check6:54 How mathematicians decide a proof is true9:04 Correct is not the same as useful12:17 Sponsor: Cyber Fund12:53 What Lean is, and the small kernel it trusts15:23 From Microsoft's Z3 to Lean18:22 The argument over Lean's foundations24:52 One researcher versus a growing user base26:16 Kevin Buzzard's house of cards30:43 Mathlib as a barn raising37:16 The perfectoid-spaces stunt47:12 Scholze's proof nobody had read: Liquid Tensor53:20 Can AI learn to find proofs? AlphaProof58:43 AI mathematics in 2026 and Tao's experiment1:01:36 Lean's place in AI mathematics1:02:17 AI brute force versus the right abstraction1:05:19 Autoformalisation's Trojan horse1:07:25 The formalisation gap1:11:45 A proof is not understanding1:16:58 Integration hell and the zlib specification1:21:14 Verification in the AI software lifecycle1:22:38 The human story behind Lean1:31:09 The birthday walk1:33:05 Farewell---REFERENCES:[2:12] Thomas Hales, A proof of the Kepler conjecture (Annals of Mathematics, 2005)https://annals.math.princeton.edu/2005/162-3/p01[2:12] The Flyspeck project (the formal proof of the Kepler conjecture)https://github.com/flyspeck/flyspeck[12:53] Leanhttps://lean-lang.org/[30:43] Mathlibhttps://github.com/leanprover-community/mathlib4[30:43] Kevin Hartnett, "Building the Mathematical Library of the Future" (Quanta Magazine, 2020)https://www.quantamagazine.org/building-the-mathematical-library-of-the-future-20201001/[47:12] The Liquid Tensor Experimenthttps://github.com/leanprover-community/lean-liquid[53:20] AlphaProof (Google DeepMind): AI achieves silver-medal standard solving International Mathematical Olympiad problemshttps://deepmind.google/blog/ai-solves-imo-problems-at-silver-medal-level/[59:23] The Equational Theories Project (Terence Tao)https://github.com/teorth/equational_theories[1:18:30] lean-zip (the zlib proof)https://github.com/kim-em/lean-zipFrontierMath Erdős (Epoch AI, 2026)https://epoch.ai/benchmarks/frontiermath-erdosBook: Kevin Hartnett, *The Proof in the Code* (Quanta Books, 2026)https://www.quantabooks.org/books/the-proof-in-the-code/---RESCRIPT: https://app.rescript.info/share/8604a0cd6ea4604a368028a05f4e6be4Support MLST on Patreon: https://www.patreon.com/mlstDiscord: https://discord.gg/HNnAwSduud
What Most People Get Wrong About Evolution | Akarsh Kumar
A lot of people in AI treat evolution as a dumb fallback, basically random search for when you can't take a gradient. Akarsh Kumar thinks that is wrong. Selection hangs on to partial solutions, so mutations …
Notes de l’épisode
A lot of people in AI treat evolution as a dumb fallback, basically random search for when you can't take a gradient. Akarsh Kumar thinks that is wrong. Selection hangs on to partial solutions, so mutations only need to be useful about 1% of the time for the search to keep making progress.Akarsh is a PhD student at MIT working with Phillip Isola, works with Sakana AI, and is first author of the Fractured Entangled Representation paper with Kenneth Stanley, Jeff Clune and Joel Lehman. He tells Tim Scarfe why the path a learner takes may shape the structure of what it learns, and why that is a different way of looking at intelligence from the statistical one.Most of the conversation is about ASAL, the method he led for searching whole spaces of artificial worlds. Rather than predict what a rule will do, ASAL runs the simulation and asks a foundation model what happened. Mapped across all 262,144 Life-like rules, the most interesting worlds sit on one small island. Along the way: Game of Life, Lenia (with a clip from its creator, Bert Chan), neural cellular automata, Boids, Particle Life and the emergence of persistence.---TIMESTAMPS:00:00:00 Intro: artificial life, ASAL and Core War in four minutes00:04:19 Life as it could be, not just as it is00:05:46 Why the order you learn things in matters00:08:33 No shortcuts: Wolfram, novelty search and regularisation00:10:44 Kenneth Stanley: the path matters, not just the destination00:11:31 Statistical intelligence vs regularity-based intelligence00:13:49 Artificial chemistry and other possible universes00:15:59 Convergent patterns: shadows of the substrate?00:18:02 How Conway's Game of Life works00:20:49 Change one cell, change everything?00:22:31 Lenia (with Bert Chan) and neural cellular automata00:25:26 Boids, Particle Life and cell-like creatures00:28:47 Persistence, entropy and what life is00:30:15 ASAL: a foundation model as the critic00:34:04 Impostors inside the simulation00:36:50 From primordial soup to alien animals00:38:26 262,144 rules and the island of open-endedness00:40:17 From artificial life to AGI00:41:35 Core War: Game of Thrones, Turing edition00:43:22 LLMs as the mutation step: evolving warriors00:46:03 Evolution is anything but random---REFERENCES:paper:[00:04:19] ASAL (Kumar et al.)https://arxiv.org/abs/2412.17799[00:08:08] Assembly theory https://www.nature.com/articles/s41586-023-06600-9[00:09:54] FEP paperhttps://arxiv.org/abs/2505.11581[00:22:38] Lenia: Biology of Artificial Life (Bert Chan)https://arxiv.org/abs/1812.05433[00:23:20] Growing Neural Cellular Automata (Mordvintsev et al.)https://distill.pub/2020/growing-ca/[00:41:35] Digital Red Queen: Core War with LLMs (Kumar et al.)https://arxiv.org/abs/2601.03335[00:43:42] MAP-Elites https://arxiv.org/abs/1504.04909[00:44:55] AlphaEvolve https://arxiv.org/abs/2506.13131[00:47:03] AutoML-Zero (Real et al.)https://arxiv.org/abs/2003.03384book:[00:07:04] Why Greatness Cannot Be Planned (Stanley and Lehman)https://link.springer.com/book/10.1007/978-3-319-15524-1tool:[00:18:12] Conway's Game of Lifehttps://en.wikipedia.org/wiki/Conway%27s_Game_of_Life[00:25:27] Boids (Craig Reynolds)https://www.red3d.com/cwr/boids/[00:27:12] Particle Life (Tom Mohr)https://github.com/tom-mohr/particle-life[00:41:50] Core Warhttps://corewar.co.uk/other:[00:08:49] Computational irreducibility (Stephen Wolfram)https://www.wolframscience.com/nks/p737--computational-irreducibility/[00:10:46] MLST: Why Every AI Model Is an Impostor (Kenneth Stanley, FER documentary)https://www.youtube.com/watch?v=o1q6Hhz0MAg[00:28:30] MLST: Blaise Agüera y Arcas on life emerging from codehttps://www.youtube.com/watch?v=rMSEqJ_4EBk---LINKS:Akarsh Kumar: https://akarshkumar.com/ASAL project page and demos: https://pub.sakana.ai/asal/Digital Red Queen project page: https://sakana.ai/drq/RESCRIPT:https://app.rescript.info/public/share/Qfv3T0EVzqOXL_CYYeRr9Blv8HnNm4JsXBh7ByFjJbc
How a Voice Agent Learns the Rhythm of Conversation — Shawn Wen
Tsung-Hsien (Shawn) Wen, CTO of PolyAI, tells Tim Scarfe why voice agents are harder than text agents. Voice adds time, and a good conversation depends on adapting to the person on the line, not just on …
Notes de l’épisode
Tsung-Hsien (Shawn) Wen, CTO of PolyAI, tells Tim Scarfe why voice agents are harder than text agents. Voice adds time, and a good conversation depends on adapting to the person on the line, not just on reasoning to the best answer. Shawn describes an audio-native model (Dialog-RSN-1) that first predicts a turn-taking signal, then replies in text with citations, and writes the transcript last so enterprises can audit it.Along the way: training on real, noisy calls with synthetic noise added, and why over-cleaned audio made the new model worse. Latency, and what a voice agent should do while it thinks. Why a voice with a hint of regional accent beats a generic one. Why public benchmarks fall short for voice, why enterprises want to own their agent harness, and whether behaviour belongs in the harness or in the weights.The last stretch is about working with agents: cognitive debt, the shift from producing content to checking it, Wispr Flow, building tools that agents can use, and whether slop is in the eye of the reader.This episode was produced in partnership with PolyAI.https://poly.ai CHAPTERS0:00 Why voice agents are harder than text4:27 What enterprises want, and why PolyAI built its own model9:04 How an audio-native voice model works15:21 Training data, spectrograms and synthetic noise20:36 The cocktail party problem and the future of turn-taking25:21 Latency, adaptive reasoning and keeping callers' trust31:33 Voices, personality and the uncanny valley36:32 How do you benchmark a voice agent?42:00 Harness engineering and owning the intelligence45:09 Well-specified problems and auditable agents49:46 Weight adaptation and cognitive debt56:31 Agents at work: Wispr Flow, voice and tool building1:02:50 The next decade of voice, and what counts as slopREFERENCESThe Bitter Lesson: http://www.incompleteideas.net/IncIdeas/BitterLesson.html [9:05]Retrieval-augmented generation: https://arxiv.org/abs/2005.11401 [13:23]Mel scale: https://en.wikipedia.org/wiki/Mel_scale [17:16]Victor Zue: https://en.wikipedia.org/wiki/Victor_Zue [17:45]Cocktail party effect: https://en.wikipedia.org/wiki/Cocktail_party_effect [20:39]Speaker diarisation: https://en.wikipedia.org/wiki/Speaker_diarisation [21:16]
Who Checks a Proof No Human Can Read? — Leo de Moura
Leonardo de Moura created Lean and co-created Z3. ---This episode is sponsored by Parallel.Parallel, where agents find answers: web search, extraction and deep research APIs built for AI agents.Start free with …
Notes de l’épisode
Leonardo de Moura created Lean and co-created Z3. ---This episode is sponsored by Parallel.Parallel, where agents find answers: web search, extraction and deep research APIs built for AI agents.Start free with the Parallel MCP server and $5 of credits every month: https://parallel.ai/mlst?utm_source=creator&utm_medium=podcast&utm_content=MLST---Tim Scarfe talks with Leo about how Lean escaped its original audience, why dependent types and Mathlib made it useful to working mathematicians, and what happens when formal verification leaves the lab. De Moura explains the small trusted kernel and independent checkers, and gives his account of the recent Collatz incident, in which a purported proof was accepted by both Lean's official kernel and Nanoda, apparently by exploiting a different bug in each.---TIMESTAMPS:00:00:00 Cold open: the green checkmark can lie00:00:59 Cathedral or bazaar: who controls Lean's core?00:04:55 Why Lean's core stays small and protected00:08:07 The Slack purge, Brandolini's law and the Lean FRO00:11:12 The Collatz exploit: two kernels, two bugs00:16:44 More kernels, reward hacking and safety by transparency00:21:04 Sponsor: Parallel00:21:59 Kim Morrison, Claude and the zlib proof00:25:25 Can we specify complex systems?00:28:04 Specs change: proofs are cheaper to redo with AI00:31:27 From Lean 1 to Lean 400:34:57 Dependent types in plain terms00:37:25 Lean 4's extensibility and Mathlib's growth00:41:44 Mathlib as infrastructure: Formal Frontiers00:44:15 Creativity, abstraction and nut-sniping00:48:46 Breadcrumbs, not learning: what AI agents lack00:53:03 Competence without comprehension, and verified guardrails00:56:21 Is the human still the author?01:01:44 AlphaProof, LLMs and why certificates still matter01:06:38 What's next for Lean, and its legacy01:11:42 How to start learning Lean---REFERENCES:tool:[00:00:48] Leanhttps://lean-lang.org/[00:01:24] Mathlibhttps://github.com/leanprover-community/mathlib4[00:12:09] nanoda_libhttps://github.com/ammkrn/nanoda_lib[00:12:19] CollatzLeanhttps://github.com/xrchz/CollatzLean/blob/a79357462a33d2a6babd4cf6c8d8bcd25425d653/README.md[00:13:06] Lean issue 14576https://github.com/leanprover/lean4/issues/14576[00:13:21] Lean pull request 14577https://github.com/leanprover/lean4/pull/14577[00:13:45] nanoda_lib pull request 22https://github.com/ammkrn/nanoda_lib/pull/22[00:15:33] Lean comparatorhttps://github.com/leanprover/comparator[00:18:02] Lean4Leanhttps://github.com/digama0/lean4lean[00:19:50] ARC-AGI-3https://arcprize.org/arc-agi/3[00:21:59] lean-ziphttps://github.com/kim-em/lean-zip[00:29:45] CompCerthttps://compcert.org/[00:29:45] seL4https://www.sel4.org/[00:30:23] Z3https://github.com/Z3Prover/z3[00:38:10] Veilhttps://github.com/verse-lab/veil[00:38:10] Velvethttps://github.com/verse-lab/velvetorganization:[00:00:52] Lean FROhttps://lean-lang.org/fro/[00:42:39] Mathlib Initiativehttps://mathlib-initiative.org/about/person:[00:05:11] Ilya Sergeyhttps://ilyasergey.net/[00:10:31] Joachim Breitnerhttps://www.joachim-breitner.de/[00:32:58] Adam Chlipalahttps://adam.chlipala.net/[00:38:42] Kevin Buzzardhttps://www.ma.imperial.ac.uk/~buzzard/[01:10:16] Terence Taohttps://terrytao.wordpress.com/book:[00:08:19] The Proof in the Codehttps://us.macmillan.com/books/9780374620059/theproofinthecode/other:[00:10:05] Brandolini's lawhttps://en.wikipedia.org/wiki/Brandolini%27s_law[00:45:33] A new result on unit distanceshttps://openai.com/index/model-disproves-discrete-geometry-conjecture/[01:01:08] Fermat's Last Theorem formalisationhttps://imperialcollegelondon.github.io/FLT/paper:[00:34:36] The Lean 4 theorem prover and programming languagehttps://doi.org/10.1007/978-3-030-79876-5_37---RESCRIPT:https://app.rescript.info/share/7d3d4a0059443236a01f6c9acbf4db58https://app.rescript.info/api/public/sessions/b007264c0ce89047/pdf
When AI Research Starts Moving Faster Than Human Research - Zhengyao Jiang
Weco let an AI coding agent rewrite the harness around another agent for eight days: its code, prompts and tools, while the underlying language model stayed fixed. Tim Scarfe asks Weco co-founder Zhengyao Jiang …
Notes de l’épisode
Weco let an AI coding agent rewrite the harness around another agent for eight days: its code, prompts and tools, while the underlying language model stayed fixed. Tim Scarfe asks Weco co-founder Zhengyao Jiang what the reported gains over two years of human engineering actually demonstrate.The discussion examines AIDE 85's generated code, held-out evaluation and the difficulty of separating useful discoveries from reward hacking. Jiang explains Weco's four levels of recursive self-improvement and compares the experiment with AlphaEvolve and the Darwin Gödel Machine.The limits matter as much as the gains. Jiang explains why the experiment did not establish that the system had become a better improver. The conversation closes with open-ended search, human-designed primitives and Parameter Golf: where does the next useful idea come from when the agent is searching inside a space that people designed?---TIMESTAMPS:00:00:00 Eight days of self-improvement: what counts?00:03:25 AIDE and the puzzle of useful spaghetti code00:08:38 Four levels of recursive self-improvement00:12:02 What AIDE 85 changed and how it was tested00:20:04 AlphaEvolve, Darwin Gödel Machine and the RSI claim00:26:21 Reward hacking and the limits of detection00:33:09 Open-ended search, harness tuning and creativity00:39:43 Parameter Golf and the limits of self-improvement---REFERENCES:organization:[00:00:30] Weco AIhttps://www.weco.ai/other:[00:00:33] AIDE²: The First Evidence of Recursive Self-Improvementhttps://www.weco.ai/blog/first-evidence-of-recursive-self-improvement[00:14:11] Faulty reward functions in the wildhttps://openai.com/index/faulty-reward-functions/[00:29:59] The Hugging Face incident and the road aheadhttps://openai.com/index/hugging-face-incident-and-the-road-ahead/tool:[00:03:29] AIDEhttps://github.com/WecoAI/aideml[00:04:29] MLE-benchhttps://github.com/openai/mle-bench[00:04:33] ALE-Benchhttps://github.com/SakanaAI/ALE-Bench[00:04:52] WeatherBench 2https://github.com/google-research/weatherbench2[00:08:18] ReActhttps://react-lm.github.io/[00:39:43] Parameter Golfhttps://github.com/openai/parameter-golfpaper:[00:20:08] AlphaEvolve: A coding agent for scientific and algorithmic discoveryhttps://arxiv.org/abs/2506.13131v1[00:21:35] Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agentshttps://arxiv.org/abs/2505.22954v3[00:23:45] Hyperagentshttps://arxiv.org/abs/2603.19461v1[00:27:01] SpecBench: Measuring Reward Hacking in Long-Horizon Coding Agentshttps://arxiv.org/abs/2605.21384book:[00:33:14] Why Greatness Cannot Be Planned: The Myth of the Objectivehttps://link.springer.com/book/10.1007/978-3-319-15524-1---LINKS:https://app.rescript.info/share/3a9dc6189cb539c6a05fcc4f75c101b3PDF:https://app.rescript.info/api/public/sessions/9eda60ede2b31c92/pdf
How Deep Learning Finally Cracked Messy Tables - Frank Hutter
Frank Hutter, co-founder of Prior Labs, talks about TabPFN, a tabular foundation model that makes predictions in a single forward pass, and the research behind it.TabPFN is pre-trained on synthetic datasets …
Notes de l’épisode
Frank Hutter, co-founder of Prior Labs, talks about TabPFN, a tabular foundation model that makes predictions in a single forward pass, and the research behind it.
TabPFN is pre-trained on synthetic datasets drawn from a prior over structural causal models, rather than on real data. At prediction time it takes the whole training table as context and outputs an approximation of the Bayesian posterior predictive distribution, without per-dataset training or hyperparameter search. Frank explains how this grew out of his earlier work on AutoML and neural architecture search, how the priors are built and revised, and why tabular data was hard for deep learning for so long.
The conversation also covers the TabArena benchmark, how the architecture changed from TabPFN v1 to v3, scaling to larger tables, using the model with coding agents, test-time compute, Google's TabFM, causal inference and interventions, and relational data. At the end, a short update Frank recorded after the interview covers the TabPFN-3.5 release.
Prior Labs:
TabPFN-3.5: https://priorlabs.ai/tabpfn-3-5
https://priorlabs.ai/careers
TOC:
00:00 Introduction
00:44 Welcome and Frank's background
02:05 Why tabular data was hard for deep learning
10:17 Pre-training on synthetic data
12:52 The TabArena benchmark
19:28 From AutoML to neural architecture search
26:34 TabPFN as a learned algorithm
30:50 Bayesian prediction in one forward pass
39:37 Scaling to larger tables
47:48 Using TabPFN with coding agents
57:47 Output heads and architecture from v1 to v3
1:05:29 Test-time compute and adaptation
1:13:32 Google's TabFM
1:16:53 How the priors are designed
1:18:40 Correlation, causation and interventions
1:35:22 Relational and multimodal data
1:38:31 Use in organisations
1:46:38 The open research arm
1:50:21 Update: TabPFN-3.5
REFS:
TabPFN v2, Nature (Hollmann et al., 2025)
https://www.nature.com/articles/s41586-024-08328-6
Transformers Can Do Bayesian Inference (Müller et al.)
https://arxiv.org/abs/2112.10510
TabArena (Erickson et al.)
https://arxiv.org/abs/2506.16791
AutoGluon-Tabular (Erickson et al.)
https://arxiv.org/abs/2003.06505
Beyond IID: How General Are Tabular Foundation Models, Really?
https://arxiv.org/abs/2606.30410
Neural Architecture Search: A Survey (Elsken, Metzen & Hutter)
https://arxiv.org/abs/1808.05377
Auto-WEKA (Thornton et al.)
https://www.cs.ubc.ca/~hutter/papers/AutoWEKA-KDD2013.pdf
TabPFN v1 (Hollmann et al., 2022)
https://arxiv.org/abs/2207.01848
TabPFN-3 technical report
https://arxiv.org/abs/2605.13986
TabPFN-2.5 report
https://arxiv.org/abs/2511.08667
CAAFE (Hollmann et al.)
https://arxiv.org/abs/2305.03403
TabICL (Qu et al.)
https://arxiv.org/abs/2502.05564
TabICLv2 (Qu et al.)
https://arxiv.org/abs/2602.11139
Google TabFM
https://research.google/blog/introducing-tabfm-a-zero-shot-foundation-model-for-tabular-data/
TALENT benchmark (Ye et al.)
https://arxiv.org/abs/2407.00956
Do-PFN (Robertson et al.)
https://arxiv.org/abs/2506.06039
CausalPFN (Balazadeh et al.)
https://arxiv.org/abs/2506.07918
Causal Foundation Models with Partial Graphs (Reuter et al.)
https://arxiv.org/abs/2602.14972
RelBench (Robinson et al.)
https://arxiv.org/abs/2407.20060
RelArena-α, TabPFN-Rel and RPI
https://arxiv.org/abs/2608.16319
TabPFN on GitHub
https://github.com/PriorLabs/TabPFN
TabPFN-3.5 technical report
https://arxiv.org/abs/2609.17895
Otto Group Product Classification Challenge (Kaggle, 2015)
https://www.kaggle.com/competitions/otto-group-product-classification-challenge
---RESCRIPT:https://app.rescript.info/share/e99676c25ee6189fbf54c9be07eb623e
Why Scaling Prediction Cannot Create Intelligence - Alexander Mattick
Alexander Mattick is a researcher at Fraunhofer IIS and a PhD researcher at the University of Technology Nuremberg (UTN), and a regular on Yannic Kilcher's Discord. He first came on MLST in 2022, after helping …
Notes de l’épisode
Alexander Mattick is a researcher at Fraunhofer IIS and a PhD researcher at the University of Technology Nuremberg (UTN), and a regular on Yannic Kilcher's Discord. He first came on MLST in 2022, after helping research the Yann LeCun and Randall Balestriero episode on interpolation.
SPONSOR:
---
Cyber Fund built the Monastery to help founders ship products that were impossible a year ago. Applications for Batch 1 are now open.
Apply now: https://cyber.fund
---
Alexander treats inference as the thread running through modern machine learning: once you have a model, what does it cost to get an answer out of it? He works through Monte Carlo, GFlowNets, energy-based models, diffusion, normalising flows and flow matching, with four short explainers he recorded himself. He is blunt about energy-based models: you can sample from them in principle, but it is rarely worth the compute. JEPA and "world model", he says, are closer to branding than to technical categories.
Next: theories of deep learning, none of which he thinks predicts enough yet to guide practice, then reinforcement learning.
---
0:00 Cold open: information is expensive
0:51 Welcome back, Alexander Mattic
2:08 Alexander's research background
2:50 Inference: densities, sampling and Monte Carlo
6:42 GFlowNets, energy functions and MCMC
9:45 Explainer: energy-based models
11:03 Why model a density at all?
17:30 From learned energies to flow matching
25:08 Explainers: diffusion and normalising flows
28:33 Are energy-based models generative?
33:22 JEPA, contrastive learning and collapse
41:13 Why non-language modalities need flows
44:51 Inference as search: branch and bound
49:43 Q-learning and delayed consequences
55:14 Flow matching, optimal transport, Fokker-Planck
1:00:03 Explainer: flow matching
1:01:49 AlphaFold, latents and scale versus architecture
1:07:52 Two families of deep learning theory
1:15:04 What a good theory would predict
1:23:53 The manifold hypothesis and compression
1:28:25 Is reward enough?
1:32:01 Control theory versus reinforcement learning
1:37:22 The Bitter Lesson and expensive information
1:42:08 Constrained RL: the constrained MDP toolbox
1:50:12 Creativity as constrained search
1:55:44 Reality is protean: when abstractions hold
2:00:32 What is a world model?
2:04:38 Prediction is not control
2:08:13 Robot demos, MPC and reliability
---
REFERENCES:
[6:55] GFlowNets (Bengio et al., 2021)
https://arxiv.org/abs/2106.04399
[38:46] Contrastive Self-Supervised Learning (Anand, 2020)
https://ankeshanand.com/blog/2020/01/26/contrative-self-supervised-learning.html
[38:56] LeJEPA (Balestriero and LeCun, 2025)
https://arxiv.org/abs/2511.08544v3
[47:10] RL for Node Selection in Branch-and-Bound (Mattick)
https://openreview.net/forum?id=0ez68a5UqI
[56:20] Flow Matching for Generative Modeling
https://arxiv.org/abs/2210.02747v2
[1:12:41] Disentangling feature and lazy training in deep neural networks
https://arxiv.org/abs/1906.08034v4
[1:31:05] Reward is enough (Silver)
https://doi.org/10.1016/j.artint.2021.103535
[1:35:12] Learning ReLU networks to high uniform accuracy is intractable (Berner et al.)
https://arxiv.org/abs/2205.13531v2
[1:40:20] Dota 2 with Large Scale Deep RL
https://arxiv.org/abs/1912.06680v1
[1:45:41] Constrained Update Projection for Safe Policy Optimization (Yang et al., 2022)
https://arxiv.org/abs/2209.07089
[1:46:11] SafeMPO (ICLR 2026)
https://openreview.net/forum?id=1m0EU6QXj6
[1:50:17] Why Creativity Cannot Be Interpolated
https://archive.mlst.ai/paper/why-creativity-cannot-be-interpolated/
[1:51:39] Invalid Action Masking (Huang and Ontañón)
https://arxiv.org/abs/2006.14171
[2:00:04] Probability Theory: The Logic of Science (Jaynes, 2003)
https://www.cambridge.org/core/books/probability-theory/9CA08E224FF30123304E6D8935CF1A99
[2:01:53] Training Agents Inside of Scalable World Models (Hafner et al., 2025)
https://arxiv.org/abs/2509.24527v1
[2:03:43] World Models (Ha and Schmidhuber, 2018)
https://arxiv.org/abs/1803.10122v4
How Physical AI Learns Across Language, Video and Action — Ming-Yu Liu
The car making a left turn at the start of this episode was never filmed. Cosmos 3 generated it. Ming-Yu Liu, who leads the Cosmos research at NVIDIA, explains how one model can describe a video, generate one …
Notes de l’épisode
The car making a left turn at the start of this episode was never filmed. Cosmos 3 generated it. Ming-Yu Liu, who leads the Cosmos research at NVIDIA, explains how one model can describe a video, generate one, and produce robot actions.
He walks Tim through the architecture. A vision language model reasons one token at a time; its weights then initialise a bidirectional diffusion generator for video, audio and action, and a shared temporal position scheme lines up signals that run at different rates. Ming-Yu treats "world model" as a set of tools, not one definition: forward dynamics, inverse dynamics and policy, trained together under a capacity limit so that each helps the others. He also explains why plentiful first-person human video carries over to robots, which have far less data of their own, and why a Cosmos model post-trained on the DROID dataset is a good starting point for pick-and-place policies.
The most practical thread is testing. A neural simulator does not need accurate success rates. It only needs to rank policy A above policy B the way the real world would, so a team can narrow down which checkpoints deserve a real trial. Cosmos Dreams applies that closed-loop idea to driving and robotics, and Ming-Yu argues that humanoids around children and pets make safety matter even more than it does for cars. The conversation ends on the Super, Nano and Edge sizes (Edge targets Jetson Thor, Orin and DGX Spark) and where to find the open weights, code and data.
This episode is a paid partnership with NVIDIA.
Learn more about Cosmos: https://nvda.ws/4cJoY1S
Explore Cosmos Lab: https://research.nvidia.com/labs/cosmos-lab/cosmos3/
---
TIMESTAMPS:
00:00:00 A road that was never filmed
00:02:28 Inside Cosmos 3: reasoning and generator towers
00:05:02 World models: dynamics, policy and one clock
00:08:59 Learning robot skills from human video
00:11:06 Ambiguous tasks and system 2 planning
00:12:53 Neural simulators for policy verification
00:16:41 Cosmos as a starting point for robot policies
00:19:00 Cosmos Dreams and robot safety
00:22:04 Super, Nano and Edge model sizes
00:24:24 Open models, the Cosmos repo and feedback
---
REFERENCES:
tool:
[00:00:13] Cosmos 3 (NVIDIA Cosmos Lab project page)
https://research.nvidia.com/labs/cosmos-lab/cosmos3/
[00:18:27] NVIDIA Cosmos GitHub repository
https://github.com/NVIDIA/cosmos
[00:22:05] Cosmos3-Edge model card
https://huggingface.co/nvidia/Cosmos3-Edge
[00:22:15] Cosmos3-Super model card
https://huggingface.co/nvidia/Cosmos3-Super
[00:22:16] Cosmos3-Nano model card
https://huggingface.co/nvidia/Cosmos3-Nano
[00:22:50] NVIDIA Jetson Thor
https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-thor/
[00:22:52] NVIDIA Jetson Orin
https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-orin/
[00:22:53] NVIDIA DGX Spark
https://www.nvidia.com/en-us/products/workstations/dgx-spark/
[00:24:42] Cosmos 3 collection on Hugging Face
https://huggingface.co/collections/nvidia/cosmos3
other:
[00:01:07] Cosmos-Dreams closed-loop simulators (NVIDIA SIGGRAPH 2026 blog)
https://blogs.nvidia.com/blog/siggraph-news-2026/
paper:
[00:08:54] Cosmos 3: Omnimodal World Models for Physical AI
https://arxiv.org/abs/2606.02800
[00:17:43] DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset
https://arxiv.org/abs/2403.12945
---
RESCRIPT: https://app.rescript.info/share/e2385948cf465f0d6a2c0930150fc3ab
Speech Recognition Is Not a Solved Problem — Pavan Kumar Reddy
Pavan Kumar Reddy leads audio research at Mistral AI. He joins Tim Scarfe for a deep technical tour of Voxtral — and explains why the frontier of deployed voice is still a cascade of specialised models rather …
Notes de l’épisode
Pavan Kumar Reddy leads audio research at Mistral AI. He joins Tim Scarfe for a deep technical tour of Voxtral — and explains why the frontier of deployed voice is still a cascade of specialised models rather than one end-to-end system.
IN PARTNERSHIP WITH MISTRAL AI:
---
This episode was produced in partnership with Mistral AI.
Mistral AI: https://mistral.ai/
---
The conversation opens on architecture. Voxtral Chat feeds a 3B Ministral text trunk with continuous embeddings from an audio encoder, passed to the decoder as direct token input rather than through cross-attention as in Whisper, so the model can answer questions about emotion, timing and who spoke when without an intermediate transcript to lose them. The real-time model becomes a dual-stream decoder that consumes audio and emits text at once, at a target delay down to 160ms, with slower streams in parallel for anything that can wait for more context.
On generation, Pavan explains why Voxtral TTS predicts continuous latents rather than discrete codec tokens, traces the lineage from SoundStream through EnCodec to Mimi's split of semantic and acoustic codebooks, and places FSQ and flow matching in it. Tim presses on the priors underneath: why a mel spectrogram instead of raw waveform, what noise augmentation buys, and when acoustic overfitting becomes somebody's fine-tuning problem. Then the failure modes. Diarisation is emitted autoregressively inside the transcript rather than by a separate head, which makes streaming diarisation fragile — less context, late speaker changes, invented extra speakers. And because the architecture commits to what it has already predicted, one out-of-distribution mistake compounds into looping or skipped segments, which is what DPO corrects: the negative supervision pre-training and SFT cannot give.
The last third is the argument Tim keeps returning to. Customers running voice agents over millions of sessions describe scaffolding, not a solved problem, with a sharp drop outside the top few languages. Cascades survive because each component stays separately adaptable, observable and constrainable. And voice alone is cognitive debt: absorbing information and deciding in one serial stream is harder than glancing at a menu. Voice becomes ubiquitous beside a screen, not instead of one.
---
TIMESTAMPS:
00:00:00 Cold open
00:00:46 Why Mistral moved into audio
00:09:27 Inside Voxtral: trunk, encoder, dual streams
00:20:22 Speech that works in real time
00:30:52 How a voice becomes tokens
00:39:59 Flow matching, FSQ and the new codec
00:52:51 When speech models lose the speaker
01:03:23 Correcting hallucinations with preferences
01:12:12 Controlling synthetic speech
01:20:06 Why cascades still win
01:29:25 Speech in the wild
01:33:46 Audio models as interfaces
01:37:54 Why voice still needs a screen
---
REFERENCES:
paper:
[00:01:42] Mistral 7B
https://arxiv.org/abs/2310.06825
[00:09:38] Voxtral
https://arxiv.org/abs/2507.13264
[00:14:41] Whisper: Robust Speech Recognition
https://arxiv.org/abs/2212.04356
[00:19:11] Voxtral Realtime
https://arxiv.org/abs/2602.11298
[00:21:52] Delayed Streams Modeling (Kyutai)
https://arxiv.org/abs/2509.08753
[00:30:52] Voxtral TTS
https://arxiv.org/abs/2603.25551
[00:32:38] SoundStream neural audio codec
https://arxiv.org/abs/2107.03312
[00:34:59] Flow Matching for Generative Modeling
https://arxiv.org/abs/2210.02747
[00:37:03] EnCodec: High Fidelity Neural Audio Compression
https://arxiv.org/abs/2210.13438
[00:37:42] Moshi and the Mimi codec
https://arxiv.org/abs/2410.00037
[00:39:05] Finite Scalar Quantization (FSQ)
https://arxiv.org/abs/2309.15505
[01:03:33] Direct Preference Optimization (DPO)
https://arxiv.org/abs/2305.18290
dataset:
[00:46:14] Mozilla Common Voice
https://commonvoice.mozilla.org/en/datasets
organization:
[00:50:47] Hugging Face
https://huggingface.co/
How Replication Could Teach Machines What Good Science Looks Like — Edward Hughes
Can a machine learn the judgement that separates a plausible-looking result from a faithful experiment? Edward Hughes, Chief Scientist and co-founder of Inherent, joins Tim Scarfe to argue that creativity is …
Notes de l’épisode
Can a machine learn the judgement that separates a plausible-looking result from a faithful experiment? Edward Hughes, Chief Scientist and co-founder of Inherent, joins Tim Scarfe to argue that creativity is not optimisation, and that the missing capability in AI is choosing which questions are worth asking.
SPONSOR:
---
Cyber Fund built the Monastery to help founders ship products that were impossible a year ago.
Apply now: https://cyber.fund
---
Edward makes the case that Move 37 was innovative rather than creative, and that the field, not the individual, decides what counts as a discovery. That reframing runs through Csikszentmihalyi, Deutsch and exaptation into open-endedness, where deceptive goals and imperfect world models turn out to be the point rather than the problem. The second half turns to the paper: Replica, a task space built by redacting figures from real papers, and Faraday, a 27-billion-parameter model trained to steer a frontier coding agent that then beats the frontier on held-out replications.
---
TIMESTAMPS:
00:00:00 Cold open: Move 37, Faraday and collective intelligence
00:01:08 Sponsor: CyberFund
00:01:46 Inherent's $50M raise and the road from string theory
00:09:14 Three timescales of learning: weights, context, culture
00:13:47 Move 37 was innovative, not creative: the field decides
00:20:39 Creativity as satisficing: the urinal and evolution
00:25:06 Exaptation and the Tristan chord: creativity in context
00:30:56 Coherence for whom? Deutsch's hard-to-vary explanations
00:35:53 Why copying is creative: Deutsch and the constraint engineer
00:42:27 Societies of agents and the strong Moravec paradox
00:45:51 Evaluate in hindsight: from Lean proofs to climate change
00:51:56 Picbreeder, local goals and why discovery needs deception
00:57:21 Spaghetti proofs, translation layers and superhuman Go
01:00:37 Does nature compress? Naturalness and real patterns
01:07:36 Why replicate? Replica's redacted figures and Faraday
01:12:31 Faraday beats Codex, Claude and GLM 5.2 on held-out tasks
01:15:31 Replication to innovation: how the Transformer happened
01:18:26 Deep replication: what Faraday learns from Voyager and GNoME
01:23:37 Can the AI scientist cheat? Goodharting the judge
01:29:09 Inside Replica: scale-down, 8xB300 runs, per-task rubrics
01:34:11 The RL crisis: getting GRPO to work with per-turn credit
01:39:43 Weights vs harnesses: AlphaEvolve, DGM and EvoTune
01:45:45 The recursive company: agents cross a phase transition
01:50:35 Collective intelligence and the electric dynamo
01:55:46 What replaces OKRs? Incumbents and the burden of knowledge
---
REFERENCES:
MLST Creativity Article:
https://archive.mlst.ai/read/why-creativity-cannot-be-interpolated
organization:
[00:01:47] Inherent
https://inherentlabs.ai/
other:
[00:20:51] Marcel Duchamp, Fountain
https://www.tate.org.uk/art/artworks/duchamp-fountain-t07573
[00:05:19] Human-Timescale Adaptation in an Open-Ended Task Space (Adaptive Agent)
https://arxiv.org/abs/2301.07608
[00:06:05] The AI Scientist
https://arxiv.org/abs/2408.06292
[00:12:13] Training AI Scientists to Replicate Research (Replica and Faraday)
https://arxiv.org/abs/2608.13331
[01:44:46] Evolutionary Principles in Self-Referential Learning
https://people.idsia.ch/~juergen/diploma.html
[01:59:33] Are Ideas Getting Harder to Find?
https://www.nber.org/papers/w23782
book:
[00:16:04] Creativity: Flow
https://search.worldcat.org/title/254487436
[00:26:22] Why Greatness Cannot Be Planned
https://link.springer.com/book/10.1007/978-3-319-15524-1
[00:33:03] The Beginning of Infinity
https://www.penguinrandomhouse.com/books/293575/the-beginning-of-infinity-by-david-deutsch/
[01:55:47] Laws of Knowledge
https://www.penguin.co.nz/books/the-infinite-alphabet-9780241655672
(Full list refs on YT/rescript)
---
RESCRIPT:
https://app.rescript.info/session/670296ba913761d0?share=6281911cac9bdbff637f10819d4d1e5c
AI 2040: Plan A report - Daniel Kokotajlo & Thomas Larsen
Could slowing AI development make superintelligence safer? Daniel Kokotajlo and Thomas Larsen of the AI Futures Project join Tim Scarfe to examine AI 2040: Plan A, a proposal to buy time before AI exceeds human …
Notes de l’épisode
Could slowing AI development make superintelligence safer? Daniel Kokotajlo and Thomas Larsen of the AI Futures Project join Tim Scarfe to examine AI 2040: Plan A, a proposal to buy time before AI exceeds human control.
SPONSOR:
---
Cyber Fund built the Monastery to help founders ship products that were impossible a year ago.
Apply now: https://cyber.fund
---
After revisiting AI 2027 and the limits of forecasting, they ask what happens when AI can automate research and sustain an economy without human workers. Tim challenges the case for general models and asks whether intelligence alone explains power. Plan A proposes an initial pause to build safety infrastructure, then cautious development up to the strongest AI that can still be reliably controlled. The discussion tests the distinction between control and alignment, the case for public AI research, and whether the US and China could enforce a slowdown. It ends with the evidence that would change their forecasts.
---
TIMESTAMPS:
00:00:00 AI 2040: a slower route to superintelligence
00:01:34 Sponsor: Cyber Fund
00:02:12 From OpenAI to AI 2027
00:06:58 Forecasts, war games and self-fulfilling prophecies
00:17:44 Why AI sceptics are changing their minds
00:23:04 When AI can replace its own researchers
00:28:45 Could an AI economy grow without human workers?
00:37:32 One general model or a society of specialists?
00:47:43 Brains, machines and collective intelligence
00:56:12 Plan A: buy time at the controllable frontier
01:00:02 Why control buys time but cannot replace alignment
01:06:36 Why AI research should be public
01:10:32 Can the US and China enforce an AI slowdown?
01:19:04 Why AI policy debates miss the technology
01:21:56 Is AI normal technology? The remaining disagreement
Many thanks to James Wilken-Smith for helping with show research.
---
REFERENCES:
other:
[00:00:01] AI 2040: Plan A
https://ai-2040.com/
[00:03:27] AI 2027
https://ai-2027.com/
[00:13:47] Scenario Scrutiny for AI Policy
https://blog.aifutures.org/p/scenario-scrutiny-for-ai-policy
[00:33:11] The 2028 Global Intelligence Crisis
https://www.citriniresearch.com/p/2028gic
[01:00:40] Brief independent investigation of agents' behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident
https://www.redwoodresearch.org/research/hugging-face-incident
[01:09:21] The Hugging Face incident and the road ahead
https://openai.com/index/hugging-face-incident-and-the-road-ahead/
[01:22:01] AI as Normal Technology
https://www.normaltech.ai/p/ai-as-normal-technology
[01:22:51] Common Ground between AI 2027 & AI as Normal Technology
https://asteriskmag.substack.com/p/common-ground-between-ai-2027-and
person:
[00:19:43] Geoffrey Hinton
https://www.cs.toronto.edu/~hinton/
[00:20:07] Ryan Greenblatt
https://www.lesswrong.com/users/ryan_greenblatt
[00:26:06] Elon Musk
https://www.tesla.com/elon-musk
tool:
[00:21:46] ARC-AGI-3
https://arcprize.org/arc-agi/3
[00:21:53] AlphaGo and Move 37
https://deepmind.google/research/alphago/
[00:39:41] Claude
https://claude.com/product/overview
[00:39:58] NVIDIA H100 GPU
https://www.nvidia.com/en-us/data-center/h100/
paper:
[00:24:42] Training AI Scientists to Replicate Research
https://arxiv.org/abs/2608.13331v1
[01:27:19] Validity of the single processor approach to achieving large scale computing capabilities
https://www.cs.cmu.edu/~18742/papers/Amdahl1967.pdf
book:
[00:28:52] Bullshit Jobs: A Theory
https://www.simonandschuster.com/books/Bullshit-Jobs/David-Graeber/9781501143335
organization:
[01:05:09] Redwood Research
https://www.redwoodresearch.org/
---
RESCRIPT:
https://app.rescript.info/public/share/33d1a58fa8f307ae7dfd504d4fdaa9d5
Designing How AI Grows — Tom McGrath
Tom McGrath is co-founder and Chief Scientist at Goodfire, and a former Google DeepMind researcher. He joins Tim Scarfe to ask what neural networks actually learn, whether their internal representations …
Notes de l’épisode
Tom McGrath is co-founder and Chief Scientist at Goodfire, and a former Google DeepMind researcher. He joins Tim Scarfe to ask what neural networks actually learn, whether their internal representations converge on structures in the world, and whether interpretability can extract new scientific knowledge rather than merely explain model outputs.
Beginning with AlphaZero and learned modularity, the conversation moves into neural geometry: concept manifolds, reusable computation inside Llama, and why activation steering can fail when it pushes a model off-manifold. McGrath then makes the case for intentional design, using interpretability as part of the training loop. They examine controlled generalisation, features as rewards, predictive data debugging, and the uncomfortable fact that a model may recognise a hallucination or reward hack and still produce it.
The discussion closes on grader awareness, oversight and collusion between adaptive agents, then returns to sparse autoencoders. SAEs are useful, McGrath argues, but they may fracture the higher-dimensional structures networks actually use. This episode was made with support from Goodfire.
---
TIMESTAMPS:
00:00:00 Introduction: Can interpretability speed-run science?
00:02:03 The invisible grader
00:06:51 What AlphaZero learned from the world
00:12:24 Interpretability as a control loop
00:21:54 The forbidden method and safer interventions
00:37:36 Why models catch hallucinations too late
00:46:19 Debug the dataset before training
00:50:44 Why neural networks become modular
00:55:57 Finding the geometry inside a network
01:02:55 Why steering falls off the manifold
01:12:10 A reusable calculator inside Llama
01:17:19 From abstractions to goals
01:25:28 Reward hacking, oversight and collusion
01:37:23 Are sparse autoencoders dead?
---
REFERENCES:
paper:
[00:05:45] Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs
https://arxiv.org/abs/2502.17424v7
[00:11:05] Acquisition of Chess Knowledge in AlphaZero
https://arxiv.org/abs/2111.09259
[00:25:30] Steering Out-of-Distribution Generalization with Concept Ablation Fine-Tuning
https://arxiv.org/abs/2507.16795
[00:29:30] Persona Vectors: Monitoring and Controlling Character Traits in Language Models
https://arxiv.org/abs/2507.21509
[00:41:14] Features as Rewards: Scalable Supervision for Open-Ended Tasks via Interpretability
https://arxiv.org/abs/2602.10067
[00:47:03] Anatomy of Post-Training: Using Interpretability to Characterize Data and Shape the Learning Signal
https://arxiv.org/abs/2606.12360
[01:00:26] Do Sparse Autoencoders Capture Concept Manifolds?
https://arxiv.org/abs/2604.28119
[01:03:04] Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behavior
https://arxiv.org/abs/2605.05115
[01:14:20] Arithmetic in the Wild: Llama uses Base-10 Addition to Reason About Cyclic Concepts
https://arxiv.org/abs/2605.01148
[01:29:35] Measuring Reward-Seeking via Contrastive Belief Updates
https://arxiv.org/abs/2607.18966v1
other:
[00:15:44] Intentional Design
https://www.goodfire.com/blog/intentional-design
[00:56:12] The World Inside Neural Networks
https://www.goodfire.com/research/the-world-inside-neural-networks
[01:37:28] A Pragmatic Vision for Interpretability
https://www.alignmentforum.org/posts/StENzDcD3kpfGJssR/a-pragmatic-vision-for-interpretability
---
RESCRIPT:
https://app.rescript.info/share/846cfee4131b664fd09209cc3b98018e
Stealing Reasoning Traces from Proprietary LLM APIs — Ilia Shumailov & Alexander Panfilov
Tim Scarfe speaks with Ilia Shumailov and Alexander Panfilov about their paper, Stealing Reasoning Traces from Proprietary LLM APIs.The core bug sounds deceptively simple: providers return encrypted reasoning …
Notes de l’épisode
Tim Scarfe speaks with Ilia Shumailov and Alexander Panfilov about their paper, Stealing Reasoning Traces from Proprietary LLM APIs.The core bug sounds deceptively simple: providers return encrypted reasoning state so conversations can be resumed or forked. But those blobs can be replayed across users and sibling models. A smaller model can ask the provider to decrypt the trace, then repeat the hidden reasoning in plain text. The discussion covers leaked private data, a broadly reusable jailbreak, poisoned agent traces, chain-of-thought monitoring, responsible disclosure, and possible defenses.Ilia Shumailov is an AI and security researcher, formerly at Google DeepMind, who completed his Cambridge PhD under Ross Anderson. Alexander Panfilov is a PhD researcher at the ELLIS Institute Tübingen and the Max Planck Institute for Intelligent Systems, working on AI safety, adversarial machine learning, and LLM red-teaming. They close by separating the demonstrated jailbreaking threat from ordinary benign distillation, and by arguing for controlled experiments over sweeping claims.---TIMESTAMPS:00:00:00 Intro montage00:01:33 Portable encrypted thought and decoded reasoning00:24:55 How the attack works and what it means00:39:04 Doom, defense, and scientific restraint---REFERENCES:paper:[00:00:00] Stealing Reasoning Traces from Proprietary LLM APIshttps://arxiv.org/abs/2608.09867[00:09:22] Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safetyhttps://arxiv.org/abs/2507.11473[00:11:30] Reasoning Models Don’t Always Say What They Thinkhttps://www.anthropic.com/research/reasoning-models-dont-say-think[00:37:22] PostTrainBench: Can LLM Agents Automate LLM Post-Training?https://arxiv.org/abs/2603.08640[00:41:02] Large-scale online deanonymization with LLMshttps://arxiv.org/abs/2602.16800other:[00:09:28] OpenAI and Hugging Face partner to address security incident during model evaluationhttps://openai.com/index/hugging-face-model-evaluation-security-incident/[00:10:22] Claude, GPT, and Gemini All Struggle to Evade Monitorshttps://metr.org/notes/2025-08-22-claude-gpt-gemini-struggle-evade-monitors/tool:[00:42:08] Isabelle proof assistanthttps://isabelle.in.tum.de/---RESCRIPT: https://app.rescript.info/share/07fc38276e0823dc9b8986c32e202c7f
Every Exponential Ends — Silicon Valley Forgot — Adam Becker
Astrophysicist Adam Becker, author of "What Is Real?", joins Tim Scarfe to take apart the futures Silicon Valley keeps selling: the 2045 singularity, mind uploading, Mars colonies, and the AI apocalypse. His …
Notes de l’épisode
Astrophysicist Adam Becker, author of "What Is Real?", joins Tim Scarfe to take apart the futures Silicon Valley keeps selling: the 2045 singularity, mind uploading, Mars colonies, and the AI apocalypse. His new book *More Everything Forever* argues these ideas are hugely influential, mostly evidence-free, and bankrolled by tech billionaires who need a story in which growth never ends.Becker does the physics the boosters skip. Kurzweil's "law of accelerating returns" rests on cherry-picked data, and every exponential ends. Grant Bezos his perpetual energy growth and humanity boils the oceans within a few centuries, then exhausts the observable universe in under 4,000 years. The stars are too far away, Mars dirt is poison, and the day the dinosaur-killing asteroid hit Earth was still nicer than any day on Mars. On AI, Becker calls LLMs pocket calculators for language: hallucination is the model doing exactly what it always does, and the intelligence explosion assumes intelligence is a single number you can buy with compute.The sting is that Becker thinks the doomers are sincere. Yudkowsky, Bostrom and the effective altruists are not grifters, he says, just wrong, and their warnings that AI could end the world feed the same growth story the money depends on. He closes with his own prescription: take social problems seriously, regulate the whole tech industry, and tax billionaires out of existence.---TIMESTAMPS:00:00:00 Cold open and the thesis of More Everything Forever00:04:24 Kurzweil's singularity and the physical limits of exponential growth00:14:02 High agency and the fantasy of imprinting humanity on the cosmos00:16:55 Mind uploading, functionalism, and embodied cognition00:24:24 AI psychosis and anthropomorphizing LLMs00:26:24 Calculators, hallucination, and the limits of scale00:32:20 Yudkowsky and the intelligence-explosion argument00:40:37 True believers, venture capital, and the sci-fi growth narrative00:47:21 From Extropians to EA: utilitarianism and longtermism00:53:50 Brain worms and Becker's prescription: take social science seriously00:56:49 Why the AI-ethics discourse is broken01:01:42 The eugenics and IQ argument against 'intelligence'01:06:07 Why space settlement fails: Mars, the moon, and orbital data centers01:10:42 Billionaire myths and the search for purpose01:13:38 Tax billionaires, regulate tech: closing prescriptions---REFERENCES:book:[00:00:07] More Everything Forever (Adam Becker, 2025)https://www.hachettebookgroup.com/titles/adam-becker/more-everything-forever/9781541619593/[00:00:15] What Is Real? (Adam Becker, 2018)https://en.wikipedia.org/wiki/What_Is_Real%3F[00:15:46] What We Owe the Future (Will MacAskill, 2022)https://www.hachettebookgroup.com/titles/william-macaskill/what-we-owe-the-future/9781541618626/other:[00:00:27] Dreaming Against the Machine (podcast)https://www.dreamingagainstthemachine.com[00:01:04] The Useful Idiots of AI Doomsaying (Adam Becker, The Atlantic, 2025)https://www.theatlantic.com/books/archive/2025/09/what-ais-doomers-and-utopians-have-in-common/684270/
RESCRIPT: https://app.rescript.info/share/d6e37f9866673d8f74a39076efa5926b
AI Is Learning at the Wrong Level of Abstraction — Matthieu Wyart
This episode is sponsored by Notion. Learn more about Notion's Developer Platform today at https://notion.com/mlstWhy can deep networks discover abstractions that shallow models miss? Statistical physicist …
Notes de l’épisode
This episode is sponsored by Notion. Learn more about Notion's Developer Platform today at https://notion.com/mlstWhy can deep networks discover abstractions that shallow models miss? Statistical physicist Matthieu Wyart joins Tim Scarfe to argue that the answer lies in the hidden hierarchy of data. Language and images are built from parts within parts; depth lets a network recover those coarse-grained variables and escape the curse of dimensionality.The conversation moves from jamming transitions and rough loss surfaces to Chomsky, context-free grammars and machine creativity. Wyart explains why next-token prediction can still recover compositional structure, where current systems fall short of genuine scientific invention, and why predicting latent representations rather than raw tokens could make learning far more sample-efficient.They also examine diffusion models, neural scaling laws and the limits of physics-inspired theory. The final question is on a personal note: if mistakes are the price of leaving the beaten path, how much scientific risk is worth taking?---TIMESTAMPS:00:00:00 Can machines learn abstractions from data?00:02:00 Notion agentic workspace00:02:49 From statistical physics to machine learning00:06:40 What physics can explain about learning00:16:37 From Carnot to Chomsky bulldozer00:21:21 How deep networks recover hidden hierarchies00:32:43 Where machine creativity still falls short00:40:48 How deep nets escape the curse of dimensionality00:52:19 Why predict latents instead of tokens01:02:49 The sample-efficiency case for latent prediction01:08:31 Diffusion, scaling laws and text entropy01:16:40 The scientists we learn from and the mistakes we make---REFERENCES:person:[00:00:43] Noam Chomskyhttps://linguistics.mit.edu/user/chomsky/tool:[00:02:08] Notion Developer Platformhttps://www.notion.com/en-gb/blog/introducing-developer-platformpaper:[00:04:43] Mastering the game of Go with deep neural networks and tree searchhttps://www.nature.com/articles/nature16961[00:05:52] Reconciling modern machine-learning practice and the bias-variance trade-offhttps://arxiv.org/abs/1812.11118[00:25:54] How Deep Neural Networks Learn Compositional Data: The Random Hierarchy Modelhttps://arxiv.org/abs/2307.02129[00:42:12] Efficient Estimation of Word Representations in Vector Spacehttps://arxiv.org/abs/1301.3781[00:52:46] Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturehttps://arxiv.org/abs/2301.08243[00:52:54] Learn from your own latents and not from tokens: A sample-complexity theoryhttps://arxiv.org/abs/2605.27734[01:08:31] A Phase Transition in Diffusion Models Reveals the Hierarchical Nature of Datahttps://arxiv.org/abs/2402.16991[01:11:39] Scaling Laws for Neural Language Modelshttps://arxiv.org/abs/2001.08361[01:12:17] Deriving Neural Scaling Laws from the statistics of natural languagehttps://arxiv.org/abs/2602.07488[01:13:34] Prediction and Entropy of Printed Englishhttps://ieeexplore.ieee.org/document/6773263---LINKS:Download PDF transcript: https://app.rescript.info/share/f7644cdaa86c5cc1e41e484e290f2bd4
How Researchers Test AI for Hidden Goals — Apollo Research
Can an AI do the right thing for the wrong reason? Tim Scarfe speaks with Apollo Research’s Alexander Meinke, Axel Højmark and Jérémy Scheurer about Measuring Reward-Seeking via Contrastive Belief Updates …
Notes de l’épisode
Can an AI do the right thing for the wrong reason? Tim Scarfe speaks with Apollo Research’s Alexander Meinke, Axel Højmark and Jérémy Scheurer about Measuring Reward-Seeking via Contrastive Belief Updates, their new research with OpenAI.
The panel asks how models infer what graders reward, why good behaviour can come from the wrong reason, and whether that difference can be measured. The conversation moves through promise-breaking, grader awareness, reward hacking, scheming, opaque reasoning and corrigibility, then turns to a detailed walkthrough of the contrastive-belief method and what its results do and do not show. The o3 results discussed here concern an intermediate checkpoint without safety training.
This episode was made in partnership with Apollo Research. MLST retained full editorial control.
Reference
Apollo Research: https://www.apolloresearch.ai/
---
TIMESTAMPS:
00:00:00 Cold Open
00:02:12 Right Things, Wrong Reasons
00:12:47 Grader Awareness
00:26:22 Legibility
00:32:35 What To Call It
00:35:58 Intelligence, Agency, Anthropomorphism
00:45:16 Apollo’s Mission
00:48:54 The End of the Exponential
00:55:45 The Paper
01:16:34 Closing Reflection
---
REFERENCES:
tool:
[00:00:08] Claude Fable
https://www.anthropic.com/claude/fable
[00:12:50] AlphaGo Zero
https://deepmind.google/blog/alphago-zero-starting-from-scratch/
[00:44:30] AlphaFold 3
https://deepmind.google/science/alphafold/
paper:
[00:01:02] Measuring Reward-Seeking via Contrastive Belief Updates
https://arxiv.org/abs/2607.18966
[00:16:19] Natural Language Autoencoders Produce Unsupervised Explanations of LLM Activations
https://transformer-circuits.pub/2026/nla/
[00:26:48] Stress Testing Deliberative Alignment for Anti-Scheming Training
https://arxiv.org/abs/2509.15541
[00:35:33] Shortcut learning in deep neural networks
https://arxiv.org/abs/2004.07780
[00:53:49] Measuring AI Ability to Complete Long Software Tasks
https://arxiv.org/abs/2503.14499
[00:59:52] Modifying LLM Beliefs with Synthetic Document Finetuning
https://alignment.anthropic.com/2025/modifying-beliefs-via-sdf/
[01:10:44] Alignment Faking in Large Language Models
https://arxiv.org/abs/2412.14093
[01:13:55] Natural Emergent Misalignment from Reward Hacking
https://www.anthropic.com/research/emergent-misalignment-reward-hacking
other:
[00:10:14] We Need a Science of Scheming
https://www.apolloresearch.ai/science/science-of-scheming/
[00:32:56] CoastRunners reward hacking example
https://deepmind.google/blog/specification-gaming-the-flip-side-of-ai-ingenuity/
organization:
[01:06:07] Redwood Research
https://www.redwoodresearch.org/
---
ReScript:
https://app.rescript.info/share/718ab68e18cfa3b9b800da6b3290fd42
Why a Nation Can't Outsource Its Frontier AI - Alistair Pullen (Cosine AI)
This episode is sponsored by Notion. Learn more about Notion's Developer Platform today at https://notion.com/mlstBritain's most capable coding model can't be exported, and that ban is the whole reason Cosine …
Notes de l’épisode
This episode is sponsored by Notion. Learn more about Notion's Developer Platform today at https://notion.com/mlst
Britain's most capable coding model can't be exported, and that ban is the whole reason Cosine set out to build one from scratch. Alistair Pullen, CEO and co-founder of Cosine, sits down with Tim Scarfe to explain how a frontier system he calls Fable, locked behind US export controls, became the founding case for a UK sovereign model trained on the Isambard supercomputer in Bristol.
The bet underneath it is economic. Pullen argues that an inference company, rather than a training-first lab, doesn't need billions to compete: millions, a national compute allocation, and a consortium feedback loop can be enough. From there it gets into the machinery, why open-weight models still trail the frontier on size, active parameters and data, the mixture-of-experts versus dense trade-off and why active params dominate how a model actually feels, and the edge that real coding trajectories confer.
The back half is about making agents trustworthy. Pullen makes the case for beating "slop" by rewarding the process instead of the final answer, reframes code review as runtime proof (spin the bug up in a VM and force the agent to actually exploit it), and walks through Swarm, Cosine's system running hundreds of sub-agents in one shot. It ends on why memory is still an unsolved hack, how synthetic graders let you run RL on tasks with no built-in test, and why Pullen reads US export controls as an accidental gift, with a supply-chain sting in the tail.
---
TIMESTAMPS:
00:00:00 The sovereign mandate and the Fable ban
00:04:02 Millions vs billions: the inference-company model
00:07:19 The consortium feedback loop
00:07:40 Why open models lag the frontier
00:14:59 MoE vs dense, and why active params matter
00:16:29 Trajectories: the process-data advantage
00:19:48 Beating slop: reward the process, not the answer
00:26:06 Reusable abstractions and the epistemic wall
00:29:56 Code review becomes runtime proof
00:37:32 Do agentic harnesses still matter?
00:40:35 Swarm: orchestrating hundreds of sub-agents
00:45:14 Why memory is still unsolved
00:48:25 Synthetic data and graders for RL
00:53:09 The US export gift and supply-chain risk
---
REFERENCES:
organization:
[00:01:15] Cosine
https://cosine.sh
[00:04:14] Mistral AI
https://mistral.ai
[00:05:50] Anthropic
https://www.anthropic.com
[00:07:42] Cohere
https://cohere.com
[00:08:36] DeepSeek
https://www.deepseek.com
tool:
[00:02:52] Isambard-AI
https://isambard.ac.uk
[00:05:56] Colossus (xAI)
https://en.wikipedia.org/wiki/Colossus_(supercomputer)
[00:07:52] GLM (Z.ai)
https://z.ai
[00:11:52] NVIDIA B300
https://www.nvidia.com/en-us/data-center/dgx-b300/
[00:15:37] gpt-oss-120b
https://huggingface.co/openai/gpt-oss-120b
[00:15:52] Devstral 2
https://mistral.ai/news/devstral
[00:16:01] Llama 70b
https://www.llama.com
[00:17:05] Claude Code
https://www.anthropic.com/claude-code
[00:26:23] ARC-AGI (Francois Chollet)
https://arcprize.org
[00:40:38] Swarm (Cosine)
https://cosine.sh
[00:40:50] OpenAI Codex
https://github.com/openai/codex
[00:41:16] Lumen Outpost (Cosine)
https://cosine.sh
[00:41:18] Kimi K2 (Moonshot)
https://huggingface.co/moonshotai/Kimi-K2-Instruct
[00:49:55] SWE-bench
https://www.swebench.com
[00:52:40] SystemVerilog
https://en.wikipedia.org/wiki/SystemVerilog
person:
[00:23:40] Andrej Karpathy
https://karpathy.ai
paper:
[00:27:10] GRPO (DeepSeekMath)
https://arxiv.org/abs/2402.03300
[00:27:13] GSPO
https://arxiv.org/abs/2507.18071
Incompressible Knowledge Probes, Bojie Li
https://arxiv.org/pdf/2604.24827
Estimating the Size of Claude Opus 4.5/4.6
https://unexcitedneurons.substack.com/p/estimating-the-size-of-claude-opus
---
ReScript:
https://app.rescript.info/session/5852d2b884c4ce4b?share=10b9799160845bb11779f8ac6cd3124f
The Benchmark With No Instructions — ARC-AGI-3 (winning team!)
Tim Scarfe travels to Zurich to sit down with the Tufa Labs ARC-AGI-3 team — founder Benjamin Crouzier, with Jeroen Cottaar, Dries Smit, Stefano Viel and Michal Tesnar — to work out what their …
Notes de l’épisode
Tim Scarfe travels to Zurich to sit down with the Tufa Labs ARC-AGI-3 team — founder Benjamin Crouzier, with Jeroen Cottaar, Dries Smit, Stefano Viel and Michal Tesnar — to work out what their leaderboard-topping system does and what the benchmark is really testing.The cut opens on the games: a walkthrough of the Locksmith game, where you read the rules of an unfamiliar world straight from raw frames. ARC-AGI-3 makes ARC interactive and agentic, so the model has to *discover* the goal rather than transduce a static grid. It stays easy for humans and breaks LLMs, and it runs through everything that follows. Dries traces his StochasticGoose preview win — brute force that only searched actions which changed the frame — and why it collapsed once the organisers added action-efficiency scoring and unseen games.Induction and transduction run through the middle of the conversation — how much of an answer is really priors leaking back the moment a model recognises a maze. The abstraction mountain, and Tim's case that LLMs reach the right answer through fractured, entangled representations — performance, not competence. Whether transformers plan at all or just fake it well enough. Why the score really measures action efficiency, not games solved, and why agents lock onto the wrong goal and cannot climb back out.Crouzier closes on the Tufa Labs thesis — a small lab against the giants, the bitter lesson against hand-built harnesses, and safety — and Tim ties it back to Kenneth Stanley, deep constraints, and creativity as competence.
Disclosure: Tufa Labs sponsors MLST. ---TIMESTAMPS:00:00:00 Meet the Tufa team and what makes ARC-AGI-3 hard00:02:11 Locksmith game: reading the rules from raw frames00:03:10 Why build an independent research lab00:04:11 StochasticGoose: a preview win, then the hardened games00:07:58 Induction, transduction, and priors inside LLMs00:10:31 Curiosity, world models, and exploring by frame change00:14:32 Understanding debt and losing sight of your own code00:15:53 Requirements-based agents and human-AI co-creativity00:19:22 Why auto-research misses the big picture00:21:54 The abstraction mountain and fractured representations00:27:36 Constraints and making LLMs act as if they understand00:34:51 Human difficulty calibration, esports priors, and emergence00:41:35 Agency, goal acquisition, and two kinds of planning00:47:31 Harnesses, the 36% number, and wrong-goal loops00:52:33 Rewards, goals, and why ARC-AGI-3 resists brute force01:00:46 Would solving ARC-AGI-3 prove AGI?01:07:53 Stripping language away, then priors leak back01:14:06 Representation and whether language is necessary01:18:04 The bitter lesson versus specialised harnesses01:22:20 Capability research, safety, and the software singularity---REFERENCES:organization:[00:02:11] ARC-AGI-3https://arcprize.org/arc-agi/3[00:03:10] Tufa Labshttps://tufalabs.ai/team/[00:04:20] ARC-AGI-3 Preview Agent Competitionhttps://arcprize.org/competitions/arc-agi-3-preview-agentstool:[00:04:55] StochasticGoose ARC-AGI-3 solutionhttps://github.com/DriesSmit/ARC3-solution[00:07:42] ArcGenticahttps://github.com/symbolica-ai/arcgentica[00:07:49] RGB-Agenthttps://github.com/alexisfox7/RGB-Agent[00:14:38] Claude Codehttps://www.anthropic.com/claude-code[01:03:42] Qwen 3.6 27Bhttps://huggingface.co/Qwen/Qwen3.6-27Bpaper:[00:13:03] On the Measure of Intelligencehttps://arxiv.org/abs/1911.01547[00:27:42] DreamCoderhttps://arxiv.org/abs/2006.08381[00:43:55] On the Biology of a Large Language Modelhttps://transformer-circuits.pub/2025/attribution-graphs/biology.html[01:18:46] ImageNet Classification with Deep CNNs (AlexNet)https://proceedings.neurips.cc/paper_files/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdfother:[01:18:16] The Bitter Lessonhttp://www.incompleteideas.net/IncIdeas/BitterLesson.html---https://app.rescript.info/share/463d7f031349b4b9db428553eed88230
The Thermodynamic AI Computing Chip - Thomas Ahle
Thomas Ahle wants Normal Computing to be the Lovable for chip design: type your intent, and a swarm of agents carries it from design through optimisation, formalisation and verification to tape-out. To get …
Notes de l’épisode
Thomas Ahle wants Normal Computing to be the Lovable for chip design: type your intent, and a swarm of agents carries it from design through optimisation, formalisation and verification to tape-out. To get there, his team at wrote their own open-source Verilog simulator, 580,000 lines in 43 days, because commercial EDA verifiers run about $10,000 per core and there are no decent open-source compilers to build on.
That sets up the question Tim keeps pressing: if an agent can produce a chip design, a proof, or a working program, how do you actually know it is correct? Passing 70% of tests is not the same as being right, and a single fabricated bug can cost a company a fortune. They dig into ProgramBench (rebuild a program from its tests, roughly 0% success), the difference between structure and competence, and the "understanding debt" you take on when nobody reads the code.
From there: auto-formalisation in Lean and the AlphaProof trick of training on prove-or-disprove; why there is no single true representation of a spec (Petri nets, TLA+, Erik Curiel's "math does not represent"); and thermodynamic computing, where Normal Computing's CN101 chip is built so that its physical noise *is* the computation, settling a stochastic differential equation in hardware to invert a matrix. Plus Bayesian uncertainty, specialisation, the Chomsky hierarchy, AI slop, and whether performance is all that matters.
Recorded in Zurich.
Disclosure: Normal Computing paid our production and travel costs for this show. We retained full editorial control. They did not see the video before publication, and we did not show it to them or discuss it with them beforehand.
---
TIMESTAMPS:
00:00:00 Meet Thomas Ahle: the Lovable for chip design
00:03:41 Why hardware needs formal verification
00:06:36 Ten thousand dollars per core and a six-month agent run
00:07:40 Rebuilding programs from tests: ProgramBench and zero percent
00:12:15 Structure vs competence: can you learn a program from behavior?
00:15:27 Continual learning, abstraction, and Claude as an ecosystem
00:23:17 Autoformalization and the AlphaProof trick
00:29:31 No single true representation: specs, Petri nets and TLA+
00:34:43 Thermodynamic computing: when noise is the computation
00:37:32 Bayesian uncertainty in the age of token streams
00:41:12 Hybrid compute: vibe-coding loops, binaries and Stockfish
00:44:44 Co-design, central-AI apps and API pricing
00:49:45 Chain of thoughtlessness and the Chomsky hierarchy
00:53:40 AI psychosis, slop and the broken social contract
00:57:34 Typing it yourself, teamwork and performance vs competence
---
REFERENCES:
person:
[00:00:10] Thomas Ahle
https://thomasahle.com
organization:
[00:00:27] Normal Computing
https://normalcomputing.com/
paper:
[00:11:21] ProgramBench: Can Language Models Rebuild Programs From Scratch?
https://arxiv.org/abs/2605.03546
[00:31:55] Autoformalizing Memory Device Specifications with Agents
https://arxiv.org/abs/2605.00058
[00:35:20] Thermo AI and the Fluctuation Frontier
https://arxiv.org/abs/2302.06584
[00:36:40] Thermo Comp System for AI Applications
https://arxiv.org/abs/2312.04836
[00:37:05] Thermodynamic Linear Algebra
https://arxiv.org/abs/2308.05660
[00:44:50] An efficient probabilistic hardware architecture for diffusion-like models
https://arxiv.org/abs/2510.23972
tool:
other:
[00:01:00] Building an Open-Source Verilog Simulator with AI: 580K Lines in 43 Days
https://normalcomputing.com/blog/building-an-open-source-verilog-simulator-with-ai-580k-lines-in-43-days
[00:02:55] Normal Computing Announces Tape-Out of the World's First Thermodynamic Computing Chip (CN101)
https://www.normalcomputing.com/blog/normal-computing-announces-tape-out-of-worlds-first-thermodynamic-computing-chip
[00:32:02] DRAMBench: Autoformalizing DRAM Specifications with Timed Petri Nets
https://www.iese.fraunhofer.de/blog/drambench-autoformalizing-dram-specifications/
---
ReScript: https://app.rescript.info/share/ff9684a112ab37744096adaeb097a263
He won a Nobel here for AlphaFold. Then he left. - John Jumper
This episode is sponsored by Notion. Learn more about Notion's Developer Platform today at https://notion.com/mlstProtein folding stalled biology for fifty years. A sequence of amino acids dictates a …
Notes de l’épisode
This episode is sponsored by Notion. Learn more about Notion's Developer Platform today at https://notion.com/mlstProtein folding stalled biology for fifty years. A sequence of amino acids dictates a three-dimensional shape, but reading that shape meant a year and roughly $100,000 of crystallography per structure. Then AlphaFold 2 won CASP14 so decisively the organizers called the problem essentially solved.In this documentary cut, John Jumper, who shared the 2024 Nobel Prize in Chemistry and has since left DeepMind for Anthropic, walks Tim Scarfe through what the system did and, more interestingly, what it did not. The architecture gets a proper dissection: MSAs, the Evoformer, invariant point attention, the FAPE loss, and Jumper's correction of the equivariance story, which ablations valued at roughly 2.5 of 30 GDT points rather than the whole win. He is blunt about the limits. AlphaFold predicts one experiment extraordinarily well; it is not a model of the cell, it does not capture dynamics, and on a given drug target it is "wrong nine times out of ten."From there: the AlphaFold Database of 200M+ predicted structures, AlphaFold 3 and ligands, Isomorphic Labs, and Jumper's quarrel with the bitter lesson, where finite data and human hypotheses still matter. Emmanuel Nji of BioStruct Africa closes the film on what changes when work that took years now takes months, and on training the next thousand structural biologists across Africa.---TIMESTAMPS:00:00:00 Cold open: predicting nature with a button press00:01:03 The protein folding bottleneck and CASP00:04:39 The Nobel, the database, and the move to Anthropic00:05:50 Sponsor (Notion) and framing: what AlphaFold does not claim00:07:39 Proteins as self-assembling nanomachines00:12:24 From structures to biology: drug discovery and Midnolin00:17:37 The humility of AlphaFold: a narrow predictor00:22:18 Inside the architecture: Evoformer, IPA and FAPE00:30:20 Ruthless empiricism: ablations and 100x in data00:35:20 Predict, control, understand00:40:00 Against the bitter lesson; AlphaFold 3 as diffusion00:45:07 Intelligence, representations and AGI00:49:23 Epilogue: AlphaFold in Africa00:52:16 Closing: the case for hybrid science models---REFERENCES:organization:[00:01:55] Critical Assessment of Structure Prediction (CASP)https://predictioncenter.org/[00:04:39] The Nobel Prize in Chemistry 2024https://www.nobelprize.org/prizes/chemistry/2024/summary/[00:05:18] BioStruct Africahttps://www.biostructafrica.org/[00:18:03] Isomorphic Labshttps://www.isomorphiclabs.com/paper:[00:03:09] AlphaFold Protein Structure Databasehttps://doi.org/10.1093/nar/gkab1061[00:17:25] Accurate structure prediction of biomolecular interactions with AlphaFold 3https://www.nature.com/articles/s41586-024-07487-w[00:22:18] Highly accurate protein structure prediction with AlphaFoldhttps://www.nature.com/articles/s41586-021-03819-2[00:23:10] Midnolin promotes degradation of substrates independent of ubiquitinationhttps://doi.org/10.1126/science.adh5021[00:27:00] Improved protein structure prediction using potentials from deep learninghttps://www.nature.com/articles/s41586-019-1923-7tool:[00:03:09] AlphaFold Protein Structure Database (EBI)https://alphafold.ebi.ac.uk/[00:45:55] AlphaEvolve: a coding agent for designing advanced algorithmshttps://deepmind.google/blog/alphaevolve-a-gemini-powered-coding-agent-for-designing-advanced-algorithms/other:[00:39:40] The Bitter Lessonhttp://www.incompleteideas.net/IncIdeas/BitterLesson.html---ReScript: https://app.rescript.info/share/d8cde5c221fb71e2c0f5aafe94f90dfaDisclaimer - not sponsored, editorial with us - we filmed it at GDM, London
When AI Decides You're a Threat — Brad Carson
Brad Carson was the Army's General Counsel, served two terms in Congress and was Acting Under Secretary of Defense for Personnel and Readiness. He now heads Americans for Responsible Innovation, the AI-policy …
Notes de l’épisode
Brad Carson was the Army's General Counsel, served two terms in Congress and was Acting Under Secretary of Defense for Personnel and Readiness. He now heads Americans for Responsible Innovation, the AI-policy advocacy group he co-founded. Keith Duggar spends roughly eighty minutes pushing back.
SPONSOR:
---
Cyber Fund built the Monastery to help founders ship products that were impossible a year ago. Applications for Batch 1 are now open.
Apply now: https://cyber.fund
---
Carson's whole case rests on one line: the genie is not out of the bottle. We have pulled dangerous tech back before. Asilomar halted recombinant DNA in 1975, and the West still controls the chips AI runs on. Calling it unstoppable, he says, is the most dangerous idea in the room.
Then Keith drags him somewhere darker. A Palantir heat map scores you 0.73 on whether you are a combatant, and a strike follows. The model is wrong some accepted share of the time, and when it is, nobody answers for it. You cannot court-martial a model, and not even the interpretability researchers can say why it picked you.
—
Note: after recording, we learned that Americans for Responsible Innovation is backed by EA-aligned philanthropy (not sponsored)
---
TIMESTAMPS:
00:00:00 From the Pentagon to AI governance
00:04:52 Regulatory capture vs Silicon Valley networks
00:07:56 Transparency and the Claude tier changes
00:09:40 Tort liability when AI tools cause harm
00:13:40 AI is a product, not a person
00:16:01 Children, suicide, and the suicide business
00:19:59 Opaque neural nets and the law of war
00:25:54 Probabilistic targeting and the death of accountability
00:28:47 The arms race fallacy: Asilomar and restraint
00:34:02 Talking to China: track 2 talks and chip leverage
00:39:45 Air power never wins: capital for labour
00:43:29 Anthropic vs the Department of War
00:51:29 Concentration, open source, and brain drain
01:00:18 DeepSeek, Chinese culture, and AI as diplomacy
01:12:25 Upskilling Congress and why public trust matters
---
REFERENCES:
organization:
[00:02:45] ICRC position on autonomous weapons
https://www.icrc.org/en/law-and-policy/autonomous-weapons
[00:05:22] Americans for Responsible Innovation (ARI)
https://ari.us
[00:07:20] Andreessen Horowitz (a16z)
https://a16z.com/
[01:16:05] Office of Technology Assessment
https://en.wikipedia.org/wiki/Office_of_Technology_Assessment
other:
[00:03:35] Beneficial AGI 2019 Conference (Future of Life Institute, Puerto Rico)
https://futureoflife.org/event/beneficial-agi-2019/
[00:18:30] Section 230 of the Communications Decency Act
https://en.wikipedia.org/wiki/Section_230
[00:19:59] Lethal Autonomous Weapons (LAWS)
https://en.wikipedia.org/wiki/Lethal_autonomous_weapon
[00:31:35] Strategic Arms Limitation Talks (SALT)
https://en.wikipedia.org/wiki/Strategic_Arms_Limitation_Talks
[00:32:28] Asilomar Conference on Recombinant DNA (1975)
https://en.wikipedia.org/wiki/Asilomar_Conference_on_Recombinant_DNA
[00:39:45] The New Iron Triangle (ARI policy byte)
https://ari.us/policy-bytes/the-new-iron-triangle/
[00:48:05] Defense Production Act
https://en.wikipedia.org/wiki/Defense_Production_Act
person:
[00:03:35] Anthony Aguirre
https://en.wikipedia.org/wiki/Anthony_Aguirre
[00:06:48] Dean Ball — Hyperdimensional
https://www.hyperdimensional.co/
[00:23:13] Neel Nanda — mechanistic interpretability
https://www.neelnanda.io/
[00:36:02] Jack Clark (Anthropic) on Conversations with Tyler
https://conversationswithtyler.com/episodes/jack-clark/
[00:39:15] Robert Trager — Centre for the Governance of AI
https://www.governance.ai/team/robert-trager
[00:41:55] Giulio Douhet
https://en.wikipedia.org/wiki/Giulio_Douhet
[01:15:05] Don Beyer (US Congress)
https://en.wikipedia.org/wiki/Don_Beyer
tool:
[00:22:19] Phalanx CIWS
https://en.wikipedia.org/wiki/Phalanx_CIWS
---
ReScript:
https://app.rescript.info/public/share/9405ff35c0215b7cdae6402d41284171
https://app.rescript.info/api/public/sessions/0a6c081b8e5fe413/pdf
Intelligence is collective, not artificial — Prof. Michael I. Jordan (UC Berkeley / Inria)
Michael I. Jordan, described by Science magazine as the most influential computer scientist alive, has never thought of himself as an AI researcher. In this conversation he explains why that distinction …
Notes de l’épisode
Michael I. Jordan, described by Science magazine as the most influential computer scientist alive, has never thought of himself as an AI researcher. In this conversation he explains why that distinction matters.
SPONSOR:
---
Cyber Fund built the Monastery to help founders ship products that were impossible a year ago. Applications for Batch 1 are now open.
Apply now: https://cyber.fund
---
Jordan trained as a statistician and cognitive scientist, and his career has been spent building machine learning systems that work in the real world: supply chains, commerce, healthcare, and large economic systems. When the field rebranded itself as AI and then AGI, he did not follow. Instead he argues that the framing is wrong. AI is better understood as a collective economic system than as a race to build a disembodied superintelligence.
We talk about why AGI is mostly a PR term, what machine learning achieved before the LLM hype cycle, and why the assistant-on-your-shoulder vision may be less compelling than it sounds. Jordan explains why explanations need to be actionable, not merely mechanistic; why AlphaFold's missing error bars matter; how prediction-powered inference changes the picture; and why drug discovery is an incentive-design problem rather than a pure pattern-matching problem.
ERRATA: Science magazine ranked him the most influential computer scientist, not Nature
---
TIMESTAMPS:
00:00:00 Cold open: A demoralizing message to young builders
00:02:04 CyberFund sponsor read
00:02:50 From symbolic AI to machine learning systems
00:05:42 Why AGI is mostly a PR term
00:08:48 A collectivist, economic perspective on AI
00:11:33 Why LLMs need system design, not hype
00:14:50 Predictability beats faux understanding
00:17:55 AlphaFold, bias, and prediction-powered inference
00:21:48 Stop anthropomorphizing intelligence
00:27:44 Drug discovery as an incentive problem
00:32:29 The three-layer data market
00:38:07 Social knowledge, markets, and culture
00:45:39 Creator economics beyond Spotify
00:48:30 How science-fiction AI narratives mislead young builders
00:51:45 AI should improve humans, not replace them
00:56:42 Safety is a property of the whole system
00:58:12 Silicon Valley gurus and the cream off the top
01:00:47 Game theory, mechanism design, and contracts
01:04:39 Conformal prediction, e-values, and anytime inference
01:08:11 A new liberal arts triangle for the AI era
01:11:30 The Bayesian duck and markets as uncertainty reduction
ReScript (transcript, PDF, refs etc) - https://app.rescript.info/public/share/fb68f94af29d3745c6cf6125e01328b5
---
REFERENCES:
person:
[00:02:50] Michael I. Jordan (homepage)
https://people.eecs.berkeley.edu/~jordan/
paper:
[00:06:01] A Collectivist, Economic Perspective on AI
https://arxiv.org/abs/2507.06268
[00:18:09] AlphaFold
https://www.nature.com/articles/s41586-021-03819-2
[00:20:36] Prediction-Powered Inference
https://arxiv.org/abs/2301.09633
[00:33:47] On Three-Layer Data Markets
https://arxiv.org/abs/2402.09697
[01:04:39] Conformal Prediction with Conditional Guarantees
https://arxiv.org/abs/2107.07511
[01:04:51] A Tutorial on Conformal Prediction
https://www.jmlr.org/papers/v9/shafer08a.html
[01:06:00] E-Values Expand the Scope of Conformal Prediction
https://arxiv.org/abs/2503.13050
[01:08:23] Computational Thinking
https://www.cs.cmu.edu/~CompThink/papers/Wing06.pdf
other:
[00:28:20] How Should the FDA Test?
https://rdi.berkeley.edu/events/sbc-assets/pdfs/Summit%20session%20speaker%20slides%20submission%20form-s1-5%20%28File%20responses%29/Slides%20in%20PDF%20%28Please%20name%20the%20submitted%20file%20as%20_firstname_-_lastname_-slides.pdf%29.%20%28File%20responses%29/27-Michael%20Jordan-Session%20V.pdf#page=15
[00:28:40] Michael I. Jordan Session V Slides
<truncated, see ReScript link or YT VD>
The AI Models Smart Enough to Know They're Cheating — Beth Barnes & David Rein [METR]
Beth Barnes and David Rein on the one graph that ate the AI timelines discourse, and why the two people who built it are the most careful about how you read it.**SPONSOR**Prolific - Quality data. From real …
Notes de l’épisode
Beth Barnes and David Rein on the one graph that ate the AI timelines discourse, and why the two people who built it are the most careful about how you read it.**SPONSOR**Prolific - Quality data. From real people. For faster breakthroughs.https://www.prolific.com/?utm_source=mlstInterview: https://youtu.be/cnxZZTl1tkk---Beth Barnes and David Rein from METR on the one graph that ate the AI timelines discourse, and why the people who built it are the most careful about how it gets read.Beth founded METR after leaving OpenAI alignment. David is first author on GPQA and co-author on HCAST and the METR Time Horizons paper. Together they built the measurement Daniel Kokotajlo called the single most important piece of evidence on AI timelines: the log-linear line of "how long a task a frontier model can complete at 50% reliability" vs release date.The conversation opens on reward hacking. Current models can articulate in chat why a behaviour is undesired and then execute it anyway as agents. From there: construct validity, Melanie Mitchell's four-problem taxonomy, and the ARC-AGI 1-to-2 collapse as a worked example of adversarially-selected benchmarks regressing once labs target them. Beth's counter: METR deliberately does not adversarially select. David's: models do not have to do the right thing for the right reasons.Methodology, then specification — David's compiler analogy, Beth on four-month tasks as expensive to evaluate rather than unspecifiable. Then the SWE-bench reality check, the METR finding that half of passing PRs would not be merged, and Beth's horses-versus-bank-tellers analogy for the labour market.The close: monitorability, the coin-spinning boat, two-year recursive self-improvement, and Beth's line that "overhyped now" and "big deal later" are not correlated claims.---TIMESTAMPS:00:00:00 Intro00:02:06 Sponsor break: Prolific human-feedback infrastructure00:02:33 Welcome and the scalable oversight motivation00:06:02 Construct validity, benchmark pathologies and the Chollet worry00:15:45 Time Horizons: human time, HCAST tasks and the 50% logistic00:24:50 Is human difficulty really one variable?00:33:05 Agent harness evolution and the inference-compute dividend00:40:00 Scaffolding bells, token budgets and the credit-assignment problem00:44:15 Look at the damn graph: regularisation bug and reliability nuance00:50:00 Why 50%? Reliability, reward hacking and pizza-party transcripts00:55:20 Extrapolation risk and straight lines on graphs00:59:25 Software engineering as a specification acquisition problem01:07:40 Compilers also made ugly code: vibe-coding quality and Claude on METR Slack01:15:15 Strongest defensible claim, Carlini's compiler swarm and AI 202701:23:45 SWE-bench merge rates, the bank-teller analogy and horses01:31:45 Scheming, alignment faking and the mentalistic vocabulary problem01:40:45 Reward hacking, monitorability and chain-of-thought faithfulness01:45:25 Recursive self-improvement, knowledge vs intelligence and closing
ReScript: https://app.rescript.info/public/share/de3bb40cc02ee39fdf36e2c60366eb4d
(PDF, refs, transcript etc)
When AI Discovers The Next Transformer - Robert Lange (Sakana)
Robert Lange, founding researcher at Sakana AI, joins Tim to discuss *Shinka Evolve* — a framework that combines LLMs with evolutionary algorithms to do open-ended program search. The core claim: systems like …
Notes de l’épisode
Robert Lange, founding researcher at Sakana AI, joins Tim to discuss *Shinka Evolve* — a framework that combines LLMs with evolutionary algorithms to do open-ended program search. The core claim: systems like AlphaEvolve can optimize solutions to fixed problems, but real scientific progress requires co-evolving the problems themselves.
GTC is coming, the premier AI conference, great opportunity to learn about AI. NVIDIA and partners will showcase breakthroughs in physical AI, AI factories, agentic AI, and inference, exploring the next wave of AI innovation for developers and researchers. Register for virtual GTC for free, using my link and win NVIDIA DGX Spark (https://nvda.ws/4qQ0LMg)
• Why AlphaEvolve gets stuck — it needs a human to hand it the right problem. Shinka tries to invent new problems automatically, drawing on ideas from POET, PowerPlay, and MAP-Elites quality-diversity search.
• The *architecture* of Shinka: an archive of programs organized as islands, LLMs used as mutation operators, and a UCB bandit that adaptively selects between frontier models (GPT-5, Sonnet 4.5, Gemini) mid-run. The credit-assignment problem across models turns out to be genuinely hard.
• Concrete results — state-of-the-art circle packing with dramatically fewer evaluations, second place in an AtCoder competitive programming challenge, evolved load-balancing loss functions for mixture-of-experts models, and agent scaffolds for AIME math benchmarks.
• Are these systems actually thinking outside the box, or are they parasitic on their starting conditions? When LLMs run autonomously, "nothing interesting happens." Robert pushes back with the stepping-stone argument — evolution doesn't need to extrapolate, just recombine usefully.
• The AI Scientist question: can automated research pipelines produce real science, or just workshop-level slop that passes surface-level review? Robert is honest that the current version is more co-pilot than autonomous researcher.
• Where this lands in 5-20 years — Robert's prediction that scientific research will be fundamentally transformed, and Tim's thought experiment about alien mathematical artifacts that no human could have conceived.
Robert Lange: https://roberttlange.com/
---
TIMESTAMPS:
00:00:00 Introduction: Robert Lange, Sakana AI and Shinka Evolve
00:04:15 AlphaEvolve's Blind Spot: Co-Evolving Problems with Solutions
00:09:05 Unknown Unknowns, POET, and Auto-Curricula for AI Science
00:14:20 MAP-Elites and Quality-Diversity: Shinka's Evolutionary Architecture
00:28:00 UCB Bandits, Mutations and the Vibe Research Vision
00:40:00 Scaling Shinka: Meta-Evolution, Democratisation and the Three-Axis Model
00:47:10 Applications, ARC-AGI and the Future of Work
00:57:00 The AI Scientist and the Human Co-Pilot: Who Steers the Search?
01:06:00 AI Scientist v2, Slop Critique and the Future of Scientific Publishing
---
REFERENCES:
paper:
[00:03:30] ShinkaEvolve: Towards Open-Ended And Sample-Efficient Program Evolution
https://arxiv.org/abs/2509.19349
[00:04:15] AlphaEvolve: A Coding Agent for Scientific and Algorithmic Discovery
https://arxiv.org/abs/2506.13131
[00:06:30] Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agents
https://arxiv.org/abs/2505.22954
[00:09:05] Paired Open-Ended Trailblazer (POET)
https://arxiv.org/abs/1901.01753
[00:10:00] PowerPlay: Training an Increasingly General Problem Solver by Continually Searching for the Simplest Still Unsolvable Problem
https://arxiv.org/abs/1112.5309
[00:10:40] Automated Capability Discovery via Foundation Model Self-Exploration
https://arxiv.org/abs/2502.07577
[00:15:30] Illuminating Search Spaces by Mapping Elites (MAP-Elites)
https://arxiv.org/abs/1504.04909
[00:47:10] Automated Design of Agentic Systems (ADAS)
https://arxiv.org/abs/2408.08435
<trunc, see ReScript/YT>
PDF : https://app.rescript.info/api/sessions/b8a9dcf60623657c/pdf/download
Transcript: https://app.rescript.info/public/share/SDOD_3oXOcli3zTqcAtR8eibT5U3gam84oo4KRtI-Vk
"Vibe Coding is a Slot Machine" - Jeremy Howard
Dive into the realities of AI-assisted coding, the origins of modern fine-tuning, and the cognitive science behind machine learning with fast.ai founder Jeremy Howard. In this episode, we unpack why AI might be …
Notes de l’épisode
Dive into the realities of AI-assisted coding, the origins of modern fine-tuning, and the cognitive science behind machine learning with fast.ai founder Jeremy Howard. In this episode, we unpack why AI might be turning software engineering into a slot machine and how to maintain true technical intuition in the age of large language models.
GTC is coming, the premier AI conference, great opportunity to learn about AI. NVIDIA and partners will showcase breakthroughs in physical AI, AI factories, agentic AI, and inference, exploring the next wave of AI innovation for developers and researchers. Register for virtual GTC for free, using my link and win NVIDIA DGX Spark (https://nvda.ws/4qQ0LMg)
Jeremy Howard is a renowned data scientist, researcher, entrepreneur, and educator. As the co-founder of fast.ai, former President of Kaggle, and the creator of ULMFiT, Jeremy has spent decades democratizing deep learning. His pioneering work laid the foundation for modern transfer learning and the pre-training and fine-tuning paradigm that powers today's language models.
Key Topics and Main Insights Discussed:
- The Origins of ULMFiT and Fine-Tuning
- The Vibe Coding Illusion and Software Engineering
- Cognitive Science, Friction, and Learning
- The Future of Developers
RESCRIPT: https://app.rescript.info/public/share/BhX5zP3b0m63srLOQDKBTFTooSzEMh_ARwmDG_h_izk
Jeremy Howard:
https://x.com/jeremyphoward
https://www.answer.ai/
---
TIMESTAMPS (fixed):
00:00:00 Introduction & GTC Sponsor
00:04:30 ULMFiT & The Birth of Fine-Tuning
00:12:00 Intuition & The Mechanics of Learning
00:18:30 Abstraction Hierarchies & AI Creativity
00:23:00 Claude Code & The Interpolation Illusion
00:27:30 Coding vs. Software Engineering
00:30:00 Cosplaying Intelligence: Dennett vs. Searle
00:36:30 Automation, Radiology & Desirable Difficulty
00:42:30 Organizational Knowledge & The Slope
00:48:00 Vibe Coding as a Slot Machine
00:54:00 The Erosion of Control in Software
01:01:00 Interactive Programming & REPL Environments
01:05:00 The Notebook Debate & Exploratory Science
01:17:30 AI Existential Risk & Power Centralization
01:24:20 Current Risks, Privacy & Enfeeblement
---
REFERENCES:
Blog Post:
[00:03:00] fast.ai Blog: Self-Supervised Learning
https://www.fast.ai/posts/2020-01-13-self_supervised.html
[00:13:30] DeepMind Blog: Gemini Deep Think
https://deepmind.google/blog/accelerating-mathematical-and-scientific-discovery-with-gemini-deep-think/
[00:19:30] Modular Blog: Claude C Compiler analysis
https://www.modular.com/blog/the-claude-c-compiler-what-it-reveals-about-the-future-of-software
[00:19:45] Anthropic Engineering Blog: Building C Compiler
https://www.anthropic.com/engineering/building-c-compiler
[00:48:00] Cursor Blog: Scaling Agents
https://cursor.com/blog/scaling-agents
[01:05:15] fast.ai Blog: NB Dev Merged Driver
https://www.fast.ai/posts/2022-08-25-jupyter-git.html
[01:17:30] Jeremy Howard: Response to AI Risk Letter
https://www.normaltech.ai/p/is-avoiding-extinction-from-ai-really
Book:
[00:08:30] M. Chirimuuta: The Brain Abstracted
https://mitpress.mit.edu/9780262548045/the-brain-abstracted/
[00:30:00] Daniel Dennett: Consciousness Explained
https://www.amazon.com/Consciousness-Explained-Daniel-C-Dennett/dp/0316180661
[00:42:30] Cesar Hidalgo: Infinite Alphabet / Laws of Knowledge
https://www.amazon.com/Infinite-Alphabet-Laws-Knowledge/dp/0241655676
Archive Article:
[00:13:45] MLST Archive: Why Creativity Cannot Be Interpolated
https://archive.mlst.ai/read/why-creativity-cannot-be-interpolated
Research Study:
[00:24:30] METR Study: AI OS Development
https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/
Paper:
[00:24:45] Fred Brooks: No Silver Bullet
https://www.cs.unc.edu/techreports/86-020.pdf
[00:30:15] John Searle: Minds, Brains, and Programs
https://www.cambridge.org/core/journals/behavioral-and-brain-sciences/article/minds-brains-and-programs/DC644B47A4299C637C89772FACC2706A
Evolution "Doesn't Need" Mutation - Blaise Agüera y Arcas
What if life itself is just a really sophisticated computer program that wrote itself into existence?Blaise Agüera y Arcas presenting at ALife 2025 — the most technically detailed public walkthrough of the …
Notes de l’épisode
What if life itself is just a really sophisticated computer program that wrote itself into existence?
Blaise Agüera y Arcas presenting at ALife 2025 — the most technically detailed public walkthrough of the ideas in his *What is Life?* and *What is Intelligence?* books that we've come across.He covers the BFF experiments (self-replicating programs emerging spontaneously from random noise), the mathematical framework connecting Lotka-Volterra population dynamics with Smoluchowski coagulation, eigenvalue analysis of cooperation matrices, and his central claim that symbiogenesis — not mutation — is the primary engine of evolutionary novelty.The experimental results are genuinely striking: complex self-replicating code arising from random byte strings with zero mutation, a sharp phase transition that looks like gelation, and a proof that blocking deep symbiogenetic ancestry trees prevents the transition entirely.A few things worth flagging for critical viewers:— The substrate is more carefully engineered than the framing sometimes suggests. The choice of language, tape length, interaction protocol, and step limits all shape what emerges. Their own SUBLEQ counterexample (where self-replicators *don't* arise despite being theoretically possible) highlights that these design choices matter substantially — and a general theory of which substrates support this transition is still missing.— The leap from "self-replicating programs on fixed-length tapes" to "life was computational and intelligent from the start" involves significant philosophical extrapolation beyond what the experiments directly demonstrate.— The Bedau et al. (2000) open problems paper he references at the start actually sets a higher bar for Challenge 3.2 than BFF currently meets: it asks that "the internal organization of these 'organisms' and the boundaries separating them from their environment arise and be sustained through the activities of lower-level primitives" — whereas BFF's tape boundaries are fixed by design, not emergent.
---
TIMESTAMPS:
00:00:00 Introduction: From Noise to Programs & ALife History
00:03:15 Defining Life: Function as the "Spirit"
00:05:45 Von Neumann's Insight: Life is Embodied Computation
00:09:15 Physics of Computation: Irreversibility & Fallacies
00:15:00 The BFF Experiment: Spontaneous Generation of Code
00:23:45 The Mystery: Complexity Growth Without Mutation
00:27:00 Symbiogenesis: The Engine of Novelty
00:33:15 Mathematical Proof: Blocking Symbiosis Stops Life
00:40:15 Evolutionary Implications: It's Symbiogenesis All The Way Down
00:44:30 Intelligence as Modeling Others
00:46:49 Q&A: Levels of Abstraction & Definitions
---
REFERENCES:
Paper:
[00:01:16] Open Problems in Artificial Life
https://direct.mit.edu/artl/article/6/4/363/2354/Open-Problems-in-Artificial-Life
[00:09:30] When does a physical system compute?
https://arxiv.org/abs/1309.7979
[00:15:00] Computational Life
https://arxiv.org/abs/2406.19108
[00:27:30] On the Origin of Mitosing Cells
https://pubmed.ncbi.nlm.nih.gov/11541392/
[00:42:00] The Major Evolutionary Transitions
https://www.nature.com/articles/374227a0
[00:44:00] The ARC gene
https://www.nih.gov/news-events/news-releases/memory-gene-goes-viral
Person:
[00:05:45] Alan Turing
https://plato.stanford.edu/entries/turing/
[00:07:30] John von Neumann
https://en.wikipedia.org/wiki/John_von_Neumann
[00:11:15] Hector Zenil
https://hectorzenil.net/
[00:12:00] Robert Sapolsky
https://profiles.stanford.edu/robert-sapolsky
---
LINKS:
RESCRIPT: https://app.rescript.info/public/share/ff7gb6HpezOR3DF-gr9-rCoMFzzEgUjLQK6voV5XVWY
VAEs Are Energy-Based Models? [Dr. Jeff Beck]
What makes something truly *intelligent?* Is a rock an agent? Could a perfect simulation of your brain actually *be* you? In this fascinating conversation, Dr. Jeff Beck takes us on a journey through the …
Notes de l’épisode
What makes something truly *intelligent?* Is a rock an agent? Could a perfect simulation of your brain actually *be* you? In this fascinating conversation, Dr. Jeff Beck takes us on a journey through the philosophical and technical foundations of agency, intelligence, and the future of AI.
Jeff doesn't hold back on the big questions. He argues that from a purely mathematical perspective, there's no structural difference between an agent and a rock – both execute policies that map inputs to outputs. The real distinction lies in *sophistication* – how complex are the internal computations? Does the system engage in planning and counterfactual reasoning, or is it just a lookup table that happens to give the right answers?
*Key topics explored in this conversation:*
*The Black Box Problem of Agency* – How can we tell if something is truly planning versus just executing a pre-computed response? Jeff explains why this question is nearly impossible to answer from the outside, and why the best we can do is ask which model gives us the simplest explanation.
*Energy-Based Models Explained* – A masterclass on how EBMs differ from standard neural networks. The key insight: traditional networks only optimize weights, while energy-based models optimize *both* weights and internal states – a subtle but profound distinction that connects to Bayesian inference.
*Why Your Brain Might Have Evolved from Your Nose* – One of the most surprising moments in the conversation. Jeff proposes that the complex, non-smooth nature of olfactory space may have driven the evolution of our associative cortex and planning abilities.
*The JEPA Revolution* – A deep dive into Yann LeCun's Joint Embedding Prediction Architecture and why learning in latent space (rather than predicting every pixel) might be the key to more robust AI representations.
*AI Safety Without Skynet Fears* – Jeff takes a refreshingly grounded stance on AI risk. He's less worried about rogue superintelligences and more concerned about humans becoming "reward function selectors" – couch potatoes who just approve or reject AI outputs. His proposed solution? Use inverse reinforcement learning to derive AI goals from observed human behavior, then make *small* perturbations rather than naive commands like "end world hunger."
Whether you're interested in the philosophy of mind, the technical details of modern machine learning, or just want to understand what makes intelligence *tick,* this conversation delivers insights you won't find anywhere else.
---
TIMESTAMPS:
00:00:00 Geometric Deep Learning & Physical Symmetries
00:00:56 Defining Agency: From Rocks to Planning
00:05:25 The Black Box Problem & Counterfactuals
00:08:45 Simulated Agency vs. Physical Reality
00:12:55 Energy-Based Models & Test-Time Training
00:17:30 Bayesian Inference & Free Energy
00:20:07 JEPA, Latent Space, & Non-Contrastive Learning
00:27:07 Evolution of Intelligence & Modular Brains
00:34:00 Scientific Discovery & Automated Experimentation
00:38:04 AI Safety, Enfeeblement & The Future of Work
---
REFERENCES:
Concept:
[00:00:58] Free Energy Principle (FEP)
https://en.wikipedia.org/wiki/Free_energy_principle
[00:06:00] Monte Carlo Tree Search
https://en.wikipedia.org/wiki/Monte_Carlo_tree_search
Book:
[00:09:00] The Intentional Stance
https://mitpress.mit.edu/9780262540537/the-intentional-stance/
Paper:
[00:13:00] A Tutorial on Energy-Based Learning (LeCun 2006)
http://yann.lecun.com/exdb/publis/pdf/lecun-06.pdf
[00:15:00] Auto-Encoding Variational Bayes (VAE)
https://arxiv.org/abs/1312.6114
[00:20:15] JEPA (Joint Embedding Prediction Architecture)
https://openreview.net/forum?id=BZ5a1r-kVsf
[00:22:30] The Wake-Sleep Algorithm
https://www.cs.toronto.edu/~hinton/absps/ws.pdf
<trunc, see rescript>
---
RESCRIPT:
https://app.rescript.info/public/share/DJlSbJ_Qx080q315tWaqMWn3PixCQsOcM4Kf1IW9_Eo
PDF:
https://app.rescript.info/api/public/sessions/0efec296b9b6e905/pdf
Abstraction & Idealization: AI's Plato Problem [Mazviita Chirimuuta]
Professor Mazviita Chirimuuta joins us for a fascinating deep dive into the philosophy of neuroscience and what it really means to understand the mind.*What can neuroscience actually tell us about how the mind …
Notes de l’épisode
Professor Mazviita Chirimuuta joins us for a fascinating deep dive into the philosophy of neuroscience and what it really means to understand the mind.*What can neuroscience actually tell us about how the mind works?* In this thought-provoking conversation, we explore the hidden assumptions behind computational theories of the brain, the limits of scientific abstraction, and why the question of machine consciousness might be more complicated than AI researchers assume.Mazviita, author of *The Brain Abstracted,* brings a unique perspective shaped by her background in both neuroscience research and philosophy. She challenges us to think critically about the metaphors we use to understand cognition — from the reflex theory of the late 19th century to today's dominant view of the brain as a computer.*Key topics explored:**The problem of oversimplification* — Why scientific models necessarily leave things out, and how this can sometimes lead entire fields astray. The cautionary tale of reflex theory shows how elegant explanations can blind us to biological complexity.*Is the brain really a computer?* — Mazviita unpacks the philosophical assumptions behind computational neuroscience and asks: if we can model anything computationally, what makes brains special? The answer might challenge everything you thought you knew about AI.*Haptic realism* — A fresh way of thinking about scientific knowledge that emphasizes interaction over passive observation. Knowledge isn't about reading the "source code of the universe" — it's something we actively construct through engagement with the world.*Why embodiment matters for understanding* — Can a disembodied language model truly understand? Mazviita makes a compelling case that human cognition is deeply entangled with our sensory-motor engagement and biological existence in ways that can't simply be abstracted away.*Technology and human finitude* — Drawing on Heidegger, we discuss how the dream of transcending our physical limitations through technology might reflect a fundamental misunderstanding of what it means to be a knower.This conversation is essential viewing for anyone interested in AI, consciousness, philosophy of mind, or the future of cognitive science. Whether you're skeptical of strong AI claims or a true believer in machine consciousness, Mazviita's careful philosophical analysis will give you new tools for thinking through these profound questions.---TIMESTAMPS:00:00:00 The Problem of Generalizing Neuroscience00:02:51 Abstraction vs. Idealization: The "Kaleidoscope"00:05:39 Platonism in AI: Discovering or Inventing Patterns?00:09:42 When Simplification Fails: The Reflex Theory00:12:23 Behaviorism and the "Black Box" Trap00:14:20 Haptic Realism: Knowledge Through Interaction00:20:23 Is Nature Protean? The Myth of Converging Truth00:23:23 The Computational Theory of Mind: A Useful Fiction?00:27:25 Biological Constraints: Why Brains Aren't Just Neural Nets00:31:01 Agency, Distal Causes, and Dennett's Stances00:37:13 Searle's Challenge: Causal Powers and Understanding00:41:58 Heidegger's Warning & The Experiment on Children---REFERENCES:Book:[00:01:28] The Brain Abstractedhttps://mitpress.mit.edu/9780262548045/the-brain-abstracted/[00:11:05] The Integrated Action of the Nervous Systemhttps://www.amazon.sg/integrative-action-nervous-system/dp/9354179029[00:18:15] The Quest for Certainty (Dewey)https://www.amazon.com/Quest-Certainty-Relation-Knowledge-Lectures/dp/0399501916[00:19:45] Realism for Realistic People (Chang)https://www.cambridge.org/core/books/realism-for-realistic-people/ACC93A7F03B15AA4D6F3A466E3FC5AB7<truncated, see ReScript>---RESCRIPT:https://app.rescript.info/public/share/A6cZ1TY35p8ORMmYCWNBI0no9ChU3-Kx7dPXGJURvZ0PDF Transcript:https://app.rescript.info/api/public/sessions/0fb7767e066cf712/pdf
Why Every Brain Metaphor in History Has Been Wrong [SPECIAL EDITION]
What if everything we think we know about the brain is just a really good metaphor that we forgot was a metaphor?This episode takes you on a journey through the history of scientific simplification, from a …
Notes de l’épisode
What if everything we think we know about the brain is just a really good metaphor that we forgot was a metaphor?This episode takes you on a journey through the history of scientific simplification, from a young Karl Friston watching wood lice in his garden to the bold claims that your mind is literally software running on biological hardware.We bring together some of the most brilliant minds we've interviewed — Professor Mazviita Chirimuuta, Francois Chollet, Joscha Bach, Professor Luciano Floridi, Professor Noam Chomsky, Nobel laureate John Jumper, and more — to wrestle with a deceptively simple question: *When scientists simplify reality to study it, what gets captured and what gets lost?**Key ideas explored:**The Spherical Cow Problem* — Science requires simplification. We're limited creatures trying to understand systems far more complex than our working memory can hold. But when does a useful model become a dangerous illusion?*The Kaleidoscope Hypothesis* — Francois Chollet's beautiful idea that beneath all the apparent chaos of reality lies simple, repeating patterns — like bits of colored glass in a kaleidoscope creating infinite complexity. Is this profound truth or Platonic wishful thinking?*Is Software Really Spirit?* — Joscha Bach makes the provocative claim that software is literally spirit, not metaphorically. We push back on this, asking whether the "sameness" we see across different computers running the same program exists in nature or only in our descriptions.*The Cultural Illusion of AGI* — Why does artificial general intelligence seem so inevitable to people in Silicon Valley? Professor Chirimuuta suggests we might be caught in a "cultural historical illusion" — our mechanistic assumptions about minds making AI seem like destiny when it might just be a bet.*Prediction vs. Understanding* — Nobel Prize winner John Jumper: AI can predict and control, but understanding requires a human in the loop. Throughout history, we've described the brain as hydraulic pumps, telegraph networks, telephone switchboards, and now computers. Each metaphor felt obviously true at the time. This episode asks: what will we think was naive about our current assumptions in fifty years?Featuring insights from *The Brain Abstracted* by Mazviita Chirimuuta — possibly the most influential book on how we think about thinking in 2025.---TIMESTAMPS:00:00:00 The Wood Louse & The Spherical Cow00:02:04 The Necessity of Abstraction00:04:42 Simplicius vs. Ignorantio: The Boxing Match00:06:39 The Kaleidoscope Hypothesis00:08:40 Is the Mind Software?00:13:15 Critique of Causal Patterns00:14:40 Temperature is Not a Thing00:18:24 The Ship of Theseus & Ontology00:23:45 Metaphors Hardening into Reality00:25:41 The Illusion of AGI Inevitability00:27:45 Prediction vs. Understanding00:32:00 Climbing the Mountain vs. The Helicopter00:34:53 Haptic Realism & The Limits of Knowledge---REFERENCES:Person:[00:00:00] Karl Friston (UCL)https://profiles.ucl.ac.uk/1236-karl-friston[00:06:30] Francois Chollethttps://fchollet.com/[00:14:41] Cesar Hidalgo, MLST interview.https://www.youtube.com/watch?v=vzpFOJRteeI[00:30:30] Terence Tao's Bloghttps://terrytao.wordpress.com/Book:[00:02:25] The Brain Abstractedhttps://mitpress.mit.edu/9780262548045/the-brain-abstracted/[00:06:00] On Learned Ignorancehttps://www.amazon.com/Nicholas-Cusa-learned-ignorance-translation/dp/0938060236[00:24:15] Science and the Modern Worldhttps://amazon.com/dp/0684836394<truncated, see ReScript>
RESCRIPT:https://app.rescript.info/public/share/CYy0ex2M2kvcVRdMnSUky5O7H7hB7v2u_nVhoUiuKD4PDF Transcript: https://app.rescript.info/api/public/sessions/6c44c41e1e0fa6dd/pdf
Thank you to Dr. Maxwell Ramstead for early script work on this show (Ph.D student of Friston) and the woodlice story came from him!
Voici les 40 derniers épisodes sur 273. Les épisodes plus anciens se trouvent dans le flux du podcast.
Podcasts similaires
Plus de podcasts : Technologies
C’est votre podcast ?
Confirmez-le avec une entrée dans votre flux : vous recevez ici un code, vous l’ajoutez comme podcast:txt dans le canal de votre flux, et nous le vérifions. Votre podcast apparaît ensuite comme « vérifié » sur SoundNode.