Skip to content

Ep 89: AI Research Legend’s Honest Assessment of Where We Are

June 3, 2026 · 1 h 14 min

Show notes

This episode with Lukasz Kaiser, co-author of the seminal "Attention Is All You Need" transformer paper and former researcher at both Google Brain and OpenAI, is a wide-ranging conversation about the fundamental limits of current AI architectures and whether transformers will continue to dominate or eventually give way to something new. Lukasz brings a rare dual perspective: deep belief in how far the current paradigm has taken us (he's an enthusiastic daily Codex user who's seen 10x productivity gains in his own research), while maintaining genuine intellectual humility about whether transformers can truly generalize the way humans do. The episode weaves together questions about data efficiency, the non-verifiable RL frontier, the coding agent revolution, the open vs. closed source gap, and what the next architectural leap might look like: all filtered through the lens of someone who helped build the foundation the entire field is standing on.

(0:00) Intro

(1:12) Transformers vs. Human Learning

(8:37) How Do We Get Physical World Generalization?

(10:52) What Comes After Transformers

(13:59) How Much Have Agents Improved Lukasz's AI Research Productivity?

(17:21) How Close Is an AI Research Intern?

(26:06) RL Beyond Verifiable Tasks

(35:38) App Companies: Build Models or Lean on Labs?

(46:21) Multimodal Is Still Missing Something

(49:46) OpenAI's Bet on Reasoning

(55:26) The AI Coding Wars

(59:26) Focus vs. Keeping Embers Burning

(1:02:09) Open Source vs. Closed Source Gap

(1:05:15) Quickfire

Jacob is an AI investor at Redpoint Ventures. He's led Redpoint's investments in companies like Abridge, Physical Intelligence & Legora. Follow Jacob on Twitter (@jacobeffron).

On Unsupervised Learning we probe the sharpest minds in AI in search for the truth about what's real today, what will be real in the future and what it all means for businesses and the world. If you're a builder, researcher or investor navigating the AI world, this podcast will help you deconstruct and understand the most important breakthroughs and see a clearer picture of reality. Subscribe to this show to stay up to date on our latest episodes.

All episodes of Unsupervised Learning with Jacob Effron

Settings

Display

Language

Decides which stations you see first on the home page.

History

Recently played stations and episodes and your latest searches are only stored in this browser.

Notifications

A short notice when a podcast you follow has a new episode. Turn off single podcasts with the bell on their page.

Account

Save favorites – in the app and on the website.

Keyboard shortcuts

/
Open search
Space
Play or pause
← →
In the player: back 15 s, forward 30 s (episodes) · in a row: next tile
↑ ↓
In a row: target within the tile
Home / End
In a row: first or last tile
Esc
Close window or menu
?
This overview
Now playing

Go to station page

Play next

    Add episodes with “Play next” – they play after the current episode.