Ep 97: Jev Creator on Decision Models, Benchmark Fatigue & Why the Labs Missed It

Jev creator and TypeSafe AI co-founder, Diogo Almeida, joins Jacob Effron to talk about why AI still isn't automating the easy work it's been smart enough to do for years, and why he built a new class of model around reliability.

Diogo coauthored OpenAI's InstructGPT paper and spent two years in stealth building Jev, a "system one" model that returns structured decisions instead of text, which launched in September.

Diogo and Jacob discuss what the viral launch was really like, why he thought Jev would take a week and it took years, why almost every LLM call should be meant for a machine, and why developers should push model providers harder on jaggedness. They also cover the tyranny of the KV cache in coding agents, why new model shapes are cheap and reliability is the hard part, his estimate that coding agents deliver a 20 to 40% speedup, why he sees financial engineering as the labs' underrated strength, and why he thinks we're two gigantic breakthroughs from true AI agency.

(0:00) Intro
(1:14) The Early Bet on RLCD
(4:40) Built for Humans vs Computers
(6:39) Roadmap & Model Limitations
(11:04) Mental Models of Models
(16:07) Do LLMs Have a Place in 10 Years?
(17:32) New Model Shapes
(21:24) Problems Still Unsolved
(27:00) The Tyranny of the KV Cache
(31:21) Rethinking Coding Agents
(34:40) Multimodality
(37:52) Images, Games & Computer Use
(39:36) Could This Have Been Built 5 Years Ago?
(42:01) Benchmarks & Copycat Releases
(44:12) Fundraising
(45:27) Quickfire
 

OpenAI's InstructGPT paper: https://arxiv.org/abs/2203.02155

Jacob is an AI investor at Redpoint Ventures. He's led Redpoint's investments in companies like Abridge, Physical Intelligence & Legora. Follow Jacob on Twitter (@jacobeffron).

On Unsupervised Learning we probe the sharpest minds in AI in search for the truth about what's real today, what will be real in the future and what it all means for businesses and the world. If you're a builder, researcher or investor navigating the AI world, this podcast will help you deconstruct and understand the most important breakthroughs and see a clearer picture of reality. Subscribe to this show to stay up to date on our latest episodes.