Skip to content

1028: The Chip Built for Agentic AI Inference, with SambaNova's Anton McGonnell

September 18, 2026 · 30 min

Show notes

In Episode #1028, Anton McGonnell (VP of Product at SambaNova) joins Jon Krohn to explain why the chips running most AI inference today were never designed for the job. Agentic AI has changed the computational profile of inference, with much larger inputs and far heavier caches feeding the token generation that follows, and that shift has exposed where GPU architecture struggles. SambaNova has raised over $2 billion to build an alternative, the reconfigurable dataflow unit, which lays a whole model out spatially across the chip rather than executing it kernel by kernel. In this episode, Anton discusses why the speed that matters is payback, and how speed and concurrency are what turn a fixed hardware cost into a six-month payback. He also walks through the trade-off every inference provider faces between speed per user and throughput per chip, what the RDU architecture changes about scaling and data center deployment, the economics of the new SN50, and why four out of five AI infrastructure leaders say they would pay a premium for faster tokens.


Additional materials: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://www.superdatascience.com/1028⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠


Interested in sponsoring a SuperDataScience Podcast episode? Email [email protected] for sponsorship information.


In this episode you will learn:

  • (00:02:41) Why agentic AI is reshaping inference workloads
  • (00:08:39) How SambaNova's RDU differs from a GPU
  • (00:17:54) The economics of the SN50

All episodes of Super Data Science: ML & AI Podcast with Jon Krohn

Settings

Display

Language

Decides which stations you see first on the home page.

History

Recently played stations and episodes and your latest searches are only stored in this browser.

Notifications

A short notice when a podcast you follow has a new episode. Turn off single podcasts with the bell on their page.

Account

Save favorites – in the app and on the website.

Keyboard shortcuts

/
Open search
Space
Play or pause
← →
In the player: back 15 s, forward 30 s (episodes) · in a row: next tile
↑ ↓
In a row: target within the tile
Home / End
In a row: first or last tile
Esc
Close window or menu
?
This overview
Now playing

Go to station page

Play next

    Add episodes with “Play next” – they play after the current episode.