1031: Tokenomics: Why Your Agentic AI Bill Is Exploding (and How to Fix It), with Tyler Cox and Ish Shah
Show notes
In Episode #1031, Ish Shah and Tyler Cox (Distinguished Engineers in the Office of the CTO for Dell Technologies' client group) join Jon Krohn to work out why agentic AI bills are exploding and what can be done about it. Over one weekend Ish burned roughly two billion tokens on a side project, and that is the ordinary shape of agentic work now: agents spawn sub-agents, the pie of work grows, and cheaper tokens only invite more ambitious projects. Tyler runs a small Dell lab that pushes hundreds of millions of tokens a day through local hardware instead.
In this episode, they define what makes a system agentic, explain how to read a Pareto curve when choosing models, work through the jagged frontier and why most tasks do not need a frontier model, and lay out what moving agentic workloads onto your own hardware does to the economics.
Additional materials: https://www.superdatascience.com/1031
Interested in sponsoring a SuperDataScience Podcast episode? Email [email protected] for sponsorship information.
In this episode you will learn:
- (00:03:42) What makes a system agentic
- (00:12:45) Picking the right model for the task
- (00:16:46) How to read a Pareto curve
- (00:27:02) Why agents burn so many more tokens
All episodes of Super Data Science: ML & AI Podcast with Jon Krohn