Skip to content

#8 – Sara Hooker: Big AI, The Compute Frenzy, and Grumpy Models

August 5, 2024 · 55 min

Show notes

My guest today is ⁠Sara Hooker⁠, VP of Research at Cohere, where she leads Cohere for AI, a non-profit research lab that seeks to solve complex machine learning problems with researchers from over 100 countries. Sara is the author of numerous research papers, some of which focus specifically on scaling theory in AI. She has been listed as one of AI’s top 13 innovators by Fortune.

In our conversation, we first delve into the scaling laws behind foundation models. We explore what powers the scaling of AI systems and the limits to scaling laws. We then move on to discussing openness in AI, Cohere’s business strategy, the power of ecosystems, the importance of building multilingual LLMs, and the recent change in terms of access to data in the space. I hope you enjoy our conversation.

Find me on X at @⁠⁠ProfSchrepel⁠⁠⁠⁠⁠. Also, be sure to subscribe.

**

References:

➝ Sara Hooker, On the Limitations of Compute Thresholds as a Governance Strategy (2024)

➝ Sara Hooker, The Hardware Lottery (2020)

➝ Sara Hooker, Moving beyond “algorithmic bias is a data problem” (2021)➝ Longpre et al., Consent in Crisis: The Rapid Decline of the AI Data Commons (2024)

All episodes of Scaling Theory

Settings

Display

Language

Decides which stations you see first on the home page.

History

Recently played stations and episodes and your latest searches are only stored in this browser.

Notifications

A short notice when a podcast you follow has a new episode. Turn off single podcasts with the bell on their page.

Account

Save favorites – in the app and on the website.

Keyboard shortcuts

/
Open search
Space
Play or pause
← →
In the player: back 15 s, forward 30 s (episodes) · in a row: next tile
↑ ↓
In a row: target within the tile
Home / End
In a row: first or last tile
Esc
Close window or menu
?
This overview
Now playing

Go to station page

Play next

    Add episodes with “Play next” – they play after the current episode.