Datadog Deleted All Its AI Context. It Worked.

Datadog's Language Foundations team deleted an entire folder of AI context files that had been carefully written and maintained for over a year, expecting a performance hit. Instead, their evals got better. Simon Boudrias, who runs Language Foundations at Datadog, walks Guy through what that taught his team about context rot, and the full journey of scaling AI coding agents to 4,000 engineers.

What we cover:
– How Datadog scaled Cursor and Claude Code to 4,000 engineers in under a year
– Why Datadog deleted all of its AI context files, and what happened to eval scores
– Building an eval-driven code review system that replays old PRs to catch real incidents
– Where open-weight models like GLM 5.2 stand against the frontier
– Rethinking hiring and career ladders now that AI can run a real codebase interview

Chapters:
00:00:00 - Introduction
00:03:19 - Simon's role and Datadog's 4,000-engineer org
00:04:31 - The Cursor rollout that took off overnight
00:07:53 - How Claude Code entered the picture
00:10:51 - Building dedicated Signals and Flows teams
00:11:58 - Why Datadog bet early on evals
00:33:44 - Deleting all their AI context and getting better evals
00:47:51 - Where open-weight models stand today
00:51:45 - Rethinking hiring and career ladders for AI
00:59:24 - The real prize: better decisions, not just productivity

🌐 Tessl: https://tessl.io
🔔 Subscribe for weekly episodes on AI-native development

What's the oldest file in your AGENTS.md or CLAUDE.md that you're afraid to delete? Tell us in the comments.