Data, AI, and DuckDB

In this episode of the Data Engineering Central Podcast, I sit down with Jacob Matson, Developer Advocate at MotherDuck, to unpack one of the most interesting shifts happening in data engineering right now.

Jacob didn’t start in tech the way most people expect. He began in accounting, working with Excel and financial systems, before slowly realizing that the real problem he loved solving wasn’t finance, it was data pipelines. That path eventually led him deep into SQL Server, data warehousing, and ultimately to DuckDB, a tool that fundamentally changed how he thought about processing data.

* What we get into is bigger than just tools, though.

We talk about why DuckDB exploded in popularity, what it gets right that traditional databases and even modern cloud warehouses struggle with, and why the industry may be swinging back toward simplicity after years of over-engineered “modern data stacks.”

There’s a really interesting thread here around how engineers accidentally created too much complexity, and now tools like DuckDB are winning by removing it.

We also go deep on the evolution of the data stack itself. From SQL Server’s “everything in one box” model, to the unbundled chaos of the modern stack, and now back toward a more unified, simpler approach. Jacob shares how MotherDuck is thinking about that shift and where things are headed next.

* One of the more important parts of this conversation is around AI.

There’s a strong argument here that AI doesn’t kill data engineering; it massively expands it. Instead of fewer queries being written, we may be heading toward a world where AI agents generate orders of magnitude more queries than humans ever could. That flips a lot of assumptions on their head, especially around things like data modeling, which suddenly becomes more important, not less.

We also talk about:

* Why most Spark workloads are overkill

* When single-node tools like DuckDB actually win

* The real tradeoffs behind Lakehouse architectures

* Why data modeling is still critical in an AI-driven world

* How engineers should think about building in 2026 and beyond

This is one of those conversations that helps you zoom out and see where things are actually going, not just what tools are trending this week.

If you’re building data platforms, experimenting with AI, or just trying to simplify your stack, this one is worth your time.

Thanks for reading Data Engineering Central! This post is public so feel free to share it.

This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribe