Skip to content

The Future of Data Engineering in the Age of AI | Erfan Hesami

September 16, 2026 · 47 min

Show notes

I sat down with Erfan Hesami, founder of Pipeline to Insights, to talk about where data engineering is headed as AI and agents become part of everyday engineering work.

We dig into the changing role of the data engineer, the growing overlap between data and AI engineering, and why strong fundamentals may actually become more important as AI makes writing code cheaper and easier.

We also talk about AI agents inside modern data platforms, governance and security, AI-generated code and technical debt, controlling AI costs, the importance of human judgment, and why communication and business skills are becoming increasingly valuable for engineers.

* Erfan also shares the story behind Pipeline to Insights, how writing helped accelerate his own learning, and how consistency helped him grow the publication from its first post in October 2024 into a successful data and AI community.

If you’re a data engineer, analytics engineer, AI engineer, or someone trying to break into data, this conversation offers a grounded look at what skills are likely to matter as AI changes how we build software and data systems.

Thanks for reading Data Engineering Central! This post is public so feel free to share it.



This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribe

All episodes of Data Engineering Central Podcast

Settings

Display

Language

Decides which stations you see first on the home page.

History

Recently played stations and episodes and your latest searches are only stored in this browser.

Notifications

A short notice when a podcast you follow has a new episode. Turn off single podcasts with the bell on their page.

Account

Save favorites – in the app and on the website.

Keyboard shortcuts

/
Open search
Space
Play or pause
← →
In the player: back 15 s, forward 30 s (episodes) · in a row: next tile
↑ ↓
In a row: target within the tile
Home / End
In a row: first or last tile
Esc
Close window or menu
?
This overview
Now playing

Go to station page

Play next

    Add episodes with “Play next” – they play after the current episode.