Spark, Lakehouse & AI: A Deep Conversation with Bart Konieczny
Show notes
In this episode of Data Engineering Central, I sit down with Bart Konieczny — data engineer, distributed systems expert, and well-known author in the Data and Spark ecosystem — for a deep technical conversation about modern data engineering.
We cover:
* How Bart got into tech and distributed systems
* His journey through different engineering roles
* Spark internals and why they still matter
* The realities of lakehouse architecture
* Streaming vs batch systems
* AI’s impact on data engineering
* What engineers should focus on in 2026
In a world obsessed with abstractions and AI tooling, we explore whether understanding the internals is still worth it — or if the game has fundamentally changed.
If you’re a data engineer, architect, or platform leader trying to navigate the next phase of the lakehouse era, this one’s for you.
Thanks for reading Data Engineering Central! This post is public so feel free to share it.
—
🎙️ Data Engineering Central PodcastHosted by Daniel Beach
If you’re a CTO or data leader looking for help building or optimizing your data platform, reach out — consulting inquiries welcome.
Data Engineering Central is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.
This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribe