Data Engineering Central Podcast
- Episodios
- 44
- Se publica
- cada semana
- Último episodio
- 11 Nov 2026
Sobre el podcast
Long Live the Data Engineer. No holds barred. Talking about Data Engineering news, topics, and general mayhem.
Episodios 44
AI Is Changing Software Engineering Forever | Gregor Ojstersek
What happens to software engineers when AI can write most of the code?In this episode of the Data Engineering Central Podcast, I sit down with Gregor Ojstersek, former CTO, author of The Multiplier Mindset …
Notas del episodio
What happens to software engineers when AI can write most of the code?
In this episode of the Data Engineering Central Podcast, I sit down with Gregor Ojstersek, former CTO, author of The Multiplier Mindset (O’Reilly), and creator of the Engineering Leadership Newsletter, which recently surpassed 200,000 subscribers.
We dive into the changing world of software engineering, why technical skills alone aren’t enough anymore, and what it takes to become a successful engineer and engineering leader in the age of AI.
Gregor shares his journey from self-taught programmer to CTO, how he built one of the largest engineering leadership newsletters, and why the most valuable engineers of the future won’t necessarily be the ones writing the most code.
Thanks for reading Data Engineering Central! This post is public so feel free to share it.
Topics we cover:
* How Gregor went from self-taught developer to CTO without a computer science degree
* Why AI is changing the skills engineers need to succeed
* The future of junior, senior, staff, and principal engineering roles
* Whether engineering management is becoming a harder career path
* Why generalists and problem-solvers are becoming increasingly valuable
* The danger of AI-generated code, poor engineering practices, and losing fundamental skills
* The ideas behind Gregor’s new O’Reilly book, The Multiplier Mindset
* How Gregor grew his Engineering Leadership Newsletter to over 200,000 subscribers
* Why slowing down and building the right things matters more than ever
Chapters
00:00 – The Multiplier Mindset01:00 – Gregor’s background and journey into engineering08:00 – Learning engineering fundamentals before AI11:00 – From software engineer to engineering leader17:00 – Why technical skills alone aren’t enough20:00 – AI, layoffs, and the future of engineering jobs27:00 – The Multiplier Mindset and writing a book with O’Reilly32:00 – Growing a newsletter to 200,000 subscribers36:00 – Junior, senior, staff, and principal engineers in the AI era41:00 – The changing path to CTO44:00 – AI-generated code and engineering quality49:00 – Why writing code was never the real bottleneck52:00 – Final advice for engineers
Find Gregor:
Engineering Leadership Newsletter:
Book: The Multiplier Mindset by Gregor Ojstersek (O’Reilly)
Thanks for reading Data Engineering Central! This post is public so feel free to share it.
This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribeLife After Big Tech
What happens when you finally achieve financial independence, walk away from a successful career in Big Tech, and have the freedom to do whatever you want? Is it everything you imagined it would be?In this …
Notas del episodio
What happens when you finally achieve financial independence, walk away from a successful career in Big Tech, and have the freedom to do whatever you want? Is it everything you imagined it would be?
In this episode of the Data Engineering Central Podcast, I’m joined once again by Kevin Martin, who spent nearly 13 years working at Apple before deciding he’d had enough of the corporate grind. When we last spoke, Kevin had just left Apple and was preparing to spend much of 2026 backpacking some of America’s most beautiful wilderness.
Thanks for reading Data Engineering Central! This post is public so feel free to share it.
Well, Kevin is back, and he’s got some stories to tell.
Since our last conversation, Kevin has completed hundreds of miles of backpacking, including the Benton MacKaye Trail and the legendary John Muir Trail. Along the way, he spent a terrifying night with multiple black bears circling his tent in the Smoky Mountains, experienced the incredible beauty of the Sierra Nevada, and learned some unexpected lessons about himself.
One of the most interesting things Kevin discovered was that leaving the corporate world doesn’t necessarily mean leaving behind the mindset that made you successful there. After years of working in a high-performance environment at Apple, he found himself approaching retirement and backpacking with the same intensity, planning, and expectations that he’d brought to his career.
* We dig into that realization and what it means to actually enjoy the financial freedom you’ve worked so hard to achieve.
Of course, we also wander into plenty of other topics, including the changing landscape of Big Tech, AI-driven layoffs, whether anyone’s job is really safe anymore, and the importance of building financial independence in an increasingly uncertain world.
* We talk about lifestyle inflation, the FIRE movement, the psychology of money, and why doing difficult things in the outdoors can completely change your perspective on everyday problems.
Perhaps my favorite part of the conversation is our discussion about happiness and the tendency to constantly chase the next big milestone. Whether it’s a promotion, a financial goal, retirement, or climbing a mountain, there’s always something else waiting on the horizon.
At what point do we stop chasing the next thing and actually enjoy the life we’ve built?
It’s a fun, thoughtful conversation that goes well beyond technology and money. If you work in tech, dream about financial independence, love the outdoors, or simply find yourself wondering what life might look like outside the traditional career path, I think you’ll enjoy this one.
Grab a cup of coffee and join us!
Thanks for reading Data Engineering Central! This post is public so feel free to share it.
This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribeAI Is a Mirror: MLOps, LLMOps & the Future of AI Engineering | Maria Vechtomova
What does it actually take to build AI systems that work in production?In this episode of Data Engineering Central, I sit down with Maria Vechtomova, AI Engineering Lead, co-founder of CAUCHY, Databricks MVP …
Notas del episodio
What does it actually take to build AI systems that work in production?
In this episode of Data Engineering Central, I sit down with Maria Vechtomova, AI Engineering Lead, co-founder of CAUCHY, Databricks MVP, and author of MLOps with Databricks.
Maria has spent more than 12 years working across data, machine learning, MLOps, and AI engineering. We talk about how production ML has evolved from the early days of homegrown model platforms to today’s world of LLMs, agents, Databricks, MLflow, and AI-assisted coding.
We dig into why LLM applications can be significantly harder to evaluate and monitor than traditional ML, the growing importance of tracing and observability, and why governance becomes more complicated when agents can interact with data and other systems. Maria also explains why many of the problems we’re facing with AI aren’t actually new — AI has simply made them much harder to ignore.
* We also get into AI coding tools, testing AI-generated code, Databricks and Unity Catalog, the rapidly changing LLMOps ecosystem, and what engineers should focus on as AI changes the way software is built.
One of my favorite ideas from the conversation: AI is a mirror. It can help you write a lot more code, a lot faster — but whether that code is actually good still depends on the engineer behind it.
Maria also shares what she thinks matters most for engineers entering the field today: common sense, soft skills, and business sense.
Topics include: MLOps and LLMOps, production AI, Databricks and MLflow, AI agents, observability and tracing, model and LLM evaluation, AI governance, Unity Catalog, AI-generated code, testing, and the future of AI engineering.
Thanks for reading Data Engineering Central! This post is public so feel free to share it.
This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribeThe Future of Data Engineering in the Age of AI | Erfan Hesami
I sat down with Erfan Hesami, founder of Pipeline to Insights, to talk about where data engineering is headed as AI and agents become part of everyday engineering work.We dig into the changing role of the data …
Notas del episodio
I sat down with Erfan Hesami, founder of Pipeline to Insights, to talk about where data engineering is headed as AI and agents become part of everyday engineering work.
We dig into the changing role of the data engineer, the growing overlap between data and AI engineering, and why strong fundamentals may actually become more important as AI makes writing code cheaper and easier.
We also talk about AI agents inside modern data platforms, governance and security, AI-generated code and technical debt, controlling AI costs, the importance of human judgment, and why communication and business skills are becoming increasingly valuable for engineers.
* Erfan also shares the story behind Pipeline to Insights, how writing helped accelerate his own learning, and how consistency helped him grow the publication from its first post in October 2024 into a successful data and AI community.
If you’re a data engineer, analytics engineer, AI engineer, or someone trying to break into data, this conversation offers a grounded look at what skills are likely to matter as AI changes how we build software and data systems.
Thanks for reading Data Engineering Central! This post is public so feel free to share it.
This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribeBig Tech Laid Him Off. He Realized He Didn’t Need a Job. — Asian Dad Energy
What happens when you spend decades building a successful career in tech, finally reach Big Tech, and then suddenly get laid off?In this episode of the Data Engineering Central Podcast, I sit down with …
Notas del episodio
What happens when you spend decades building a successful career in tech, finally reach Big Tech, and then suddenly get laid off?
In this episode of the Data Engineering Central Podcast, I sit down with AsianDadEnergy to talk about his journey from writing BASIC on an IBM 386 and working with COBOL mainframes to consulting, becoming a chief architect in Big Tech, and eventually losing his job after seven years.
But the layoff wasn’t the end of the story.
Years earlier, the 2008 financial crisis convinced him and his wife to live below their means, eliminate debt, save aggressively, and invest. By the time his Big Tech layoff arrived, those investments had grown enough to cover their living expenses—and he eventually realized he didn’t actually need to find another job.
We also get into AI and the future of software engineering, why junior engineers may be particularly vulnerable, whether AI is taking away some of the struggle that traditionally made programmers better, and why communication and other human skills are becoming survival skills for engineers—not just skills you need to get promoted.
Finally, we talk about separating your identity from your job, building multiple streams of income, creating content on YouTube and Substack, and why there is a lot more to life than what happens in front of a computer screen.
https://www.youtube.com/@AsianDadEnergy
Big Tech. AI. Layoffs. Financial independence. Content creation. And figuring out what actually matters when the career you’ve spent decades building suddenly disappears.
I think this one will resonate with a lot of people working in tech right now.
Thanks for reading Data Engineering Central! This post is public so feel free to share it.
This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribeSpark Isn't Going Anywhere. So They Rebuilt It in Rust. — Shehab Amin, CEO of LakeSail
In this episode, I sit down with Shehab Amin, co-founder and CEO of LakeSail, to talk about what happens when you decide to rebuild one of data engineering’s most important technologies, Apache Spark, in Rust … …
Notas del episodio
In this episode, I sit down with Shehab Amin, co-founder and CEO of LakeSail, to talk about what happens when you decide to rebuild one of data engineering’s most important technologies, Apache Spark, in Rust … sorta.
We dig into Shehab’s journey from writing C++ as a kid to studying computer science at Berkeley, becoming a founder, and eventually building Sail. We talk about why Spark has remained so sticky, what Rust, Apache Arrow, and DataFusion are changing about data infrastructure, and why rebuilding Spark compatibility turned out to be much harder—and more interesting—than expected.
We also get into the increasingly complicated modern data stack: Delta Lake vs. Iceberg, catalogs, streaming vs. batch, agentic coding, and what happens when AI agents start interacting directly with data and compute.
Most importantly, we explore LakeSail’s bigger idea: what if the data pipelines companies already have could become the foundation for their AI pipelines?
A wide-ranging conversation about Spark, Rust, AI, and where data engineering goes from here.
Thanks for reading Data Engineering Central! This post is public so feel free to share it.
This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribeAgentic Data Engineering Is Here — But Can It Close the Loop?
AI can now write SQL, generate Python, review pull requests, and build entire applications with a few well-crafted prompts. Yet despite all of that progress, data engineering still feels oddly resistant to the …
Notas del episodio
AI can now write SQL, generate Python, review pull requests, and build entire applications with a few well-crafted prompts. Yet despite all of that progress, data engineering still feels oddly resistant to the full AI revolution. Pipelines break. Data models drift. Production data needs validation. Someone still has to close the loop.
In this episode of the Data Engineering Central Podcast, I sat down with Hugo Lu, founder and CEO of Orchestra, to talk about what “agentic data engineering” actually means beyond the buzzwords. Rather than another conversation about AI replacing engineers, we dug into the infrastructure that’s still missing before autonomous data platforms become reality.
* Hugo shares his unlikely path into data engineering, from investment banking to helping build data systems at Juul, before eventually founding Orchestra.
Thanks for reading Data Engineering Central! This post is public so feel free to share it.
What started as an effort to simplify orchestration has evolved into a broader vision where AI agents don’t just generate code, but can safely execute work, observe the results, validate changes, and iteratively improve pipelines inside secure environments.
* That ability to observe outcomes, what many are calling “closing the loop,” may be the missing ingredient preventing today’s coding agents from becoming truly autonomous.
We also explore why data engineering has not experienced the same AI disruption as traditional software engineering. While AI can produce application code remarkably well, production data systems introduce a completely different set of problems. Branching production data, validating schema changes, testing transformations against realistic datasets, and understanding business semantics all remain difficult challenges that require much more than simply generating code.
The conversation naturally turns toward the future of the profession itself. We discuss whether junior engineers are losing the traditional apprenticeship path, how senior engineers are shifting from writing code to reviewing AI-generated work, and why decades of experience debugging production systems may actually become even more valuable in an AI-first world. Rather than eliminating engineering expertise, AI may simply be changing where that expertise is applied.
Data Engineering Central is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.
Finally, we dive into the rapidly changing data infrastructure landscape. From DuckDB and Polars to serverless compute, Iceberg, semantic layers, AI-native orchestration, and the growing concern over rising LLM token costs, we discuss where the industry appears to be heading and which trends are likely to stick long after today’s hype cycle fades.
If you’ve been wondering what comes after Orchestration, how AI agents will actually manage production data pipelines, or whether data engineering itself is about to undergo its biggest transformation in a decade, I think you’ll enjoy this conversation.
In this episode we discuss
* What “agentic data engineering” actually means.
* Why AI still struggles to fully automate data engineering.
* The importance of closing the loop with production feedback.
* Why orchestration may become the operating system for AI agents.
* The changing role of data engineers in an AI-first world.
* Why junior engineers face a very different career path than previous generations.
* DuckDB, Polars, serverless data platforms, and where modern infrastructure is heading.
* Whether today’s dependence on proprietary LLMs will create tomorrow’s vendor lock-in.
* How Orchestra is building infrastructure for AI-native data platforms.
Thanks for reading Data Engineering Central! This post is public so feel free to share it.
This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribeWhat Happens When a Software Engineer Builds a Company Alone?
When people think about starting a software company, they usually imagine raising venture capital, hiring engineers, and growing a team as quickly as possible.Michael Drogalis chose a different path.After …
Notas del episodio
When people think about starting a software company, they usually imagine raising venture capital, hiring engineers, and growing a team as quickly as possible.
Michael Drogalis chose a different path.
After helping build technology in the Kafka ecosystem, founding a startup that was ultimately acquired by Confluent, and leading product for stream processing, he walked away from big tech to see if one person could build a serious B2B software company.
* The result became ShadowTraffic, a product that helps engineering teams generate realistic production traffic for testing, demos, and development.
Data Engineering Central is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.
In this conversation, we talk about much more than streaming systems. We discuss why most engineers underestimate the importance of understanding customers, how AI is changing software development without replacing experienced engineers, what it takes to market technical products, and why writing publicly can become one of the biggest accelerators of your career.
If you’ve ever considered building your own product, becoming a solopreneur, or simply becoming a better engineer, this conversation is packed with practical advice from someone who’s actually done it.
I think this episode has broad appeal beyond data engineering. It’s really about engineering careers, entrepreneurship, and building products that solve real problems, which should make it one of your more accessible interviews.
* Building and selling a Kafka startup
* Life inside Confluent during its rapid growth
* Why Michael left big tech to become a solopreneur
* Building ShadowTraffic from scratch
* Finding customers before writing code
* Why marketing matters more than most engineers think
* Using AI without becoming dependent on it
* The future of software engineering
* Writing online and building an audience
* Advice for engineers who want to start their own business
Thanks for reading Data Engineering Central! This post is public so feel free to share it.
This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribeThe Creator of Pandas on AI, Apache Arrow, and the Future of Software Engineering
The creator of Pandas and co-creator of Apache Arrow, Wes McKinney, joins the Data Engineering Central Podcast for an in-depth conversation about how modern data engineering came to exist, where AI is taking …
Notas del episodio
The creator of Pandas and co-creator of Apache Arrow, Wes McKinney, joins the Data Engineering Central Podcast for an in-depth conversation about how modern data engineering came to exist, where AI is taking software development, and why good engineering still matters more than ever.
We start with Wes’ journey from building GoldenEye fan websites as a teenager to creating Pandas while working at a quantitative hedge fund, and eventually launching Apache Arrow, one of the foundational technologies behind today’s modern data ecosystem. Along the way, we discuss Cloudera, Parquet, DuckDB, DataFusion, Spark, and how the industry evolved from Hadoop to today’s lakehouse architectures.
Thanks for reading Data Engineering Central! This post is public so feel free to share it.
The second half of the conversation dives deep into AI. Wes explains why large language models make experienced engineers more productive but won’t magically replace software engineering, why architecture and good taste are becoming more valuable than writing individual lines of code, and why projects like DuckDB and
* Apache Arrow remains incredibly difficult to recreate with AI alone. We also discuss open-source, local AI models, token costs, multimodal data platforms, and what new engineers should focus on to build long-term careers in software and data.
If you’re a data engineer, software engineer, architect, engineering leader, or simply interested in where AI is taking our industry, this is a conversation you won’t want to miss.
Topics We Cover
* How Pandas was created
* The story behind Apache Arrow
* Why Arrow became the standard for modern data systems
* DuckDB, DataFusion, and the next generation of data tools
* The evolution from Hadoop to lakehouses
* Why AI won’t replace great software engineers
* Architecture vs. coding in the AI era
* Building trustworthy open source software
* The future of data engineering
* Advice for new engineers entering the industry
If you enjoy conversations with the people building the future of data engineering, subscribe for more interviews with the creators of the tools we use every day.
Data Engineering Central is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.
This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribeFrom COBOL to Copilot: 30 Years of Data, BI, and AI with David Langer
What happens when someone who started programming on a Commodore 64 watches AI reshape the entire data industry?In this episode of the Data Engineering Central Podcast, I sit down with Dave Langer to explore …
Notas del episodio
What happens when someone who started programming on a Commodore 64 watches AI reshape the entire data industry?
In this episode of the Data Engineering Central Podcast, I sit down with Dave Langer to explore nearly three decades of experience across software engineering, business intelligence, analytics, data science, and AI.
Dave’s career spans COBOL programming on IBM mainframes, enterprise architecture, Microsoft’s Xbox division, machine learning, startup leadership, authorship, and building one of the largest personal brands in the data space.
We discuss why many of the biggest problems in data haven’t changed, even as the tools continue to evolve. We dive into the reality behind self-service analytics, the importance of dimensional modeling, what organizations are getting wrong about AI adoption, and why developing strong analytical skills matters more than ever.
* Dave also shares practical advice for data professionals navigating the AI era, explaining why tools like Copilot should be viewed as partners rather than replacements.
If you’re a data analyst, BI developer, data scientist, or data engineer wondering what the future holds, this conversation offers both perspective and optimism.
Thanks for reading Data Engineering Central! This post is public so feel free to share it.
What We Cover
* How Dave got started programming on a Commodore 64
* The transition from COBOL to modern analytics
* Why the core problems in data haven’t changed
* The evolution of business intelligence and dashboards
* How Dave discovered machine learning
* Why data science needs more than just Jupyter notebooks
* The limitations of self-service analytics
* Why semantic layers and governance matter for AI
* Advice for staying relevant as AI reshapes the industry
* Building a personal brand in data
* Writing a technical book and becoming an independent creator
Connect with Dave:
* LinkedIn: https://www.linkedin.com/in/davelanger/
* Substack: The DIY Data Scientist
* Book: Python and Excel Step-by-Step
Subscribe for more conversations on data engineering, analytics, AI, and building a career in modern data.
Data Engineering Central is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.
This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribeSemantic Layers, Agents, and the Future of Analytics
In this episode of the Data Engineering Central Podcast, I sit down with David Jaitillake to explore the future of data engineering, analytics, and AI. David has spent nearly two decades working across data …
Notas del episodio
In this episode of the Data Engineering Central Podcast, I sit down with David Jaitillake to explore the future of data engineering, analytics, and AI. David has spent nearly two decades working across data teams, from analyst roles in the early SQL Server days to leading teams, founding startups, serving as VP of AI at Cube, and now co-founding Quarry.
We discuss why semantic layers have suddenly become one of the most important concepts in modern data platforms, how tools like Claude Code are transforming engineering workflows, and why the core problems in data haven’t really changed despite massive advances in technology.
David shares his perspective on where agentic workflows are headed, what AI means for junior engineers entering the field, and why experienced practitioners may be more valuable than ever before. We also dive into the evolution of data platforms, lessons learned from startups, the promise of tools like DuckDB and MotherDuck, and how organizations should think about adopting AI responsibly.
Thanks for reading Data Engineering Central! This post is public so feel free to share it.
Whether you’re a data engineer, analytics engineer, engineering leader, or someone trying to understand where the industry is headed, this conversation offers a practical and honest look at what’s coming next.
What We Cover
* David’s journey from analyst to startup founder
* The rise of semantic layers and why they matter
* Why data modeling is still critical in the AI era
* How AI coding agents are changing engineering work
* What Claude Code is enabling today
* The future of agentic data pipelines
* Why DuckDB and MotherDuck are gaining traction
* The challenges facing junior engineers
* Career advice for data professionals at every stage
* Whether David is optimistic about the future of AI and data
Connect with David:
* LinkedIn: https://www.linkedin.com/in/david-jayatillake/
* Substack:
Thanks for reading Data Engineering Central! This post is public so feel free to share it.
This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribeThe Future of the Lakehouse: Delta Lake, Rust, and Data Platforms at Scale
In this episode of the Data Engineering Central Podcast, I sit down with Ethan, a maintainer of delta-rs and an expert in modern lakehouse architecture working in the pharmaceutical industry.We discuss Ethan’s …
Notas del episodio
In this episode of the Data Engineering Central Podcast, I sit down with Ethan, a maintainer of delta-rs and an expert in modern lakehouse architecture working in the pharmaceutical industry.
We discuss Ethan’s journey into tech and data engineering, the evolution of open table formats like Delta Lake and Apache Iceberg, and what it actually takes to build scalable enterprise data platforms in highly regulated environments like big pharma.
We also dive into:
* delta-rs and the future of Delta Lake outside Spark
* Lakehouse architecture and open catalogs
* Rust in the modern data ecosystem
* Data platform governance and scalability
* Enterprise analytics and infrastructure
* The future of agentic analytics and AI-enabled data systems
* Lessons learned building large-scale data platforms
If you’re interested in modern data engineering, open source infrastructure, lakehouses, or the future of analytics engineering, this is a great conversation.
Thanks for reading Data Engineering Central! This post is public so feel free to share it.
This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribeFrom Failure to AWS: What Actually Makes a Great Engineer
Victor Moreno went from failing out of a top CS program to becoming a senior engineer at AWS, and his story says a lot about what actually matters in software engineering today.In this conversation, we go deep …
Notas del episodio
Victor Moreno went from failing out of a top CS program to becoming a senior engineer at AWS, and his story says a lot about what actually matters in software engineering today.
In this conversation, we go deep into the reality behind the AI hype, what makes engineers valuable (it’s not writing more code), and why the future of the field looks very different from what most people think.
We talk about the shift from coding to system thinking, why fundamentals matter more in the age of AI, and how junior engineers will need to evolve as tools like Claude and ChatGPT take over the “grunt work.”
Victor also shares hard-earned lessons from teaching, startups, consulting, and building systems at AWS, along with practical advice for engineers looking to stand out in a crowded, uncertain job market.
This is not a hype conversation. It’s a real look at where things are going.
Thanks for reading Data Engineering Central! This post is public so feel free to share it.
🔑 What We Cover
* Why AI is making fundamentals more important, not less
* The biggest mistake engineers make is chasing promotions
* How to actually become a high-impact engineer
* Why does doing more Jira tickets not matter
* What’s broken about today’s interview process
* The future of junior engineers in an AI world
* Tactical vs strategic engineering (and why it matters)
* Why most AI-generated code is still “low quality.”
* How to think about career growth in a weird job market
💡 Key Takeaway
The best engineers aren’t the ones writing the most code—they’re the ones who understand systems, think long-term, and can drive decisions.
Data Engineering Central is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.
This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribeHow Real Data Engineers Think (Beyond Tools and Hype)
In this episode of the Data Engineering Central Podcast, I sit down with Yordan Ivanov, Head of Data Engineering at a growing fintech company, to talk through what it actually looks like to build and run real …
Notas del episodio
In this episode of the Data Engineering Central Podcast, I sit down with Yordan Ivanov, Head of Data Engineering at a growing fintech company, to talk through what it actually looks like to build and run real data platforms in production.
Yordan’s story starts like many of mine, early programming, gaming, PHP, Linux servers—but what makes this conversation interesting is how he evolved from a generalist software engineer into a data engineering leader without even realizing it at first.
We spend a lot of time digging into what actually matters in modern data engineering, and it’s not the tools.
Thanks for reading Data Engineering Central! This post is public so feel free to share it.
We talk about:
* Why the industry went too far into complexity and is now swinging back toward simplicity
* The reality of running a data platform at scale (and why most teams waste time maintaining tools instead of delivering value)
* How to think about migrations the right way without breaking everything
* The difference between junior, mid, and senior engineers—and why ambiguity tolerance and impact matter more than coding ability
* Why “perfect” engineering is a trap and how to actually ship work that matters
We also get into AI, and Yordan has one of the more grounded takes you’ll hear right now. Most companies aren’t even close to ready for AI, and the idea that it’s replacing engineers anytime soon misses the bigger problem: messy data, unclear metrics, and weak foundations.
Check out Yordan’s Substack below!
We also talk about:
* How AI is actually used on real teams today (not Twitter hype)
* Why juniors with AI can be risky without strong processes
* How to think about code reviews, testing, and slowing down when it matters
On top of that, we dig into content creation, Substack, and what it takes to stand out in a world full of generic AI-generated content. Yordan’s approach is simple: write from real experience or don’t write at all.
This is one of those conversations that cuts through a lot of noise and gets back to fundamentals, how to think, how to build, and how to grow as an engineer in a rapidly changing space.
Data Engineering Central is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.
This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribeData, AI, and DuckDB
In this episode of the Data Engineering Central Podcast, I sit down with Jacob Matson, Developer Advocate at MotherDuck, to unpack one of the most interesting shifts happening in data engineering right …
Notas del episodio
In this episode of the Data Engineering Central Podcast, I sit down with Jacob Matson, Developer Advocate at MotherDuck, to unpack one of the most interesting shifts happening in data engineering right now.
Jacob didn’t start in tech the way most people expect. He began in accounting, working with Excel and financial systems, before slowly realizing that the real problem he loved solving wasn’t finance, it was data pipelines. That path eventually led him deep into SQL Server, data warehousing, and ultimately to DuckDB, a tool that fundamentally changed how he thought about processing data.
* What we get into is bigger than just tools, though.
We talk about why DuckDB exploded in popularity, what it gets right that traditional databases and even modern cloud warehouses struggle with, and why the industry may be swinging back toward simplicity after years of over-engineered “modern data stacks.”
There’s a really interesting thread here around how engineers accidentally created too much complexity, and now tools like DuckDB are winning by removing it.
We also go deep on the evolution of the data stack itself. From SQL Server’s “everything in one box” model, to the unbundled chaos of the modern stack, and now back toward a more unified, simpler approach. Jacob shares how MotherDuck is thinking about that shift and where things are headed next.
* One of the more important parts of this conversation is around AI.
There’s a strong argument here that AI doesn’t kill data engineering; it massively expands it. Instead of fewer queries being written, we may be heading toward a world where AI agents generate orders of magnitude more queries than humans ever could. That flips a lot of assumptions on their head, especially around things like data modeling, which suddenly becomes more important, not less.
We also talk about:
* Why most Spark workloads are overkill
* When single-node tools like DuckDB actually win
* The real tradeoffs behind Lakehouse architectures
* Why data modeling is still critical in an AI-driven world
* How engineers should think about building in 2026 and beyond
This is one of those conversations that helps you zoom out and see where things are actually going, not just what tools are trending this week.
If you’re building data platforms, experimenting with AI, or just trying to simplify your stack, this one is worth your time.
Thanks for reading Data Engineering Central! This post is public so feel free to share it.
This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribeWhy I Left Facebook to Work for Myself
In this episode of the Data Engineering Central Podcast, I sit down with Ben Rogojan to talk about the real story behind data engineering careers, Big Tech, and what’s changing right now.Ben shares how he went …
Notas del episodio
In this episode of the Data Engineering Central Podcast, I sit down with Ben Rogojan to talk about the real story behind data engineering careers, Big Tech, and what’s changing right now.
Ben shares how he went from working in kitchens… to data engineering… to Facebook… and eventually walking away from it all to build his own consulting business.
And yeah, it wasn’t all glamorous.
“I was making the same money as Facebook… and I hated my life.”
We get into the stuff most people don’t talk about:
* What it’s actually like working in Big Tech
* Why high-paying jobs can still burn you out
* How he transitioned into consulting (and what people get wrong)
* The reality of modern data stacks and tool sprawl
* Whether data engineering is changing because of AI
* Why fundamentals still matter more than ever
Thanks for reading Data Engineering Central! This post is public so feel free to share it.
We also go deep on where the industry is heading:
* Is the “modern data stack” breaking down?
* Are tools like DuckDB actually replacing warehouses?
* Is data modeling dead… or just not trendy anymore?
* What AI is really changing (and what it’s not)
If you’re trying to break into data, grow your career, or figure out where things are headed… this is one of the more honest conversations you’ll hear.
Data Engineering Central is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.
Ben also runs a course and community for those interested in getting into consulting.
This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribeAcademic → CTO: What Actually Matters in Data (Matthew Housley)
Most companies don’t have a tooling problem. They have a foundation problem.In this episode, I sit down with Matthew Housley, a famed co-author of Data Engineering Fundamentals and former CTO of Ternary Data …
Notas del episodio
Most companies don’t have a tooling problem. They have a foundation problem.
In this episode, I sit down with Matthew Housley, a famed co-author of Data Engineering Fundamentals and former CTO of Ternary Data, to talk about what actually makes data teams successful and why so many organizations get it wrong despite having modern stacks, cloud platforms, and expensive dashboards.
* Matthew’s path is a little different than most. He started in academia as a mathematics instructor before moving into industry as a data scientist at Overstock.com, and eventually leading data strategy and analytics as a CTO. That mix of academic rigor and real-world execution gives him a very clear perspective on where things break down.
We get into the gap between data science and real business impact, why analytics foundations matter more than flashy models, and what companies consistently underestimate when building out data platforms. We also talk about what it actually looks like to transition from academia to industry, and how that shapes how you think about data problems at scale.
Data Engineering Central is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.
If you’ve ever felt like your data stack should be delivering more value than it is, this conversation will probably hit close to home.
⏱️ Topics we cover:
* Why most analytics efforts fail before they even start
* The difference between “doing data” and delivering value
* Data science vs data engineering vs analytics reality
* Academic thinking vs industry execution
* What CTOs actually care about when it comes to data
* Building foundations that don’t fall apart six months later
Thanks for reading Data Engineering Central! This post is public so feel free to share it.
This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribeAI Isn’t Replacing Curious Developers
AI isn’t just changing how we write code. It’s changing what it even means to build software.In this episode of the Data Engineering Central Podcast, I sit down with Neil Roberts — a developer who’s been …
Notas del episodio
AI isn’t just changing how we write code. It’s changing what it even means to build software.
In this episode of the Data Engineering Central Podcast, I sit down with Neil Roberts — a developer who’s been through every major wave of the web, from BASIC on an Atari to modern TypeScript, and now deep into LLMs and agentic workflows.
This is not another surface-level “AI will change everything” conversation. We get into what is actually happening right now, where it works, where it completely breaks, and what developers are getting wrong.
* We talk about why front-end and UX matter more than ever in an AI world, why most people misunderstand agents, and what real day-to-day workflows with LLMs actually look like.
* There’s also a hard look at who benefits from AI, who falls behind, and whether we are quietly building fragile systems that we don’t fully understand.
If you’re a developer trying to figure out where this is all going, this is one of those conversations worth paying attention to.
Data Engineering Central is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.
Expect to learn:
* Why AI is as much a UX problem as it is a backend problem
* What “agents” actually mean in practice, not in demos
* Where LLM workflows are useful today and where they fail hard
* Whether junior developers should be worried or excited
* How building apps changes when AI is part of the system
* What developers should actually be doing right now to stay relevant
Neil also has a podcast, The Skill Tree, on AI and agentic-specific topics.
We also get into a bigger question most people are avoiding:
* Are we heading toward AI-assisted coding… or AI-orchestrated systems where developers become supervisors?
* And maybe more importantly… which side of that shift do you want to be on?
Thanks for reading Data Engineering Central! This post is public so feel free to share it.
This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribeAI Is Changing Data Engineering Fast
In this episode of the Data Engineering Central Podcast, I sit down with Andreas Kretz to break down what is really happening in the industry right now. We go far beyond surface-level AI hype and talk about how …
Notas del episodio
In this episode of the Data Engineering Central Podcast, I sit down with Andreas Kretz to break down what is really happening in the industry right now. We go far beyond surface-level AI hype and talk about how data engineering actually works in the real world, what skills still matter, and where most engineers are wasting time.
Andreas shares his full journey from industrial IoT and working at Bosch to building one of the largest data engineering education platforms in the world, training over 2,000 students and reaching more than 100,000 engineers globally. We get into what production data systems actually look like, why most learning paths are broken, and how AI is reshaping the role of the modern data engineer.
Thanks for reading Data Engineering Central! This post is public so feel free to share it.
* We also dig into the uncomfortable truths. AI can write code, but it cannot replace thinking. Most engineers focus too much on tools and not enough on problem-solving, system design, and communication. That gap is only getting bigger.
If you are trying to figure out how to stay relevant in data engineering, or you are just getting started and want to avoid years of wasted effort, this conversation will change how you think about your career.
Today’s podcast is sponsored by Estuary.
Without them, content like this isn’t possible. The best way to support this Newsletter is to check out what Estuary has to offer and click the links below.
Build millisecond-latency, scalable, future-proof data pipelines in minutes.
Estuary is the Right-Time Data Platform that integrates all of the systems you use to produce, process, and consume data. Also, providing best-in-class CDC (Change Data Capture).
Estuary unifies today’s batch and streaming paradigms so that your systems, current and future, are synchronized around the same datasets, updating in milliseconds.
What we cover:
* Why most data engineers are learning the wrong things
* The shift from coding to problem-solving and system design
* How AI is actually changing data engineering workflows
* Why courses and tutorials are becoming less effective
* The difference between real production systems and “toy projects.”
* The future of data engineering jobs and whether AI will replace them
* Why fundamentals still matter more than ever
One of the biggest takeaways is simple. The tools will keep changing, but the problems stay the same. The engineers who win are those who understand systems, ask better questions, and connect business problems to real solutions.
Links:
* Learn Data Engineering Academy:
https://learndataengineering.com
* Sponsor: https://estuary.dev
Data Engineering Central is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.
This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribeMost Data Teams Are Doing It Wrong
Most data teams think they’re building value. In reality, they’ve become ticket queues.In this episode, Chris Gambill explains his storied career in tech and data through the years, dealing with data at Fortune …
Notas del episodio
Most data teams think they’re building value. In reality, they’ve become ticket queues.
In this episode, Chris Gambill explains his storied career in tech and data through the years, dealing with data at Fortune 500 company scale, and breaking out on his own.
We cover career growth, what separates senior engineers from true strategic operators, and the biggest mistakes people make early on. We discuss the classic problems that have plagued data teams for decades and why it’s all still a struggle.
Today’s podcast is sponsored by Estuary.
Without them, content like this isn’t possible. The best way to support this Newsletter is to check out what Estuary has to offer and click the links below.
Build millisecond-latency, scalable, future-proof data pipelines in minutes.
Estuary is the Right-Time Data Platform that integrates all of the systems you use to produce, process, and consume data. Also, providing best-in-class CDC (Change Data Capture).
Estuary unifies today’s batch and streaming paradigms so that your systems, current and future, are synchronized around the same datasets, updating in milliseconds.
We also dig into Databricks vs Snowflake, what matters and what doesn’t, and how to think about modern data architecture without falling for marketing hype.
* On the AI side, we talk about why most LLMs, in the context of developer lifecycles, have changed how we do data, and also about what human skills cannot be replaced.
If you care about leveling up beyond just building pipelines, this one is for you.
Thanks for reading Data Engineering Central! This post is public so feel free to share it.
This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribeFrom Industrial Data at BASF to Delta Lake Committer
In this episode, Robert Pack walks through his journey from engineering and simulation work to building large-scale data systems across 900+ plants at BASF.We break down what those systems actually looked like …
Notas del episodio
In this episode, Robert Pack walks through his journey from engineering and simulation work to building large-scale data systems across 900+ plants at BASF.
We break down what those systems actually looked like, including ingestion, modeling, and the realities of batch vs real-time in industrial environments.
We also dive into:
* AI Workflows for Developers
* His work as a committer on Delta Lake
* Where lakehouse architecture works and where it falls short
* The transition into Developer Relations at Databricks
This is a grounded, practical conversation about what actually matters when building data platforms.
Today’s podcast is sponsored by Estuary.
Without them, content like this isn’t possible. The best way to support this Newsletter is to check out what Estuary has to offer and click the links below.
Build millisecond-latency, scalable, future-proof data pipelines in minutes.
Estuary is the Right-Time Data Platform that integrates all of the systems you use to produce, process, and consume data. Also, providing best-in-class CDC (Change Data Capture).
Estuary unifies today’s batch and streaming paradigms so that your systems, current and future, are synchronized around the same datasets, updating in milliseconds.
You can find Robert on LinkedIn and GitHub, below.
Thanks for reading Data Engineering Central! This post is public so feel free to share it.
This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribeHe Quit Apple After 13 Years
In this episode of Data Engineering Central, I sit down with Kevin, who spent 13 years working at Apple before walking away at the end of 2025.* Not to jump to another job.* Not to start a company.* But to take …
Notas del episodio
In this episode of Data Engineering Central, I sit down with Kevin, who spent 13 years working at Apple before walking away at the end of 2025.
* Not to jump to another job.
* Not to start a company.
* But to take a step back from everything.
Kevin shares his full journey—from growing up in the suburbs of Atlanta to building a career at Apple, and ultimately reaching the point where he could walk away financially and mentally.
You can follow along with Kevin below.
We dive deep into what it’s really like working in tech: the high salaries, the lifestyle creep, the pressure, and the surprising reality that even people making great money often have no clear financial plan.
This conversation also explores the rise of FIRE (Financial Independence, Retire Early), how Kevin discovered it through Mr. Money Mustache, and why his perspective on it has changed over time.
Thanks for reading Data Engineering Central! This post is public so feel free to share it.
What starts as a path to freedom can easily turn into a scarcity mindset—and that’s something most people don’t talk about.
We also get into:
* Why high income does not equal financial freedom
* The hidden trap of lifestyle inflation in tech
* The simple investing strategy that actually works (and why most people ignore it)
* Why many engineers are “close” to freedom—but never pull the trigger
* The psychology of money, status, and why people stay stuck
* How a failed project and burnout became a turning point
* And how Kevin went from overworked and unhealthy… to climbing mountains and preparing to backpack 1,000 miles
This is not your typical “get rich quick” or “retire at 30” conversation. It’s a grounded, honest look at money, work, and what it actually takes to build a life you don’t need to escape from.
If you work in tech, think about FIRE, or just feel like you’re stuck on the treadmill, this one will hit home.
Thanks for reading Data Engineering Central! This post is public so feel free to share it.
This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribeSpark, AI, and the Future of Data Engineering with Daniel Aronovich
In this episode of Data Engineering Central, I sit down with the founder of DataFlint, Daniel Aronovich, to talk about the realities of working with Apache Spark, distributed data systems, and the future of …
Notas del episodio
In this episode of Data Engineering Central, I sit down with the founder of DataFlint, Daniel Aronovich, to talk about the realities of working with Apache Spark, distributed data systems, and the future of data engineering.
We start with his early journey into tech—how he first discovered large-scale data systems and the lessons he learned from working with real-world Spark workloads.
* The conversation then turns toward the future of data engineering, particularly the growing role of AI in software development and data infrastructure. We discuss why generic AI coding assistants often struggle with complex distributed systems, whether AI will eventually be able to automatically optimize data pipelines, and how the role of the data engineer may evolve in the coming years.
We covered a lot of career advice for new and upcoming data professionals.
We also discuss the origin of DataFlint, a tool designed to help engineers better understand and optimize Spark workloads by analyzing execution plans, logs, and runtime context.
If you work with Spark, large-scale data pipelines, or modern data platforms, this conversation will give you a deeper look into how the data engineering landscape is evolving.
Thanks for reading Data Engineering Central! This post is public so feel free to share it.
This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribeDuckDB, AI, and the Future of Data Engineering
In this episode, I sit down with Matt Martin, Staff Engineer, data architect, ETL practitioner, and author of a new book on DuckDB coming soon, to talk about the past, present, and future of data …
Notas del episodio
In this episode, I sit down with Matt Martin, Staff Engineer, data architect, ETL practitioner, and author of a new book on DuckDB coming soon, to talk about the past, present, and future of data engineering.
Matt has spent decades building and architecting data platforms across technologies such as SQL Server, Oracle, DB2, Hadoop, Redshift, and BigQuery, and now focuses on modern tools such as DuckDB and single-node analytics.
We discuss how the data industry has evolved, what actually makes data platforms succeed, and where tools like DuckDB, Polars, Databricks, and Snowflake fit into the future of analytics.
We also dive into the impact of AI on coding and data engineering, and whether distributed compute clusters will remain dominant — or if more workloads will move toward high-performance single-node systems.
Topics Covered
* Matt’s early career and journey into data engineering
* The evolution of data warehousing and ETL frameworks
* Traditional enterprise data systems vs modern cloud platforms
* DuckDB and the rise of single-node analytics
* Polars vs DuckDB: where each tool shines
* Databricks vs Snowflake
* AI-assisted coding and its impact on engineers
* The current data engineering job market
* Lessons learned from decades of building data systems
* Writing a book on DuckDB
This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribeWhat Decades in Software Engineering Teaches You
In this episode of Data Engineering Central, I sit down with a veteran Software Engineer John Crickett; with decades of experience in the industry to unpack what really matters in building a long and successful …
Notas del episodio
In this episode of Data Engineering Central, I sit down with a veteran Software Engineer John Crickett; with decades of experience in the industry to unpack what really matters in building a long and successful engineering career.
We talk about how he first got into software, the early jobs and tools that shaped his thinking, and the massive technology shifts he’s witnessed across decades of engineering—from early stacks and tools to today’s AI-assisted workflows.
* We also dive into the difference between coding and real-world software engineering, what separates junior, senior, and principal engineers, and why many developers misunderstand what it takes to grow in this field.
* We discuss leadership vs individual contributor paths, the origin of his Coding Challenges platform, why algorithm puzzles dominate developer culture, and what actually makes engineers improve quickly.
Finally, we tackle the big question everyone is asking right now: how AI is reshaping software engineering, and what skills will matter most over the next decade.
This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribeData Engineering, AI, and Career Growth
In this episode of the Data Engineering Central Podcast, I sit down with Yuki (Yuki Kakegawa) to talk about his journey into tech, the tools and platforms he’s worked with, and where he thinks data engineering …
Notas del episodio
In this episode of the Data Engineering Central Podcast, I sit down with Yuki (Yuki Kakegawa) to talk about his journey into tech, the tools and platforms he’s worked with, and where he thinks data engineering and AI are headed next.
We cover:
• How Yuki got into tech• Early career lessons and pivots• Tools and technologies he’s worked with over the years• How data engineering has evolved• The impact of AI on software development• What engineers should focus on right now• Advice for those building their careers in data
Yuki shares practical insights on navigating the industry, staying adaptable, and thinking long-term about your technical growth.
If you’re a data engineer, aspiring engineer, or just interested in where AI and modern software are going, this one’s for you.
Yuki writes on …
LinkedIn - https://www.linkedin.com/in/yukikakegawa/
Thanks for reading Data Engineering Central! This post is public so feel free to share it.
🔔 Subscribe for more interviews with leaders in data engineering, AI, and modern data platforms.
This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribeSpark, Lakehouse & AI: A Deep Conversation with Bart Konieczny
In this episode of Data Engineering Central, I sit down with Bart Konieczny — data engineer, distributed systems expert, and well-known author in the Data and Spark ecosystem — for a deep technical conversation …
Notas del episodio
In this episode of Data Engineering Central, I sit down with Bart Konieczny — data engineer, distributed systems expert, and well-known author in the Data and Spark ecosystem — for a deep technical conversation about modern data engineering.
We cover:
* How Bart got into tech and distributed systems
* His journey through different engineering roles
* Spark internals and why they still matter
* The realities of lakehouse architecture
* Streaming vs batch systems
* AI’s impact on data engineering
* What engineers should focus on in 2026
In a world obsessed with abstractions and AI tooling, we explore whether understanding the internals is still worth it — or if the game has fundamentally changed.
If you’re a data engineer, architect, or platform leader trying to navigate the next phase of the lakehouse era, this one’s for you.
Thanks for reading Data Engineering Central! This post is public so feel free to share it.
—
🎙️ Data Engineering Central PodcastHosted by Daniel Beach
If you’re a CTO or data leader looking for help building or optimizing your data platform, reach out — consulting inquiries welcome.
Data Engineering Central is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.
This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribeDevOps vs ClickOps with Maxine Meurer
In this episode of the Data Engineering Central Podcast, I sit down with Maxine Meurer, DevOps engineer, author, and educator behind I Love DevOps, for a wide-ranging conversation about careers, infrastructure …
Notas del episodio
In this episode of the Data Engineering Central Podcast, I sit down with Maxine Meurer, DevOps engineer, author, and educator behind I Love DevOps, for a wide-ranging conversation about careers, infrastructure, automation, and what it actually means to build systems that last.
This isn’t a buzzword-heavy DevOps chat. It’s a grounded, honest discussion between two engineers about how people really get into tech, how careers evolve over time, and why modern infrastructure is as much about systems thinking and human judgment as it is about tools.
We talk through Maxine’s journey from early technical curiosity to hands-on DevOps work, dealing with “ClickOps” to automation-first infrastructure, and how writing and teaching reshaped the way she thinks about engineering.
What we cover in this episode:
* 🛠️ From ClickOps to DevOps — what that transition actually looks like in the real world
* 🧠 Why DevOps is fundamentally about systems and people, not just pipelines and YAML
* 📚 How Maxine went from self-teaching to authoring practical guides like LLMs for Humans and The DevOps Career Switch Blueprint
* 🤯 Common mistakes engineers make when learning DevOps, cloud, and distributed systems
* 🔍 Testing failures, production realities, and where modern infrastructure still breaks down
* 🤖 What AI and LLMs actually change for engineers, and what’s mostly hype
* 🧭 Career advice for engineers without a traditional background
* 🔮 Where DevOps and platform engineering are heading over the next 3–5 years
Throughout the conversation, Maxine brings a refreshing, human-centered perspective to topics that are often over-abstracted or oversold. We dig into the tradeoffs behind tooling choices, the reality of production systems, and the importance of learning how to think, not just what to deploy.
If you’re navigating a DevOps or infrastructure career, wrestling with modern stacks, or trying to make sense of AI’s role in engineering, this episode offers clarity, context, and hard-won insight.
Learn more about Maxine’s work:
* Writing & guides:
* LinkedIn: https://www.linkedin.com/in/maxinemeurer/
* Gumroad resources: https://mameurer.gumroad.com
Thanks for reading Data Engineering Central! This post is public so feel free to share it.
This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribeThe Evolution of Software, Streaming, and Data Engineering with Robin Moffatt
In this episode, I sit down with industry veteran Robin Moffatt — Sr. Principal Advisor in Streaming Data Technologies (Kafka, etc.) and a longtime voice in the data engineering community, to unpack the journey …
Notas del episodio
In this episode, I sit down with industry veteran Robin Moffatt — Sr. Principal Advisor in Streaming Data Technologies (Kafka, etc.) and a longtime voice in the data engineering community, to unpack the journey from old-school data architectures to today’s real-time streaming ecosystems. From early mainframe data processing and COBOL through the rise of Apache Kafka, streaming ETL, and event-driven systems, Robin shares lived experience from across decades of building, scaling, and evolving data platforms.
We dive into:
* 🧠 How the role of software engineering has shifted with the rise of distributed, real-time systems
* 📊 Why event streaming and platforms like Kafka aren’t just messaging systems, but the backbone of modern data architectures
* 🚀 How the community’s tooling and mental models have had to evolve — from static databases and nightly jobs to continuous, always-on streaming applications
* 🤖 A candid look at how AI and real-time data are intersecting, shaping both tooling and expectations for the next decade
* 🔮 Robin’s perspective on where the industry is headed — beyond buzzwords toward real engineering maturity
Along the way, we get historical context, real-world lessons from conference stages and community forums, and a perspective on building resilient, scalable systems that power today’s data-rich applications.
If you’ve ever wondered how we got from batch jobs to continuous event streams, or what it really takes to build modern pipelines that support AI workflows, this conversation with Robin is a must-listen.
For more from Robin:
* 📍 His personal blog & talks:
* 🔗 LinkedIn profile: https://www.linkedin.com/in/robinmoffatt
Thanks for reading Data Engineering Central! This post is public so feel free to share it.
This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribeThe Lakehouse Architecture: Multimodal Data, Delta Lake, and the Future of Data Engineering (with R. Tyler Croy)
In this episode of the Data Engineering Central Podcast, I sit down with R. Tyler Croy for a wide-ranging conversation on the present—and future—of modern data platforms.Tyler is a long-time open-source …
Notas del episodio
In this episode of the Data Engineering Central Podcast, I sit down with R. Tyler Croy for a wide-ranging conversation on the present—and future—of modern data platforms.
Tyler is a long-time open-source contributor to projects such as delta-rs. You can watch him on YouTube, read his blog, or work directly with him through his consultancy, Buoyant Data.
Tyler has spent years deep in the open-source data ecosystem, contributing to projects such as Delta Lake and thinking critically about how real-world data systems are built and maintained. This isn’t a hype-driven conversation—it’s a grounded discussion about what’s working, what’s breaking, and what’s coming next.
We dig into:
* What the Lakehouse architecture gets right—and where it still falls short
* Why multimodal data (text, images, audio, video, embeddings) changes everything
* How open table formats like Delta Lake fit into the next generation of data platforms
* The growing gap between data tooling hype and day-to-day data engineering reality
* What skills and architectural thinking will matter most for data engineers over the next decade
If you’re building or operating modern data platforms—and trying to separate real signal from noise—this episode is for you.
Thanks for reading Data Engineering Central! This post is public so feel free to share it.
This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribe
Aquí están los 40 episodios más recientes de 44. Los anteriores están en el feed del podcast.
Podcasts similares
Más podcasts de Noticias
¿Es tu podcast?
Confírmalo con una entrada en tu feed: aquí recibes un código, lo añades como podcast:txt en el canal de tu feed y nosotros lo comprobamos. Después tu podcast aparecerá como «verificado» en SoundNode.