The State & Future of Apache Kafka (with Anatoly Zelenin)

May 8, 2025 · 1 h 12 min

Show notes

I’m joined this week by one of the authors of Apache Kafka In Action, to take a look at the state of Kafka, event systems & stream-processing technology. It’s an approach (and a whole market) that’s had at least a decade to mature, so how has it done? What does Kafka offer to developers and businesses, and which parts do they actually care about? What have streaming data systems promised and what have they actually delivered? What’s still left to build?

–

Apache Kafka in Action: https://www.manning.com/books/apache-kafka-in-action

Pat Helland, Data on the Inside vs Data on the Outside: https://queue.acm.org/detail.cfm?id=3415014

Out of the Tar Pit: https://curtclifton.net/papers/MoseleyMarks06a.pdf

Martin Kleppmann, Turning the Database Inside-Out: https://martin.kleppmann.com/2015/11/05/database-inside-out-at-oredev.html

Data Mesh by Zhamak Dehghani: https://www.amazon.co.uk/Data-Mesh-Delivering-Data-Driven-Value/dp/1492092398

Quix Streams: https://github.com/quixio/quix-streams

XTDB: https://xtdb.com/

Support Developer Voices on Patreon: https://patreon.com/DeveloperVoices

Support Developer Voices on YouTube: https://www.youtube.com/@developervoices/join

Anatoly’s Website: https://zelenin.de/

Kris on Mastodon: http://mastodon.social/@krisajenkins