AI Engineering Podcast
NeuWhat Do Humans Do Now? Rethinking Software Engineering in the Age of AI Agents
- Folgen
- 82
- Erscheint
- alle zwei Wochen
- Letzte Folge
- 16. Okt. 2026
Über den Podcast
This show is your guidebook to building scalable and maintainable AI systems. You will learn how to architect AI applications, apply AI to your work, and the considerations involved in building or customizing new models. Everything that you need to know to deliver real impact and value with machine learning and artificial intelligence.
Mitwirkende
Folgen 82
What Do Humans Do Now? Rethinking Software Engineering in the Age of AI Agents
Mit Jonathan Ellis
Summary In this episode, I spoke with Jonathan Ellis about how AI agents are changing the economics of software development and forcing teams to rethink where human time is best spent. We explored his argument …
Shownotes
Summary
In this episode, I spoke with Jonathan Ellis about how AI agents are changing the economics of software development and forcing teams to rethink where human time is best spent. We explored his argument that, now that code generation is dramatically faster, traditional human-centered code review is becoming a poor use of attention. Instead, Jonathan makes the case for shifting review earlier in the lifecycle toward specifications, RFCs, and architecture decisions, while investing more heavily in automated validation, static analysis, fuzzing, and other quality gates that can help verify whether software does what it is supposed to do regardless of how the code was written.
Jonathan also shared lessons from building Brokk AI’s tooling, including Bifrost for fast, agent-friendly static analysis and Mjolnir for coordinating multiple coding agents across harnesses and subscriptions. We discussed the emerging need for better agent-oriented collaboration systems, the risks of blindly outsourcing judgment to AI, and the challenge of training newer engineers in a world where agents can do much of the implementation work. Overall, this conversation examines what remains uniquely valuable for humans as agents take over more of the mechanics of software delivery: setting goals, making product and design decisions, building better feedback loops, and deciding what is worth building in the first place.
Announcements- Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems
- Your host is Tobias Macey and today I'm interviewing Jonathan Ellis about how to make the best use of human time when using agents to build software
Interview
- Introduction
- How did you get involved in machine learning?
- Agentic coding has become much more viable this year, which dramatically shifts the bottlenecks in software delivery. What are the ways that you are seeing that impact engineering teams?
- As an industry we have built up numerous expectations and "best practices" around the software lifecycle, with many assumptions about the need for human review before sending code into production. How are you seeing those expectations impact teams who have shifted to agentic coding?
- Advanced teams have been working toward removing the human bottleneck in the validation and deployment path for several years now. What are the lessons that we can all learn from them?
- What are the new tools or practices that we need to build or adopt to gain the confidence that we need to deploy AI-written code to end-users?
- What are the highest leverage points of the software lifecycle that we can focus our limited human attention and review processes?
- What are you building that helps us work with agents to produce better software and build confidence in it?
- What lessons can we learn from the changes happening in software engineering that we can apply to the design and development of other agentic workflows?
- What are the most interesting, innovative, or unexpected ways that you have seen Bifrost/Mjolnir/etc. used?
- What are the most interesting, unexpected, or challenging lessons that you have learned while working on tools for agentic software engineering?
- When is the Brokk suite the wrong choice?
- What do you have planned for the future of Bifrost/Mjolnir/etc.?
Contact Info
Parting Question
- From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?
Closing Announcements
- Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
- Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
- If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.
- To help other people find the show please leave a review on iTunes and tell your friends and co-workers.
Links
- Bifrost
- Mjolnir
- Stop Doing Code Review Blog Post
- SlopCop
- IntelliJ
- ADR == Architecture Decision Record
- RFC == Request for Comments
- Witan Context - Agent-native memory, tasks, and projects (built by Tobias)
- Coverity
- Fuzz Testing
- Apache Cassandra
- Agent Skills
- OpenAI Article - Using Exec Plans To Drive Agents For Longer Tasks
- Vectorize
- SWE-Bench
- Deep-SWE
- Cursor Bench
The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0Kapitel
Kapitel werden geladen …
How to Evaluate RAG Systems When Ground Truth Keeps Changing
Mit Ofer Mendelevitch
Summary In this episode Ofer Mendelevitch shares what it really takes to evaluate RAG systems in production when your source data is incomplete, constantly changing, or difficult to validate against a clean …
Shownotes
Summary
In this episode Ofer Mendelevitch shares what it really takes to evaluate RAG systems in production when your source data is incomplete, constantly changing, or difficult to validate against a clean ground truth. He explores how RAG has evolved from simple “chat with your PDF” demos into enterprise-grade retrieval systems that require robust ingestion pipelines, hybrid search, reranking, multimodal support, access controls, and refresh strategies for large and dynamic document collections. Ofer also explains why evaluation becomes one of the hardest parts of the stack, particularly when retrieval quality, generation quality, and system behavior can all shift as you change chunking, models, prompts, or ranking components.
He digs into practical approaches for offline and online evaluation, including synthetic question generation, sampling production traffic, and LLM-as-a-judge techniques such as UMBRELA and Auto Nuggetizer. Along the way, he discusses the rise of agentic RAG, context engineering, MCP-connected tools, text-to-SQL systems, and the growing role of coding agents in building and maintaining AI pipelines. Ofer closes with his perspective on where retrieval, memory, multimodal systems, and agent orchestration are headed next, along with a reminder that the biggest gaps in AI today are often not just technical, but educational and organizational as well.
Announcements- Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems
- Your host is Tobias Macey and today I'm interviewing Ofer Mendelevitch about how to evaluate your RAG systems when you have incomplete or changing data to grade against
- Introduction
- How did you get involved in machine learning?
- RAG was one of the first concepts to gain traction in the early explosion of LLMs and their application to real-world problems. How would you characterize its role in the current landscape?
- RAG is conceptually straight-forward, but can be quite complex to do well. What are some of the common pitfalls that teams encounter when bringing a retrieval-oriented LLM service into production?
- What are some of the predominant methods to avoid regressions or increased error rates as the data, code, and models change in a given deployment?
- There are numerous software projects and companies that offer various forms of evaluation/validation for RAG workloads. What are the factors that teams should be evaluating when selecting which one(s) to adopt?
- Especially when working with data that is subject to change (user-contributed content, changing products, news-based data, etc.), what are the techniques that teams can use to be sure that the generated responses based on that information is accurate?
- As models have become more powerful the idea of "agentic RAG" started circulating. How is that different from the first iteration of RAG systems?
- What new pressures and challenges does that prevent?
- What are the ways that it can improve or simplify a content-based generative system?
- How do you think about the differentiation of RAG and "context engineering"?
- What are the most interesting, innovative, or unexpected ways that you have seen teams address the production needs of RAG systems?
- What are the most interesting, unexpected, or challenging lessons that you have learned while working on RAG evaluation?
- How do you see the future of RAG, context engineering, and data-driven agents changing in the near to medium term?
Contact Info
Parting Question- From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?
Links- Naive Bayes
- Hands on RAG for Production (affiliate link)
- RAG == Retrieval Augmented Generation
- Vectara
- Hybrid Search
- TF-IDF == Term-Frequency Inverse Document Frequency
- BM25
- UMBRELA
- AutoNuggetizer
- Agentic RAG
- Malloy
The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0Kapitel
Kapitel werden geladen …
Harness Engineering for Reliable, Governed AI Agents
Mit Nikunj Bajaj
Summary In this episode Nikunj Bajaj, co-founder and CEO of TrueFoundry, talks about the challenge of building reliable agents on top of inherently variable foundation models. He explores the idea of the agent …
Shownotes
Summary
In this episode Nikunj Bajaj, co-founder and CEO of TrueFoundry, talks about the challenge of building reliable agents on top of inherently variable foundation models. He explores the idea of the agent harness as everything around the model engine: memory and context management, tool and MCP integration, sandboxed code execution, permissions, observability, and guardrails. Nikunj explained how TrueFoundry approaches enterprise AI as a centralized control plane for token traffic, while TrueForge provides an open source, vendor-neutral harness for building agents without locking teams into a proprietary stack. He talks about the complexity of model routing across both the harness and gateway layers, the tradeoffs between unified APIs and provider-specific capabilities, and the architectural decisions that affect cost, accuracy, and reliability. He highlighted the importance of benchmark-driven validation, durable execution, human-in-the-loop workflows, sandbox security, and especially agent identity and least-privilege access across chains of agents. Overall, this was a deep look at harness engineering as an emerging discipline, and why organizations need more than isolated point solutions if they want to operate agents safely, efficiently, and at scale.
Announcements- Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems
- Your host is Tobias Macey and today I'm interviewing Nikunj Bajaj about the challenges of keeping your agents reliable when the underlying models aren't and how your harness can help
Interview
- Introduction
- How did you get involved in AI?
- Can you describe what TrueForge is and the story behind it?
- A substantial amount of focus over the past few months has been on "harness engineering". What are the key aspects of the harness?
- What are the key challenges of building a harness that works well?
- How do you define what "works well" means given the massive variety of potential use-cases and operating contexts for an agent?
- Given the proliferation of harnesses, what are the key factors that someone should be looking at to determine which one to use?
- What are your opinions on why TypeScript and JS runtimes are such a popular choice for these harnesses?
- What are the key characteristics that you see in harnesses across different language ecosystems?
- Can you describe the implementation and architecture of TrueForge?
- How have the design and goals of the project evolved since you first started working on it?
- What are the elements of a harness that contribute the most to the cost efficiency, accuracy, and reliability (or lack thereof)?
- The general wisdom suggests that changing a model while keeping the same prompts for a given use-case can have wildly different outcomes. How accurate is that assumption today?
- What are the most interesting, innovative, or unexpected ways that you have seen TrueForge used?
- What are the most interesting, unexpected, or challenging lessons that you have learned while working on TrueForge?
- When is TrueForge the wrong choice?
- What do you have planned for the future of TrueForge?
Contact Info
Parting Question
- From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?
Closing Announcements
- Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
- Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
- If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.
- To help other people find the show please leave a review on iTunes and tell your friends and co-workers.
Links
- TrueFoundry
- TrueForge
- FPGA == Field-Programmable Gate Array
- AI Gateway
- Harness Engineering
- MCP == Model Context Protocol
- A2A == Agent-to-Agent Protocol
- Pi Agent Harness
- vLLM Semantic Router
- Testing Pyramid
The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0Kapitel
Kapitel werden geladen …
Kubernetes, Compliance, and Control: The Operational Backbone of AI Sovereignty
Mit Stephen Watt
Summary In this episode of the AI Engineering Podcast, Steven Watt, leader of the Office of the CTO at Red Hat, discusses practical paths to achieving AI sovereignty for organizations. He shares his two-decade …
Shownotes
Summary
In this episode of the AI Engineering Podcast, Steven Watt, leader of the Office of the CTO at Red Hat, discusses practical paths to achieving AI sovereignty for organizations. He shares his two-decade experience in AI, highlighting how governments are building GPU platforms and protected data hubs to maintain control over AI workloads. Steve emphasizes why self-managed infrastructure is becoming a strategic necessity as companies outgrow cloud costs and require tighter control over models, data, and compliance. The conversation explores the operational substrate for AI sovereignty, including Kubernetes as the scale-out backbone for LLM serving, bridging the gap with PyTorch ecosystems, observability and policy for non-deterministic systems, and emerging security needs such as confidential inference and agentic identity. They also discuss model and hardware optionality (GPUs, CPUs, and new accelerators), the growing demand for energy-efficient inference, and the importance of open models and post-training to create durable differentiation. Steve identifies access to GPUs as the biggest gap hindering sovereign AI adoption today, emphasizing the need for broad access to GPUs for AI workloads to thrive. The conversation also touches on evolving architectures beyond transformers, the interplay between AI and data sovereignty, consolidation pressures from pilot chaos to standardized platforms, and the societal triad of universities, startups, and sovereign infrastructure.
Announcements- Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems
- Unlock the full potential of your AI workloads with a seamless and composable data infrastructure. Bruin is an open source framework that streamlines integration from the command line, allowing you to focus on what matters most - building intelligent systems. Write Python code for your business logic, and let Bruin handle the heavy lifting of data movement, lineage tracking, data quality monitoring, and governance enforcement. With native support for ML/AI workloads, Bruin empowers data teams to deliver faster, more reliable, and scalable AI solutions. Harness Bruin's connectors for hundreds of platforms, including popular machine learning frameworks like TensorFlow and PyTorch. Build end-to-end AI workflows that integrate seamlessly with your existing tech stack. Join the ranks of forward-thinking organizations that are revolutionizing their data engineering with Bruin. Get started today at aiengineeringpodcast.com/bruin, and for dbt Cloud customers, enjoy a $1,000 credit to migrate to Bruin Cloud.
- Your host is Tobias Macey and today I'm interviewing Stephen Watt about how to adapt your existing infrastructure investments to support your AI workloads and gain "AI Sovereignty"
Interview
- Introduction
- How did you get involved in machine learning?
- Can you describe what you mean by the term "AI sovereignty"?
- What are the motivating factors for investing in that as an organizational capability?
- What do you see as the scale, sophistication, regulatory triggers that tip someone from buying off-the-shelf AI services and into operating their own AI stacks?
- There has been substantial investment in MLOps toolchains and patterns over the past decade, along with corresponding evolution of LLMOps techniques. What do you see as the areas of overlap between those technology patterns and the "traditional" infrastructure capabilities that organizations have matured over the past ~20 years?
- What are the aspects that are disjoint and contribute to operational pain for DevOps/platform teams?
- How do AI/agentic workloads strain the ability of existing security and governance frameworks that teams are operating for existing cloud-native workloads?
- What are the options for extending those frameworks and what are the requirements that force a new approach? (e.g. guardrails, LLM interpretability, etc.)
- What are the elements of cloud-native architecture that have left us (as an industry) well situated to absorb the complexity of AI/agentic workloads?
- How does the complexity shift as you go along the continuum of model training to finetuning to inference?
- Beyond the ability to host and execute inference on a model are the various data stores and tool availability that make generative AI a competitive advantage. How much of that (e.g. agentic memory, vector stores, MCP/A2A tools, etc.) are actually net new vs. a new coat of paint on existing techniques?
- What are the most interesting, innovative, or unexpected ways that you have seen teams operationalizing AI workloads on their infrastructure?
- What are the most interesting, unexpected, or challenging lessons that you have learned while working on empowering organizations to achieve AI sovereignty?
- When is operating your own AI infrastructure the wrong choice?
- What are your predictions for the future evolution of operational substrates for AI workloads?
Contact Info
Parting Question
- From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?
Closing Announcements
- Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
- Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
- If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.
- To help other people find the show please leave a review on iTunes and tell your friends and co-workers.
Links
- RedHat
- Bayesian Classifier
- Hadoop
- HBase
- DeepSeek
- Reflection AI
- Nvidia Blackwell
- vLLM
- IBM Spyre
- vLLM CPU
- IBM Watson on Jeopardy
- Neuromorphic Computing
- Kubernetes
- PyTorch Foundation
- MLOps
- LLMOps
- Semantic Router
- BERT
- AGNTCY
- OPA == Open Policy Agent
- CEDAR
- WSDL == Web Services Description Language
- UDDI
- Spark
- llama.cpp
- Ollama
- ARPA-H
The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0Kapitel
Kapitel werden geladen …
From Blind Spots to Observability: Operationalizing LLM Apps with OpenLit
Mit Aman Agarwal
Summary In this episode of the AI Engineering Podcast, Aman Agarwal, creator of OpenLit, discusses the operational foundations required to run LLM-powered applications in production. He highlights common early …
Shownotes
Summary
In this episode of the AI Engineering Podcast, Aman Agarwal, creator of OpenLit, discusses the operational foundations required to run LLM-powered applications in production. He highlights common early blind spots teams face, including opaque model behavior, runaway token costs, and brittle prompt management, emphasizing that strong observability and cost tracking must be established before an MVP ships. Aman explains how OpenLit leverages OpenTelemetry for vendor-neutral tracing across models, tools, and data stores, and introduces features such as prompt and secret management with versioning, evaluation workflows (including LLM-as-a-judge), and fleet management for OpenTelemetry collectors. The conversation covers experimentation patterns, strategies to avoid vendor lock-in, and how detailed stepwise traces reshape system design and debugging. Aman also shares recent advancements like a Kubernetes operator for zero-code instrumentation, multi-database configurations for environment isolation, and integrations with platforms such as Grafana and Dash0. They conclude by discussing lessons learned from building in the open, prioritizing reliability, developer experience, and data security, and preview future work on context management and closing the loop from experimentation to prompt/dataset improvements.
Announcements- Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems
- Unlock the full potential of your AI workloads with a seamless and composable data infrastructure. Bruin is an open source framework that streamlines integration from the command line, allowing you to focus on what matters most - building intelligent systems. Write Python code for your business logic, and let Bruin handle the heavy lifting of data movement, lineage tracking, data quality monitoring, and governance enforcement. With native support for ML/AI workloads, Bruin empowers data teams to deliver faster, more reliable, and scalable AI solutions. Harness Bruin's connectors for hundreds of platforms, including popular machine learning frameworks like TensorFlow and PyTorch. Build end-to-end AI workflows that integrate seamlessly with your existing tech stack. Join the ranks of forward-thinking organizations that are revolutionizing their data engineering with Bruin. Get started today at aiengineeringpodcast.com/bruin, and for dbt Cloud customers, enjoy a $1,000 credit to migrate to Bruin Cloud.
- Your host is Tobias Macey and today I'm interviewing Aman Agarwal about the operational investments that are necessary to ensure you get the most out of your AI models
Interview
- Introduction
- How did you get involved in the area of AI/data management?
- Can you start by giving your assessment of the main blind spots that are common in the existing AI application patterns?
- As teams adopt agentic architectures, how common is it to fall prey to those same blind spots?
- There are numerous tools/services available now focused on various elements of "LLMOps". What are the major components necessary for a minimum viable operational platform for LLMs?
- There are several areas of overlap, as well as disjoint features, in the ecosystem of tools (both open source and commercial). How do you advise teams to navigate the selection process? (point solutions vs. integrated tools, and handling frameworks with only partial overlap)
- Can you describe what OpenLit is and the story behind it?
- How would you characterize the feature set and focus of OpenLit compared to what you view as the "major players"?
- Once you have invested in a platform like OpenLit, how does that change the overall development workflow for the lifecycle of AI/agentic applications?
- What are the most complex/challenging elements of change management for LLM-powered systems? (e.g. prompt tuning, model changes, data changes, etc.)
- How can the information collected in OpenLit be used to develop a self-improvement flywheel for agentic systems?
- Can you describe the architecture and implementation of OpenLit?
- How have the scope and goals of the project changed since you started working on it?
- Given the foundational aspects of the project that you have built, what are some of the adjacent capabilities that OpenLit is situated to expand into?
- What are the sharp edges and blind spots that are still challenging even when you have OpenLit or similar integrated?
- What are the most interesting, innovative, or unexpected ways that you have seen OpenLit used?
- What are the most interesting, unexpected, or challenging lessons that you have learned while working on OpenLit?
- When is OpenLit the wrong choice?
- What do you have planned for the future of OpenLit?
Contact Info
Parting Question
- From your perspective, what is the biggest gap in the tooling or technology for data/AI management today?
Closing Announcements
- Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
- Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
- If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.
- To help other people find the show please leave a review on iTunes and tell your friends and co-workers.
Links
- OpenLit
- Fleet Hub
- OpenTelemetry
- LangFuse
- LangSmith
- TensorZero
- AI Engineering Podcast Episode
- Traceloop
- Helicone
- Clickhouse
The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0Kapitel
Kapitel werden geladen …
Taming Voice Complexity with Dynamic Ensembles at Modulate
Mit Carter Huffman
Summary In this episode of the AI Engineering Podcast, Carter Huffman, co-founder and CTO of Modulate, discusses the engineering behind low-latency, high-accuracy Voice AI. He explains why voice is a uniquely …
Shownotes
Summary
In this episode of the AI Engineering Podcast, Carter Huffman, co-founder and CTO of Modulate, discusses the engineering behind low-latency, high-accuracy Voice AI. He explains why voice is a uniquely challenging modality due to its rich non-textual signals like tone, emotion, and context, and how simple speech-to-text-to-speech pipelines can't capture the necessary nuance. Carter introduces Modulate's Ensemble Listening Model (ELM) architecture, which uses dynamic routing and cost-based optimization to achieve scalability and precision in various audio environments. He covera topics such as reliability under distributed systems constraints, watchdogging with periodic model checks, structured long-horizon memory for conversations, and the trade-offs that make ensemble approaches compelling for repeated tasks at scale. Carter also shares insights on how ELMs generalize beyond voice, draws parallels to database query planners and mixture-of-experts, and discusses strategies for observability and evaluation in complex processing pipelines.
Announcements- Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems
- Unlock the full potential of your AI workloads with a seamless and composable data infrastructure. Bruin is an open source framework that streamlines integration from the command line, allowing you to focus on what matters most - building intelligent systems. Write Python code for your business logic, and let Bruin handle the heavy lifting of data movement, lineage tracking, data quality monitoring, and governance enforcement. With native support for ML/AI workloads, Bruin empowers data teams to deliver faster, more reliable, and scalable AI solutions. Harness Bruin's connectors for hundreds of platforms, including popular machine learning frameworks like TensorFlow and PyTorch. Build end-to-end AI workflows that integrate seamlessly with your existing tech stack. Join the ranks of forward-thinking organizations that are revolutionizing their data engineering with Bruin. Get started today at aiengineeringpodcast.com/bruin, and for dbt Cloud customers, enjoy a $1,000 credit to migrate to Bruin Cloud.
- Your host is Tobias Macey and today I'm interviewing Carter Huffman about his work building an ensemble approach to low latency voice AI
Interview
- Introduction
- How did you get involved in machine learning?
- Can you describe the "Ensemble Listening" approach and the story behind why Modulate moved away from monolithic architectures?
- When designing a real-time voice system, how do you handle the routing logic between specialized models without blowing your latency budget?
- What does the "gatekeeper" or routing layer actually look like in code?
- You’ve mentioned "evals that don’t lie." How do you build a validation pipeline for noisy, adversarial voice data that catches regressions that a simple word-error-rate (WER) might miss?
- In an ensemble of models, a failure in one specialized node might not crash the system, but it can degrade the output quality. How do you monitor for these "silent failures" in real-time without introducing massive overhead?
- For many teams, the default is to call an API for a frontier model. At what point in the scaling or latency curve does it become technically (or economically) necessary to swap a general LLM for a suite of specialized, smaller models?
- How do you track the real-world costs associated with the technical and human overhead of this more complex system?
- What are the most interesting, innovative, or unexpected ways that you have seen orchestrated ensembles used in live conversation environments?
- What are the most interesting, unexpected, or challenging lessons that you have learned while managing the lifecycle of multiple specialized models simultaneously?
- When is an ensemble approach the wrong choice? (e.g., At what level of complexity or throughput is the overhead of orchestration more trouble than it’s worth?)
- What do you have planned for the future of Ensemble Listening Models?
- Are we looking at self-optimizing routers, or perhaps moving these ensembles closer to the edge?
Contact Info
Parting Question
- From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?
Closing Announcements
- Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
- Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
- If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.
- To help other people find the show please leave a review on iTunes and tell your friends and co-workers.
Links
- Modulate
- Nasa Jet Propulsion Laboratory
- OpenAI Whisper
- Multi-Armed Bandit
- Cost-Based Optimizer
- GPT 5
- LLM Attention
- Transformer Architecture
- Mixture of Experts
- Dilated Convolution
- Wavenet
The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0Kapitel
Kapitel werden geladen …
GPU Clouds, Aggregators, and the New Economics of AI Compute
Mit Hugo Shi
Summary In this episode I sit down with Hugo Shi, co-founder and CTO of Saturn Cloud, to map the strategic realities of sourcing and operating GPUs across clouds. Hugo breaks down today’s provider …
Shownotes
Summary
In this episode I sit down with Hugo Shi, co-founder and CTO of Saturn Cloud, to map the strategic realities of sourcing and operating GPUs across clouds. Hugo breaks down today’s provider landscape—from hyperscalers to full-service GPU clouds, bare metal/concierge providers, and emerging GPU aggregators—and how to choose among them based on security posture, managed services, and cost. We explore practical layers of capability (compute, orchestration with Kubernetes/Slurm, storage, networking, and managed services), the trade-offs of portability on “Kubernetes-native” stacks, and the persistent challenge of data gravity. We also discuss current supply dynamics, the growing availability of on-demand capacity as newer chips roll out, and how AMD’s ecosystem is maturing as real competition to NVIDIA. Hugo shares patterns for separating training and inference across providers, why traditional ML is far from dead, and how usage varies wildly across domains like biotech. We close with predictions on consolidation, full‑stack experiences from GPU clouds, financial-style GPU marketplaces, and much-needed advances in reliability for long-running GPU jobs.
Announcements- Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems
- Unlock the full potential of your AI workloads with a seamless and composable data infrastructure. Bruin is an open source framework that streamlines integration from the command line, allowing you to focus on what matters most - building intelligent systems. Write Python code for your business logic, and let Bruin handle the heavy lifting of data movement, lineage tracking, data quality monitoring, and governance enforcement. With native support for ML/AI workloads, Bruin empowers data teams to deliver faster, more reliable, and scalable AI solutions. Harness Bruin's connectors for hundreds of platforms, including popular machine learning frameworks like TensorFlow and PyTorch. Build end-to-end AI workflows that integrate seamlessly with your existing tech stack. Join the ranks of forward-thinking organizations that are revolutionizing their data engineering with Bruin. Get started today at aiengineeringpodcast.com/bruin, and for dbt Cloud customers, enjoy a $1,000 credit to migrate to Bruin Cloud.
- Your host is Tobias Macey and today I'm interviewing Hugo Shi about the strategic realities of sourcing GPUs in the cloud for your training and inference workloads
Interview- Introduction
- How did you get involved in machine learning?
- Can you start by giving a summary of your understanding of the current market for "cloud" GPUs?
- How would you characterize the customer base for the "neocloud" providers?
- How is the access to the GPU compute typically mediated?
- The predominant cloud providers (AWS, GCP, Azure) have gained market share by offering numerous differentiated services and ease-of-use features. What are the types of services that you might expect from a GPU provider?
- The "cloud-native" ecosystem was developed with the promise of enabling workload portability, but the realities are often more complicated. What are some of the difficulties that teams encounter when trying to adapt their workloads to these different cloud providers?
- What are the toolchains/frameworks/architectures that you are seeing as most effective at adapting to these different compute environments?
- One of the major themes in the 2010s that worked against multi-cloud strategies was the idea of "data gravity". What are the strategies that teams are using to mitigate that tax on their workloads?
- That is a more substantial impact when dealing with training workloads than for inference compute. How are you seeing teams think about the balance of cost savings vs. operational complexity for those different workloads?
- What are the most interesting, innovative, or unexpected ways that you have seen teams capitalize on GPU capacity across these new providers?
- What are the most interesting, unexpected, or challenging lessons that you have learned while working on enabling teams to execute workloads on these neoclouds?
- When is a "neocloud" or "GPU cloud" provider the wrong choice?
- What are your predictions for the future evolutions of GPU-as-a-service as hardware availability improves and model architectures become more efficient?
Contact Info
Parting Question- From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?
Closing Announcements- Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
- Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
- If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.
- To help other people find the show please leave a review on iTunes and tell your friends and co-workers.
Links- Saturn Cloud
- Pandas
- NumPy
- MatLab
- AWS
- GCP
- Azure
- Oracle Cloud
- RunPod
- FluidStack
- SFCompute
- KubeFlow
- Lightning AI
- DStack
- Metaflow
- Flyte
- Arya AI
- Dagster
- Coreweave
- Vultr
- Nebius
- Vast.ai
- Weka
- Vast Data
- Slurm
- CNCF == Cloud-Native Computing Foundation
- Kubernetes
- Terraform
- ECS
- Helm Chart
- Block Storage
- Object Storage
- Container Registry
- Crusoe
- Alluxio
- Data Virtualization
- GB300
- H100
- Spot Instance
- AWS Trainium
- Google TPU (Tensor Processing Unit)
- AMD
- ROCM
- PyTorch
- Google Vertex AI
- AWS Bedrock
- CUDA Python
- Mojo
- XGBoost
- Random Forest
- Ludwig - Uber Deep Learning AutoML
- Paperspace
- Voltage Park
- Weights & Biases
The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0Kapitel
Kapitel werden geladen …
The Future of Dev Experience: Spotify’s Playbook for Organization‑Scale AI
Mit Niklas Gustavsson
Summary In this episode of the AI Engineering Podcast Niklas Gustavsson, Chief Architect at Spotify, talks about scaling AI across engineering and product. He explores how Spotify's highly distributed …
Shownotes
Summary
In this episode of the AI Engineering Podcast Niklas Gustavsson, Chief Architect at Spotify, talks about scaling AI across engineering and product. He explores how Spotify's highly distributed architecture was built to support rapid adoption of coding agents like Copilot, Cursor, and Claude Code, enabled by standardization and Backstage. The conversation covers the tension between bottoms-up experimentation and platform standardization, and how Spotify is moving toward monorepos and fleet management. Niklas discusses the emergence of "fleet-wide agents" that can execute complex code changes with robust testing and LLM-as-judge loops to ensure quality. He also touches on the shift in engineering workflows as code generation accelerates, the growing use of agents beyond coding, and the lessons learned in sandboxing, agent skills/rules, and shared evaluation frameworks. Niklas highlights Spotify's decade-long experience with ML product work and shares his vision for deeper end-to-end integration of agentic capabilities across the full product lifecycle and making collaborative "team-level memory" for agents a reality.
Announcements- Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems
- Unlock the full potential of your AI workloads with a seamless and composable data infrastructure. Bruin is an open source framework that streamlines integration from the command line, allowing you to focus on what matters most - building intelligent systems. Write Python code for your business logic, and let Bruin handle the heavy lifting of data movement, lineage tracking, data quality monitoring, and governance enforcement. With native support for ML/AI workloads, Bruin empowers data teams to deliver faster, more reliable, and scalable AI solutions. Harness Bruin's connectors for hundreds of platforms, including popular machine learning frameworks like TensorFlow and PyTorch. Build end-to-end AI workflows that integrate seamlessly with your existing tech stack. Join the ranks of forward-thinking organizations that are revolutionizing their data engineering with Bruin. Get started today at aiengineeringpodcast.com/bruin, and for dbt Cloud customers, enjoy a $1,000 credit to migrate to Bruin Cloud.
- Your host is Tobias Macey and today I'm interviewing Niklas Gustavsson about how Spotify is scaling AI usage in engineering and product work
Interview
- Introduction
- How did you get involved in machine learning?
- Can you start by giving an overview of your engineering practices independent of AI?
- What was your process for introducing AI into the developmer experience? (e.g. pioneers doing early work (bottom-up) vs. top-down)
- There are countless agentic coding tools on the market now. How do you balance organizational standardization vs. exploration?
- Beyond the toolchain, what are your methods for sharing best practices and upskilling engineers on use of agentic toolchains for software/product engineering?
- Spotify has been operationalizing ML/AI features since before the introduction of LLMs and transformer models. How has that history helped inform your adoption of generative AI in your overall engineering organization?
- As you use these generative and agentic AI utilities in your day-to-day, how have those lessons learned fed back into your AI-powered product features?
- What are some of the platform capabilities/developer experience investments that you have made to improve the overall effectiveness of agentic coding in your engineering organization?
- What are some examples of guardrails/speedbumps that you have introduced to avoid injecting unreliable or untested work into production?
- As the (time/money/cognitive) cost of writing code drops that increases the burden on reviewing that code. What are some of the ways that you are working to scale that side of the equation?
- What are some of the ways that agentic coding/CLI utilities have bled into other areas of engineering/opertions/product development beyond just writing code?
- What are the most interesting, innovative, or unexpected ways that you have seen your team applying AI/agentic engineering practices?
- What are the most interesting, unexpected, or challenging lessons that you have learned while working on operationalizing and scaling agentic engineering patterns in your teams?
- When is agentic code generation the wrong choice?
- What do you have planned for the future of AI and agentic coding patterns and practices in your organization?
Contact Info
Parting Question
- From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?
Closing Announcements
- Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
- Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
- If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.
- To help other people find the show please leave a review on iTunes and tell your friends and co-workers.
Links
- Spotify
- Developer Experience
- LLM == Large Language Model
- Transformers
- BackStage
- GitHub Copilot
- Cursor
- Claude Skills
- Monorepo
- MCP == Model Context Protocol
- Claude Code
- Product Manager
- DORA Metrics
- Type Annotations
- BigQuery
- PRD == Product Requirements Document
- AI Evals
- LLM-as-a-Judge
- Agentic Memory
The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0Kapitel
Kapitel werden geladen …
Generative AI Meets Accessibility: Benchmarks, Breakthroughs, and Blind Spots with Joe Devon
Mit Joe Devon
Summary In this episode Joe Devon, co-founder of Global Accessibility Awareness Day (GAAD), talks about how generative AI can both help and harm digital accessibility — and what it will take to tilt the balance …
Shownotes
Summary
In this episode Joe Devon, co-founder of Global Accessibility Awareness Day (GAAD), talks about how generative AI can both help and harm digital accessibility — and what it will take to tilt the balance toward inclusion. Joe shares his personal motivation for the work, real-world stakes for disabled users across web, mobile, and developer tooling, and compelling stories that illustrate why accessible design is a human-rights issue as much as a compliance checkbox. He digs into AI’s current and future roles: from improving caption quality and auto-generating audio descriptions to evaluating how well code-gen models produce accessible UI by default. Joe introduces AIMAC (AI Model Accessibility Checker), a new benchmark comparing top models on accessibility-minded code generation, what the results reveal, and how model providers and engineering teams can practically raise the bar with linters, training data, and cultural change. He closes with concrete guidance for leaders, why involving people with disabilities is non-negotiable, and how solving for edge cases makes AI—and products—better for everyone.
Announcements- Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems
- When ML teams try to run complex workflows through traditional orchestration tools, they hit walls. Cash App discovered this with their fraud detection models - they needed flexible compute, isolated environments, and seamless data exchange between workflows, but their existing tools couldn't deliver. That's why Cash App rely on Prefect. Now their ML workflows run on whatever infrastructure each model needs across Google Cloud, AWS, and Databricks. Custom packages stay isolated. Model outputs flow seamlessly between workflows. Companies like Whoop and 1Password also trust Prefect for their critical workflows. But Prefect didn't stop there. They just launched FastMCP - production-ready infrastructure for AI tools. You get Prefect's orchestration plus instant OAuth, serverless scaling, and blazing-fast Python execution. Deploy your AI tools once, connect to Claude, Cursor, or any MCP client. No more building auth flows or managing servers. Prefect orchestrates your ML pipeline. FastMCP handles your AI tool infrastructure. See what Prefect and Fast MCP can do for your AI workflows at aiengineeringpodcast.com/prefect today.
- Unlock the full potential of your AI workloads with a seamless and composable data infrastructure. Bruin is an open source framework that streamlines integration from the command line, allowing you to focus on what matters most - building intelligent systems. Write Python code for your business logic, and let Bruin handle the heavy lifting of data movement, lineage tracking, data quality monitoring, and governance enforcement. With native support for ML/AI workloads, Bruin empowers data teams to deliver faster, more reliable, and scalable AI solutions. Harness Bruin's connectors for hundreds of platforms, including popular machine learning frameworks like TensorFlow and PyTorch. Build end-to-end AI workflows that integrate seamlessly with your existing tech stack. Join the ranks of forward-thinking organizations that are revolutionizing their data engineering with Bruin. Get started today at aiengineeringpodcast.com/bruin, and for dbt Cloud customers, enjoy a $1,000 credit to migrate to Bruin Cloud.
- Your host is Tobias Macey and today I'm interviewing Joe Devon about opportunities for using generative AI to improve the accessibility of digital technologies
Interview
- Introduction
- How did you get involved in AI?
- Can you starty by giving an overview of what is included in the term "accessibility"?
- What are some of the major contributors to a lack of accessibility in digital experiences today?
- Beyond the web, what are some of the other platforms and interfaces that struggle with accessibility?
- What role does/can generative AI utilities play in improving the accessibility of applications?
- You recently helped create the AI Model Accessibility Checker (AIMAC) to benchmark which coding agents produce the most accessible code. What are the goals of that project and desired outcomes from its introduction?
- What were the key findings from AIMAC's initial benchmarking results? Were there any surprises in terms of which models performed better or worse at generating accessible code?
- The automation offered by using agentic software development toolchains reduces the manual effort involved in building accessible interfaces. What are the opportunities for using generative AI utilities to act as an assistive mechanism for existing sites/technologies?
- Beyond code generation, what other aspects of the AI development lifecycle need accessibility considerations - training data, model outputs, user interfaces for AI tools themselves?
- You co-host the Accessibility and Gen AI Podcast. What are some of the common misconceptions you encounter about AI's role in accessibility, either from the AI community or the accessibility community?
- There's often tension between moving fast with AI adoption and ensuring inclusive design. How do you advise engineering teams to balance innovation speed with accessibility requirements?
- What specific accessibility issues are most amenable to AI solutions today, and which ones still require human judgment and expertise?
- As AI models become more capable at generating code and interfaces, what guardrails or validation processes should engineering teams implement to ensure accessibility standards are met?
- How do you see the role of accessibility specialists evolving as AI tools become more prevalent in the development workflow? Does AI augment their work or change it fundamentally?
- For engineering leaders building platform and data infrastructure, what accessibility considerations should be baked into foundational systems that AI applications will be built upon?
- What are the most interesting, unexpected, or challenging lessons that you have learned while working on acessibility awareness?
Contact Info
Parting Question
- From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?
Closing Announcements
- Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
- Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
- If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.
- To help other people find the show please leave a review on iTunes and tell your friends and co-workers.
Links- AIMAC
- Global Accessibility Awareness Day (GAAD)
- GAAD Foundation
- AltaVista
- Cursor
- Accessibility
- Braille Display
- Ben Ogilvie
- VT-100
- Ghostty
- Warp Terminal
- LLM-as-a-Judge
- FFMPEG
- Aria Tags
- Axe-Core
- MiniMax M1
- Codex Mini
- Qwen
- Kimi
- Google Lighthouse
- GitHub Copilot
- Be-My-Eyes
- WebAIM
- XRAccess
- XR == Extended Reality
- Deque University
- Fable accessibility feedback organization
The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
Kapitel
Kapitel werden geladen …
Beyond the Chatbot: Practical Frameworks for Agentic Capabilities in SaaS
Mit Preeti Shukla
Summary In this episode product and engineering leader Preeti Shukla explores how and when to add agentic capabilities to SaaS platforms. She digs into the operational realities that AI agents must meet inside …
Shownotes
Summary
In this episode product and engineering leader Preeti Shukla explores how and when to add agentic capabilities to SaaS platforms. She digs into the operational realities that AI agents must meet inside multi-tenant software: latency, cost control, data privacy, tenant isolation, RBAC, and auditability. Preeti outlines practical frameworks for selecting models and providers, when to self-host, and how to route capabilities across frontier and cheaper models. She discusses graduated autonomy, starting with internal adoption and low-risk use cases before moving to customer-facing features, and why many successful deployments keep a human-in-the-loop. She also covers evaluation and observability as core engineering disciplines - layered evals, golden datasets, LLM-as-a-judge, path/behavior monitoring, and runtime vs. offline checks - to achieve reliability in nondeterministic systems.
Announcements- Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems
- When ML teams try to run complex workflows through traditional orchestration tools, they hit walls. Cash App discovered this with their fraud detection models - they needed flexible compute, isolated environments, and seamless data exchange between workflows, but their existing tools couldn't deliver. That's why Cash App rely on Prefect. Now their ML workflows run on whatever infrastructure each model needs across Google Cloud, AWS, and Databricks. Custom packages stay isolated. Model outputs flow seamlessly between workflows. Companies like Whoop and 1Password also trust Prefect for their critical workflows. But Prefect didn't stop there. They just launched FastMCP - production-ready infrastructure for AI tools. You get Prefect's orchestration plus instant OAuth, serverless scaling, and blazing-fast Python execution. Deploy your AI tools once, connect to Claude, Cursor, or any MCP client. No more building auth flows or managing servers. Prefect orchestrates your ML pipeline. FastMCP handles your AI tool infrastructure. See what Prefect and Fast MCP can do for your AI workflows at aiengineeringpodcast.com/prefect today.
- Unlock the full potential of your AI workloads with a seamless and composable data infrastructure. Bruin is an open source framework that streamlines integration from the command line, allowing you to focus on what matters most - building intelligent systems. Write Python code for your business logic, and let Bruin handle the heavy lifting of data movement, lineage tracking, data quality monitoring, and governance enforcement. With native support for ML/AI workloads, Bruin empowers data teams to deliver faster, more reliable, and scalable AI solutions. Harness Bruin's connectors for hundreds of platforms, including popular machine learning frameworks like TensorFlow and PyTorch. Build end-to-end AI workflows that integrate seamlessly with your existing tech stack. Join the ranks of forward-thinking organizations that are revolutionizing their data engineering with Bruin. Get started today at aiengineeringpodcast.com/bruin, and for dbt Cloud customers, enjoy a $1,000 credit to migrate to Bruin Cloud.
- Your host is Tobias Macey and today I'm interviewing Preeti Shukla about the process for identifying whether and how to add agentic capabilities to your SaaS
Interview
- Introduction
- How did you get involved in machine learning?
- Can you start by describing how a SaaS context changes the requirements around the business and technical considerations of an AI agent?
- Software-as-a-service is a very broad category that includes everything from simple website builders to complex data platforms. How does the scale and complexity of the service change the equation for ROI potential of agentic elements?
- How does it change the implementation and validation complexity?
- One of the biggest challenges with introducing generative AI and LLMs in a business use case is the unpredictable cost associated with it. What are some of the strategies that you have found effective in estimating, monitoring, and controlling costs to avoid being upside-down on the ROI equation?
- Another challenge of operationalizing an agentic workload is the risk of confident mistakes. What are the tactics that you recommend for building confidence in agent capabilities while mitigating potential harms?
- A corollary to the unpredictability of agent architectures is that they have a large number of variables. What are the evaluation strategies or toolchains that you find most useful to maintain confidence as the system evolves?
- SaaS platforms benefit from unit economics at scale and often rely on multi-tenant architectures. What are the security controls and identity/attribution mechanisms that are critical for allowing agents to operate across tenant boundaries?
- What are the most interesting, innovative, or unexpected ways that you have seen SaaS products adopt agentic patterns?
- What are the most interesting, unexpected, or challenging lessons that you have learned while working on bringing agentic workflows to SaaS products?
- When is an agent the wrong choice?
- What are your predictions for the role of agents in the future of SaaS products?
Contact Info
Parting Question- From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?
Links
- SaaS == Software as a Service
- Multi-Tenancy
- Few-shot Learning
- LLM as a Judge
- RAG == Retrieval Augmented Generation
- MCP == Model Context Protocol
- Loveable
The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0Kapitel
Kapitel werden geladen …
MCP as the API for AI‑Native Systems: Security, Orchestration, and Scale
Mit Craig McLuckie
Summary In this episode Craig McLuckie, co-creator of Kubernetes and founder/CEO of Stacklok, talks about how to improve security and reliability for AI agents using curated, optimized deployments of the Model …
Shownotes
Summary
In this episode Craig McLuckie, co-creator of Kubernetes and founder/CEO of Stacklok, talks about how to improve security and reliability for AI agents using curated, optimized deployments of the Model Context Protocol (MCP). Craig explains why MCP is emerging as the API layer for AI‑native applications, how to balance short‑term productivity with long‑term platform thinking, and why great tools plus frontier models still drive the best outcomes. He digs into common adoption pitfalls (tool pollution, insecure NPX installs, scattered credentials), the necessity of continuous evals for stochastic systems, and the shift from “what the agent can access” to “what the agent knows.” Craig also shares how ToolHive approaches secure runtimes, a virtual MCP gateway with semantic search, orchestration and transactional semantics, a registry for organizational tooling, and a console for self‑service—along with pragmatic patterns for auth, policy, and observability.
Announcements- Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems
- When ML teams try to run complex workflows through traditional orchestration tools, they hit walls. Cash App discovered this with their fraud detection models - they needed flexible compute, isolated environments, and seamless data exchange between workflows, but their existing tools couldn't deliver. That's why Cash App rely on Prefect. Now their ML workflows run on whatever infrastructure each model needs across Google Cloud, AWS, and Databricks. Custom packages stay isolated. Model outputs flow seamlessly between workflows. Companies like Whoop and 1Password also trust Prefect for their critical workflows. But Prefect didn't stop there. They just launched FastMCP - production-ready infrastructure for AI tools. You get Prefect's orchestration plus instant OAuth, serverless scaling, and blazing-fast Python execution. Deploy your AI tools once, connect to Claude, Cursor, or any MCP client. No more building auth flows or managing servers. Prefect orchestrates your ML pipeline. FastMCP handles your AI tool infrastructure. See what Prefect and Fast MCP can do for your AI workflows at aiengineeringpodcast.com/prefect today.
- Unlock the full potential of your AI workloads with a seamless and composable data infrastructure. Bruin is an open source framework that streamlines integration from the command line, allowing you to focus on what matters most - building intelligent systems. Write Python code for your business logic, and let Bruin handle the heavy lifting of data movement, lineage tracking, data quality monitoring, and governance enforcement. With native support for ML/AI workloads, Bruin empowers data teams to deliver faster, more reliable, and scalable AI solutions. Harness Bruin's connectors for hundreds of platforms, including popular machine learning frameworks like TensorFlow and PyTorch. Build end-to-end AI workflows that integrate seamlessly with your existing tech stack. Join the ranks of forward-thinking organizations that are revolutionizing their data engineering with Bruin. Get started today at aiengineeringpodcast.com/bruin, and for dbt Cloud customers, enjoy a $1,000 credit to migrate to Bruin Cloud.
- Your host is Tobias Macey and today I'm interviewing Craig McLuckie about improving the security of your AI agents through curated and optimized MCP deployment
- Introduction
- How did you get involved in machine learning?
- MCP saw huge growth in attention and adoption over the course of this year. What are the stumbling blocks that teams run into when going to production with MCP servers?
- How do improperly managed MCP servers contribute to security problems in an agent-driven software development workflow?
- What are some of the problematic practices or shortcuts that you are seeing teams implement when running MCP services for their developers?
- What are the benefits of a curated and opinionated MCP service as shared infrastructure for an engineering team?
- You are building ToolHive as a system for managing and securing MCP services as a platform component. What are the strategic benefits of starting with that as the foundation for your company?
- There are several services for managing MCP server deployment and access control. What are the unique elements of ToolHive that make it worth adopting?
- For software-focused agentic AI, the approach of Claude Code etc. to be command-line based opens the door for an effectively unbounded set of tools. What are the benefits of MCP over arbitrary CLI execution in that context?
- What are the most interesting, innovative, or unexpected ways that you have seen ToolHive/MCP used?
- What are the most interesting, unexpected, or challenging lessons that you have learned while working on ToolHive?
- When is ToolHive the wrong choice?
- What do you have planned for the future of ToolHive/Stacklok?
Contact Info
Parting Question- From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?
- StackLok
- MCP == Model Context Protocol
- Kubernetes
- CNCF == Cloud Native Computing Foundation
- SDLC == Software Development Life Cycle
- The Bitter Lesson
- TLA+
- Jepsen Tests
- ToolHive
- API Gateway
- Glean
The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
Kapitel
Kapitel werden geladen …
Context as Code, DevX as Leverage: Accelerating Software with Multi‑Agent Workflows
Mit Maxime Beauchemin
Summary In this episode Max Beauchemin explores how multiplayer, multi‑agent engineering is reshaping individual and team velocity for building data and AI systems. Max shares his journey from Airflow and …
Shownotes
Summary
In this episode Max Beauchemin explores how multiplayer, multi‑agent engineering is reshaping individual and team velocity for building data and AI systems. Max shares his journey from Airflow and Superset to going all‑in on AI coding agents, describing a pragmatic “AI‑first reflex” for nearly every task and the emerging role of humans as orchestrators of agents. He digs into shifting bottlenecks — code review, QA, async coordination — and how better DevX/AIX, just‑in‑time context via tools, and structured "context as code" can keep pace with agent‑accelerated execution. He then dives deep into Agor, a new open‑source agent‑orchestration platform: a spatial, multiplayer canvas that manages git worktrees and shared dev environments, enables templated prompts and zone‑based workflows, and exposes an internal MCP so agents can operate the system — and each other. Max discusses session forking, sub‑session trees, scheduling, and safety considerations, and how these capabilities enable parallelization, handoffs across roles, and richer visibility into prompting and cost/usage—pointing to a near future where software engineering centers on orchestrating teams of agents and collaborators. Resources: agor.live (docs, one‑click Codespaces, npm install), Apache Superset, and related MCP/CLI tooling referenced for agent workflows.
Announcements- Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems
- When ML teams try to run complex workflows through traditional orchestration tools, they hit walls. Cash App discovered this with their fraud detection models - they needed flexible compute, isolated environments, and seamless data exchange between workflows, but their existing tools couldn't deliver. That's why Cash App rely on Prefect. Now their ML workflows run on whatever infrastructure each model needs across Google Cloud, AWS, and Databricks. Custom packages stay isolated. Model outputs flow seamlessly between workflows. Companies like Whoop and 1Password also trust Prefect for their critical workflows. But Prefect didn't stop there. They just launched FastMCP - production-ready infrastructure for AI tools. You get Prefect's orchestration plus instant OAuth, serverless scaling, and blazing-fast Python execution. Deploy your AI tools once, connect to Claude, Cursor, or any MCP client. No more building auth flows or managing servers. Prefect orchestrates your ML pipeline. FastMCP handles your AI tool infrastructure. See what Prefect and Fast MCP can do for your AI workflows at aiengineeringpodcast.com/prefect today.
- Unlock the full potential of your AI workloads with a seamless and composable data infrastructure. Bruin is an open source framework that streamlines integration from the command line, allowing you to focus on what matters most - building intelligent systems. Write Python code for your business logic, and let Bruin handle the heavy lifting of data movement, lineage tracking, data quality monitoring, and governance enforcement. With native support for ML/AI workloads, Bruin empowers data teams to deliver faster, more reliable, and scalable AI solutions. Harness Bruin's connectors for hundreds of platforms, including popular machine learning frameworks like TensorFlow and PyTorch. Build end-to-end AI workflows that integrate seamlessly with your existing tech stack. Join the ranks of forward-thinking organizations that are revolutionizing their data engineering with Bruin. Get started today at aiengineeringpodcast.com/bruin, and for dbt Cloud customers, enjoy a $1,000 credit to migrate to Bruin Cloud.
- Your host is Tobias Macey and today I'm interviewing Maxime Beauchemin about the impact of multi-player multi-agent engineering on individual and team velocity for building better data systems
- Introduction
- How did you get involved in the area of data management?
- Can you start by giving an overview of the types of work that you are relying on AI development agents for?
- As you bring agents into the mix for software engineering, what are the bottlenecks that start to show up?
- In my own experience there are a finite number of agents that I can manage in parallel. How does Agor help to increase that limit?
- How does making multi-agent management a multi-player experience change the dynamics of how you apply agentic engineering workflows?
Contact Info
Links- Agor
- Apache Airflow
- Apache Superset
- Preset
- Claude Code
- Codex
- Playwright MCP
- Tmux
- Git Worktrees
- Opencode.ai
- GitHub Codespaces
- Ona
The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
Kapitel
Kapitel werden geladen …
Inside the Black Box: Neuron-Level Control and Safer LLMs
Mit Vinay Kumar
Summary In this episode of the AI Engineering Podcast Vinay Kumar, founder and CEO of Arya.ai and head of Lexsi Labs, talks about practical strategies for understanding and steering AI systems. He discusses the …
Shownotes
Summary
In this episode of the AI Engineering Podcast Vinay Kumar, founder and CEO of Arya.ai and head of Lexsi Labs, talks about practical strategies for understanding and steering AI systems. He discusses the differences between interpretability and explainability, and why post-hoc methods can be misleading. Vinay shares his approach to tracing relevance through deep networks and LLMs using DL Backtrace, and how interpretability is evolving from an audit tool into a lever for alignment, enabling targeted pruning, fine-tuning, unlearning, and model compression. The conversation covers setting concrete alignment metrics, the gaps in current enterprise practices for complex models, and tailoring explainability artifacts for different stakeholders. Vinay also previews his team's "AlignTune" effort for neuron-level model editing and discusses emerging trends in AI risk, multi-modal complexity, and automated safety agents. He explores when and why teams should invest in interpretability and alignment, how to operationalize findings without overcomplicating evaluation, and the best practices for private, safer LLM endpoints in enterprises, aiming to make advanced AI not just accurate but also acceptable, auditable, and scalable.
Announcements- Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems
- When ML teams try to run complex workflows through traditional orchestration tools, they hit walls. Cash App discovered this with their fraud detection models - they needed flexible compute, isolated environments, and seamless data exchange between workflows, but their existing tools couldn't deliver. That's why Cash App rely on Prefect. Now their ML workflows run on whatever infrastructure each model needs across Google Cloud, AWS, and Databricks. Custom packages stay isolated. Model outputs flow seamlessly between workflows. Companies like Whoop and 1Password also trust Prefect for their critical workflows. But Prefect didn't stop there. They just launched FastMCP - production-ready infrastructure for AI tools. You get Prefect's orchestration plus instant OAuth, serverless scaling, and blazing-fast Python execution. Deploy your AI tools once, connect to Claude, Cursor, or any MCP client. No more building auth flows or managing servers. Prefect orchestrates your ML pipeline. FastMCP handles your AI tool infrastructure. See what Prefect and Fast MCP can do for your AI workflows at aiengineeringpodcast.com/prefect today.
- Unlock the full potential of your AI workloads with a seamless and composable data infrastructure. Bruin is an open source framework that streamlines integration from the command line, allowing you to focus on what matters most - building intelligent systems. Write Python code for your business logic, and let Bruin handle the heavy lifting of data movement, lineage tracking, data quality monitoring, and governance enforcement. With native support for ML/AI workloads, Bruin empowers data teams to deliver faster, more reliable, and scalable AI solutions. Harness Bruin's connectors for hundreds of platforms, including popular machine learning frameworks like TensorFlow and PyTorch. Build end-to-end AI workflows that integrate seamlessly with your existing tech stack. Join the ranks of forward-thinking organizations that are revolutionizing their data engineering with Bruin. Get started today at aiengineeringpodcast.com/bruin, and for dbt Cloud customers, enjoy a $1,000 credit to migrate to Bruin Cloud.
- Your host is Tobias Macey and today I'm interviewing Vinay Kumar about strategies and tactics for gaining insights into the decisions of your AI systems
Interview
- Introduction
- How did you get involved in machine learning?
- Can you start by giving a quick overview of what explainability means in the context of ML/AI?
- What are the predominant methods used to gain insight into the internal workings of ML/AI models?
- How does the size and modality of a model influence the technique and evaluation of methods used?
- What are the contexts in which a team would incorporate explainability into their workflow?
- How might explainability be used in a live system to provide guardrails or efficiency/accuracy improvements?
- What are the aspects of model alignment and explainability that are most challenging to implement?
- What are the supporting systems that are necessary to be able to effectively operationalize the collection and analysis of model reliability and alignment?
- "Trust", "Reliability", and "Alignment" are all words that seem obvious until you try to define them concretely. What are the ways that teams work through the creation of metrics and evaluation suites to gauge compliance with those goals?
- What are the most interesting, innovative, or unexpected ways that you have seen explainability methods used in AI systems?
- What are the most interesting, unexpected, or challenging lessons that you have learned while working on explainability/reliability at AryaXAI?
- When is evaluation of explainability overkill?
- What do you have planned for the future of AryaXAI and explainable AI?
Contact Info
Parting Question
- From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?
Closing Announcements
- Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
- Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
- If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.
- To help other people find the show please leave a review on iTunes and tell your friends and co-workers.
Links
- Lexsi Labs
- Ayra.ai
- Deep Learning
- AlexNet
- DL Backtrace
- Gradient Boost
- SAE == Sparse AutoEncoder
- Shapley Values
- LRP == Layerwise Relevance Propagation
- IG == Integrated Gradients
- Circuit Discovery
- F1 Score
- LLM As A Judge
The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
Kapitel
Kapitel werden geladen …
Building the Internet of Agents: Identity, Observability, and Open Protocols
Mit Guillaume de Saint Marc
SummaryIn this episode Guillaume de Saint Marc, VP of Engineering at Cisco Outshift, talks about the complexities and opportunities of scaling multi‑agent systems. Guillaume explains why specialized agents …
Shownotes
Summary
In this episode Guillaume de Saint Marc, VP of Engineering at Cisco Outshift, talks about the complexities and opportunities of scaling multi‑agent systems. Guillaume explains why specialized agents collaborating as a team inspire trust in enterprise settings, and contrasts rigid, “lift-and-shift” agentic workflows with fully self-forming systems. We explore the emerging Internet of Agents, the need for open, interoperable protocols (A2A for peer collaboration and MCP for tool calling), and new layers in the stack for syntactic and semantic communication. Guillaume details foundational needs around discovery, identity, observability, and fine-grained, task/tool/transaction-based access control (TBAC), along with Cisco’s open-source Agency initiative, directory concepts, and OpenTelemetry extensions for agent traces. He shares concrete wins in IT/NetOps—network config validation, root-cause analysis, and the CAPE platform engineer agent—showing dramatic productivity gains. We close with human-in-the-loop UX patterns for multi-agent teams and SLIM, a high-performance group communication layer designed for agent collaboration.
Announcements- Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems
- When ML teams try to run complex workflows through traditional orchestration tools, they hit walls. Cash App discovered this with their fraud detection models - they needed flexible compute, isolated environments, and seamless data exchange between workflows, but their existing tools couldn't deliver. That's why Cash App rely on Prefect. Now their ML workflows run on whatever infrastructure each model needs across Google Cloud, AWS, and Databricks. Custom packages stay isolated. Model outputs flow seamlessly between workflows. Companies like Whoop and 1Password also trust Prefect for their critical workflows. But Prefect didn't stop there. They just launched FastMCP - production-ready infrastructure for AI tools. You get Prefect's orchestration plus instant OAuth, serverless scaling, and blazing-fast Python execution. Deploy your AI tools once, connect to Claude, Cursor, or any MCP client. No more building auth flows or managing servers. Prefect orchestrates your ML pipeline. FastMCP handles your AI tool infrastructure. See what Prefect and Fast MCP can do for your AI workflows at aiengineeringpodcast.com/prefect today.
- Unlock the full potential of your AI workloads with a seamless and composable data infrastructure. Bruin is an open source framework that streamlines integration from the command line, allowing you to focus on what matters most - building intelligent systems. Write Python code for your business logic, and let Bruin handle the heavy lifting of data movement, lineage tracking, data quality monitoring, and governance enforcement. With native support for ML/AI workloads, Bruin empowers data teams to deliver faster, more reliable, and scalable AI solutions. Harness Bruin's connectors for hundreds of platforms, including popular machine learning frameworks like TensorFlow and PyTorch. Build end-to-end AI workflows that integrate seamlessly with your existing tech stack. Join the ranks of forward-thinking organizations that are revolutionizing their data engineering with Bruin. Get started today at aiengineeringpodcast.com/bruin, and for dbt Cloud customers, enjoy a $1,000 credit to migrate to Bruin Cloud.
- Your host is Tobias Macey and today I'm interviewing Guillaume de Saint Marc about the complexities and opportunities of scaling multi-agent systems
- Introduction
- How did you get involved in machine learning?
- Can you start by giving an overview of what constitutes a "multi-agent" system?
- Many of the multi-agent services that I have read or spoken about are designed and operated by a single department or organization. What are some of the new challenges that arise when allowing agents to communicate and co-ordinate outside of organizational boundaries?
- The web is the most famous example of a successful decentralized system, with HTTP being the most ubiquitous protocol powering it. What does the internet of agents look like?
- What is the role of humans in that equation?
- The web has evolved in a combination of organic and planned growth and is vastly more complex and complicated than when it was first introduced. What are some of the most important lessons that we should carry forward into the connectivity of AI agents?
- Security is a critical aspect of the modern web. What are the controls, assertions, and constraints that we need to implement to enable agents to operate with a degree of trust while also being appropriately constrained?
- The AGNTCY project is a substantial investment in an open architecture for the internet of agents. What does it provide in terms of building blocks for teams and businesses who are investing in agentic services?
- What are the most interesting, innovative, or unexpected ways that you have seen AGNTCY/multi-agent systems used?
- What are the most interesting, unexpected, or challenging lessons that you have learned while working on multi-agent systems?
- When is a multi-agent system the wrong choice?
- What do you have planned for the future of AGNTCY/multi-agent systems?
Parting Question- From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?
- Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
- Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
- If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.
- To help other people find the show please leave a review on iTunes and tell your friends and co-workers.
- Outshift by Cisco
- Multi-Agent Systems
- Deep Learning
- Meraki
- Symbolic Reasoning
- Transformer Architecture
- DeepSeek
- LLM Reasoning
- René Descartes
- Kanban
- A2A (Agent-to-Agent) Protocol
- MCP == Model Context Protocol
- AGNTCY
- ICANN == Internet Corporation for Assigned Names and Numbers
- OSI Layers
- OCI == Open Container Initiative
- OASF == Open Agentic Schema Framework
- Oracle AgentSpec
- Splunk
- OpenTelemetry
- CAIPE == Community AI Platform Engineer
- AGNTCY Coffee Shop
Kapitel
Kapitel werden geladen …
Agents, IDEs, and the Blast Radius: Practical AI for Software Engineers
Mit Will Vincent
SummaryIn this episode of the AI Engineering Podcast Will Vincent, Python developer advocate at JetBrains (PyCharm), talks about how AI utilities are revolutionizing software engineering beyond basic code …
Shownotes
Summary
In this episode of the AI Engineering Podcast Will Vincent, Python developer advocate at JetBrains (PyCharm), talks about how AI utilities are revolutionizing software engineering beyond basic code completion. He discusses the shift from "vibe coding" to "vibe engineering," where engineers collaborate with AI agents through clear guidelines, iterative specs, and tight guardrails. Will shares practical techniques for getting real value from these tools, including loading the whole codebase for context, creating agent specifications, constraining blast radius, and favoring step-by-step plans over one-shot generations. The conversation covers code review gaps, deployment context, and why continuity across tools matters, as well as JetBrains' evolving approach to integrated AI, including support for external and local models. Will emphasizes the importance of human oversight, particularly for architectural choices and production changes, and encourages experimentation and playfulness while acknowledging the ethics, security, and reliability tradeoffs that come with modern LLMs.
Announcements- Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems
- When ML teams try to run complex workflows through traditional orchestration tools, they hit walls. Cash App discovered this with their fraud detection models - they needed flexible compute, isolated environments, and seamless data exchange between workflows, but their existing tools couldn't deliver. That's why Cash App rely on Prefect. Now their ML workflows run on whatever infrastructure each model needs across Google Cloud, AWS, and Databricks. Custom packages stay isolated. Model outputs flow seamlessly between workflows. Companies like Whoop and 1Password also trust Prefect for their critical workflows. But Prefect didn't stop there. They just launched FastMCP - production-ready infrastructure for AI tools. You get Prefect's orchestration plus instant OAuth, serverless scaling, and blazing-fast Python execution. Deploy your AI tools once, connect to Claude, Cursor, or any MCP client. No more building auth flows or managing servers. Prefect orchestrates your ML pipeline. FastMCP handles your AI tool infrastructure. See what Prefect and Fast MCP can do for your AI workflows at aiengineeringpodcast.com/prefect today.
- Unlock the full potential of your AI workloads with a seamless and composable data infrastructure. Bruin is an open source framework that streamlines integration from the command line, allowing you to focus on what matters most - building intelligent systems. Write Python code for your business logic, and let Bruin handle the heavy lifting of data movement, lineage tracking, data quality monitoring, and governance enforcement. With native support for ML/AI workloads, Bruin empowers data teams to deliver faster, more reliable, and scalable AI solutions. Harness Bruin's connectors for hundreds of platforms, including popular machine learning frameworks like TensorFlow and PyTorch. Build end-to-end AI workflows that integrate seamlessly with your existing tech stack. Join the ranks of forward-thinking organizations that are revolutionizing their data engineering with Bruin. Get started today at aiengineeringpodcast.com/bruin, and for dbt Cloud customers, enjoy a $1,000 credit to migrate to Bruin Cloud.
- Your host is Tobias Macey and today I'm interviewing Will Vincent about selecting and using AI software engineering utilities and making them work for your team
- Introduction
- How did you get involved in machine learning?
- Software engineering is a discipline that is relatively young in relative terms, but does have several decades of history. As someone working for a developer tools company, what is your broad opinion on the impact of AI on software engineering as an occupation?
- There are many permutations of AI development tools. What are the broad categories that you see?
- What are the major areas of overlap?
- What are the styles of coding agents that you are seeing the broadest adoption for?
- What are your thoughts on the role of editors/IDEs in an AI-driven development workflow?
- Many of the code generation utilities are executed on a developer's computer in a single-player mode. What are some strategies that you have seen or experimented with to extract and share techniques/best practices/prompt templates at the team level?
- While there are many AI-powered services that hook into various stages of the software development and delivery lifecycle, what are the areas where you are seeing gaps in the user experience?
- What are the most interesting, innovative, or unexpected ways that you have seen AI used in the context of software engineering workflows?
- What are the most interesting, unexpected, or challenging lessons that you have learned while working on developer tooling in the age of AI?
- When is AI-powered the wrong choice?
- What do you have planned for the future of AI in the context of Jetbrains?
- What are your predictions/hopes for the future of AI for software engineering?
Parting Question- From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?
- Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
- Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
- If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.
- To help other people find the show please leave a review on iTunes and tell your friends and co-workers.
- JetBrains
- Simon Willison
- Vibe Engineering Post
- GitHub Copilot
- AGENTS.md
- Kiro IDE
- Claude Code
- Jetbrains QuickEdit
- Claude Agent in JetBrains IDEs
- Ruff linter
- uv package manager
- ty type checker
- pyrefly
- IDE == Integrated Development Environment
- Ollama
- LM Studio
- Google Gemma
- Deepseek
- gpt-oss
- Ollama Cloud
- Gemini Diffusion
- Django Annual Survey
- Co-Intelligence by Ethan Mollick (affiliate link)
Kapitel
Kapitel werden geladen …
From MRI to World Models: How AI Is Changing What We See
Mit Tobias Macey und Daniel Sodickson
SummaryIn this episode of the AI Engineering Podcast Daniel Sodickson, Chief of Innovation in Radiology at NYU Grossman School of Medicine, talks about harnessing AI systems to truly understand images and …
Shownotes
Summary
In this episode of the AI Engineering Podcast Daniel Sodickson, Chief of Innovation in Radiology at NYU Grossman School of Medicine, talks about harnessing AI systems to truly understand images and revolutionize science and healthcare. Dan shares his journey from linear reconstruction to early deep learning for accelerated MRI, highlighting the importance of domain expertise when adapting models to specialized modalities. He explores "upstream" AI that changes what and how we measure, using physics-guided networks, prior knowledge, and personal baselines to enable faster, cheaper, and more accessible imaging. The conversation covers multimodal world models, cross-disciplinary translation, explainability, and a future where agents flag abnormalities while humans apply judgment, as well as provocative frontiers like "imaging without images," continuous health monitoring, and decoding brain activity. Dan stresses the need to preserve truth, context, and human oversight in AI-driven imaging, and calls for tools that distill core methodologies across disciplines to accelerate understanding and progress.
Announcements- Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems
- When ML teams try to run complex workflows through traditional orchestration tools, they hit walls. Cash App discovered this with their fraud detection models - they needed flexible compute, isolated environments, and seamless data exchange between workflows, but their existing tools couldn't deliver. That's why Cash App rely on Prefect. Now their ML workflows run on whatever infrastructure each model needs across Google Cloud, AWS, and Databricks. Custom packages stay isolated. Model outputs flow seamlessly between workflows. Companies like Whoop and 1Password also trust Prefect for their critical workflows. But Prefect didn't stop there. They just launched FastMCP - production-ready infrastructure for AI tools. You get Prefect's orchestration plus instant OAuth, serverless scaling, and blazing-fast Python execution. Deploy your AI tools once, connect to Claude, Cursor, or any MCP client. No more building auth flows or managing servers. Prefect orchestrates your ML pipeline. FastMCP handles your AI tool infrastructure. See what Prefect and Fast MCP can do for your AI workflows at aiengineeringpodcast.com/prefect today.
- Unlock the full potential of your AI workloads with a seamless and composable data infrastructure. Bruin is an open source framework that streamlines integration from the command line, allowing you to focus on what matters most - building intelligent systems. Write Python code for your business logic, and let Bruin handle the heavy lifting of data movement, lineage tracking, data quality monitoring, and governance enforcement. With native support for ML/AI workloads, Bruin empowers data teams to deliver faster, more reliable, and scalable AI solutions. Harness Bruin's connectors for hundreds of platforms, including popular machine learning frameworks like TensorFlow and PyTorch. Build end-to-end AI workflows that integrate seamlessly with your existing tech stack. Join the ranks of forward-thinking organizations that are revolutionizing their data engineering with Bruin. Get started today at aiengineeringpodcast.com/bruin, and for dbt Cloud customers, enjoy a $1,000 credit to migrate to Bruin Cloud.
- Your host is Tobias Macey and today I'm interviewing Daniel Sodickson about the impact and applications of AI that is capable of image understanding
- Introduction
- How did you get involved in machine learning?
- Images and vision are concepts that we understand intuitively, but which have a large potential semantic range. How would you characterize the scope and application of imagery in the context of AI and other autonomous technologies?
- Can you give an overview of the current state of image/vision capabilities in AI systems?
- A predominant application of machine vision has been for object recognition/tracking. How are advances in AI changing the range of problems that can be solved with computer vision systems?
- A substantial amount of work has been done on processing of images such as the digital pictures taken by smartphones. As you move to other types of image data, particularly in non-visible light ranges, what are the areas of similarity and in what ways do we need to develop new processing/analysis techniques?
- What are some of the ways that AI systems will change the ways that we conceive of
- What are the most interesting, innovative, or unexpected ways that you have seen AI vision used?
- What are the most interesting, unexpected, or challenging lessons that you have learned while working on imaging technologies and techniques?
- When is AI the wrong choice for vision/imaging applications?
- What are your predictions for the future of AI image understanding?
Parting Question- From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?
- Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
- Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
- If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.
- To help other people find the show please leave a review on iTunes and tell your friends and co-workers.
- MRI == Magnetic Resonance Imaging
- Linear Algorithm
- Non-Linear Algorithm
- Compressed Sensing
- Dictionary Learning Algorithm
- Deep Learning
- CT Scan
- Cambrian Explosion
- LIDAR Point Cloud
- Synthetic Aperture Radar
- Geoffrey Hinton
- Co-Intelligence by Ethan Mollick (affiliate link)
- Tomography
- X-Ray Crystallography
- CERN
- CLIP Model
- Physics-Guided Neural Network
- Functional MRI
- A Path Toward Autonomous Machine Intelligence by Yann LeCun
Kapitel
Kapitel werden geladen …
Specs, Tests, and Self‑Verification: The Playbook for Agentic Engineering Teams
Mit Tobias Macey und Andrew Filev
SummaryIn this episode Andrew Filev, CEO and founder of ZenCoder, takes a deep dive into the system design, workflows, and organizational changes behind building agentic coding systems. He traces the evolution …
Shownotes
Summary
In this episode Andrew Filev, CEO and founder of ZenCoder, takes a deep dive into the system design, workflows, and organizational changes behind building agentic coding systems. He traces the evolution from autocomplete to truly agentic models, discusses why context engineering and verification are the real unlocks for reliability, and outlines a pragmatic path from “vibe coding” to AI‑first engineering. Andrew shares ZenCoder’s internal playbook: PRD and tech spec co‑creation with AI, human‑in‑the‑loop gates, test‑driven development, and emerging BDD-style acceptance testing. He explores multi-repo context, cross-service reasoning, and how AI reshapes team communication, ownership, and architecture decisions. He also covers cost strategies, when to choose agents vs. manual edits, and why self‑verification and collaborative agent UX will define the next wave. Andrew offers candid lessons from building ZenCoder—why speed of iteration beats optimizing for weak models, how ignoring the emotional impact of vibe coding slowed brand momentum, and where agentic tools fit across greenfield and legacy systems. He closes with predictions for the next year: self‑verification, parallelized agent workflows, background execution in CI, and collaborative spec‑driven development moving code review upstream.
Announcements- Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems
- When ML teams try to run complex workflows through traditional orchestration tools, they hit walls. Cash App discovered this with their fraud detection models - they needed flexible compute, isolated environments, and seamless data exchange between workflows, but their existing tools couldn't deliver. That's why Cash App rely on Prefect. Now their ML workflows run on whatever infrastructure each model needs across Google Cloud, AWS, and Databricks. Custom packages stay isolated. Model outputs flow seamlessly between workflows. Companies like Whoop and 1Password also trust Prefect for their critical workflows. But Prefect didn't stop there. They just launched FastMCP - production-ready infrastructure for AI tools. You get Prefect's orchestration plus instant OAuth, serverless scaling, and blazing-fast Python execution. Deploy your AI tools once, connect to Claude, Cursor, or any MCP client. No more building auth flows or managing servers. Prefect orchestrates your ML pipeline. FastMCP handles your AI tool infrastructure. See what Prefect and Fast MCP can do for your AI workflows at aiengineeringpodcast.com/prefect today.
- Your host is Tobias Macey and today I'm interviewing Andrew Filev about the system design and integration strategies behind building coding agents at Zencoder
- Introduction
- How did you get involved in ML/AI?
- There have been several iterations of applications for generative AI models in the context of software engineering. How would you characterize the different approaches or categories?
- Over the course of this summer (2025) the term "vibe coding" gained prominence with the idea that the human just needs to be worried about whether the software does what you ask, not how it is written. How does that sentiment compare to your philosophies on the role of agentic AI in the lifecycle of software?
- This points at a broader challenge for software engineers in the AI era; how much control can and should we cede to the LLMs, and over what elements of the software process?
- This also brings up useful questions around the experience of the engineer collaborating with the agent. What are the different interaction patterns that individuals and teams should be thinking of in their use of AI engineering tools?
- Should the agent be proactive? reactive? what are the triggers for an action to be taken and to what extent?
- What differentiates a coding agent from an agentic editor?
- The key challenge in any agent system is context engineering. Software is inherently structured and provides strong feedback loops. But it can also be very messy or difficult to encapsulate in a single context window. What are some of the data structures/indexing strategies/retrieval methods that are most useful when providing guidance to an agent?
- Software projects are rarely fully self-contained, and often need to cross repository boundaries, as well as manage dependencies. What are some of the more challenging aspects of identifying and accounting for those sometimes implicit relationships?
- What are some of the strategies that are most effective for yielding productive results from an agent in terms of prompting and scoping of the problem?
- What are some of the heuristics that you use to determine whether and how to employ an agent for a given task vs. doing it manually?
- How can the agents assist in the decomposition and planning of complex projects?
- What are some of the ways that single-player interaction strategies can be turned into team/multi-player strategies?
- What are some of the ways that teams can create and curate productive patterns to accelerate everyone equally?
- What are the most interesting, innovative, or unexpected ways that you have seen coding agents used?
- What are the most interesting, unexpected, or challenging lessons that you have learned while working on coding agents at Zencoder?
- When is/are Zencoder/coding agents the wrong choice?
- What do you have planned for the future of Zencoder/agentic software engineering?
Parting Question- From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?
- Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
- Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
- If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.
- To help other people find the show please leave a review on iTunes and tell your friends and co-workers.
- Zencoder
- Wrike
- DARPA Robotics Challenge
- Cognitive Computing
- Andrew Ng
- Sebastian Thrun
- Github Copilot
- RAG == Retrieval Augmented Generation
- Re-ranking
- Claude Sonnet 3.5
- SWE-Bench
- Vibe Coding
- AI First Engineering
- Waterfall Software Engineering
- Agile Software Engineering
- PRD == Project Requirements Document
- BDD == Behavior-Driven Development
- VSCode
Kapitel
Kapitel werden geladen …
From Probabilistic to Trustworthy: Building Orion, an Agentic Analytics Platform
Mit Tobias Macey und Drew Gilson
SummaryIn this episode of the AI Engineering Podcast Lucas Thelosen and Drew Gillson talk about Orion, their agentic analytics platform that delivers proactive, push-based insights to business users through …
Shownotes
Summary
In this episode of the AI Engineering Podcast Lucas Thelosen and Drew Gillson talk about Orion, their agentic analytics platform that delivers proactive, push-based insights to business users through asynchronous thinking with rich organizational context. Lucas and Drew share their approach to building trustworthy analysis by grounding in semantic layers, fact tables, and quality-assurance loops, as well as their focus on accuracy through parallel test-time compute and evolving from probabilistic steps to deterministic tools. They discuss the importance of context engineering, multi-agent orchestration, and security boundaries for enterprise deployments, and share lessons learned on consistency, tool design, user change management, and the emerging role of "AI manager" as a career path. The conversation highlights the future of AI knowledge workers collaborating across organizations and tools while simplifying UIs and raising the bar on actionable, trustworthy analytics.
Announcements- Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems
- When ML teams try to run complex workflows through traditional orchestration tools, they hit walls. Cash App discovered this with their fraud detection models - they needed flexible compute, isolated environments, and seamless data exchange between workflows, but their existing tools couldn't deliver. That's why Cash App rely on Prefect. Now their ML workflows run on whatever infrastructure each model needs across Google Cloud, AWS, and Databricks. Custom packages stay isolated. Model outputs flow seamlessly between workflows. Companies like Whoop and 1Password also trust Prefect for their critical workflows. But Prefect didn't stop there. They just launched FastMCP - production-ready infrastructure for AI tools. You get Prefect's orchestration plus instant OAuth, serverless scaling, and blazing-fast Python execution. Deploy your AI tools once, connect to Claude, Cursor, or any MCP client. No more building auth flows or managing servers. Prefect orchestrates your ML pipeline. FastMCP handles your AI tool infrastructure. See what Prefect and Fast MCP can do for your AI workflows at aiengineeringpodcast.com/prefect today.
- Your host is Tobias Macey and today I'm interviewing Lucas Thelosen and Drew Gillson about their experiences building an agentic analytics platform and the challenges of ensuring accuracy to build trust
- Introduction
- How did you get involved in machine learning?
- Can you describe what Orion is and the story behind it?
- Business analytics is a field that requires a high degree of accuracy and detail because of the potential for substantial impact on the business (positive and negative). These are areas that generative AI has struggled with achieving consistently. What was your process for building confidence in your ability to achieve that threshold before committing to the path you are on now?
- There are numerous ways that generative AI can be incorporated into the process of designing, building, and delivering analytical insights. How would you characterize the different strategies that data teams and vendors have approached that problem?
- What do you see as the organizational benefits of moving to a push-based model for analytics?
- Can you describe the system architecture of Orion?
- Agentic design patterns are still in the early days of being developed and proven out. Can you give a breakdown of the approach that you are using?
- How do you think about the responsibility boundaries, communication paths, temporal patterns, etc. across the different agents?
- Tool use is a key component of agentic architectures. What is your process for identifying, developing, validating, and securing the tools that you provide to your agents?
- What are the boundaries and extension points that you see when building agentic systems?
- What are the opportunities for using e.g. A2A for protocol for managing agentic hand-offs?
- What is your process for managing the experimentation loop for changes to your models, data, prompts, etc. as you iterate on your product?
- What are some of the ways that you are using the agents that power your system to identify and act on opportunities for self-improvement?
- What are the most interesting, innovative, or unexpected ways that you have seen Orion used?
- What are the most interesting, unexpected, or challenging lessons that you have learned while working on Orion?
- When is an agentic approach the wrong choice?
- What do you have planned for the future of Orion?
Parting Question- From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?
- Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
- Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
- If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.
- To help other people find the show please leave a review on iTunes and tell your friends and co-workers.
- Gravity
- Orion
- Site Reliability Engineering
- Anthropic Claude Sonnet 4.5
- A2A (Agent2Agent) Protocol
- Simon Willison
- Behavioral Science
- Grounded Theory
- LLM as a Judge
- RLHF == Reinforcement Learning from Human Feedback
Kapitel
Kapitel werden geladen …
Building Production-Ready AI Agents with Pydantic AI
Mit Samuel Colvin
SummaryIn this episode of the AI Engineering Podcast Samuel Colvin, creator of Pydantic and founder of Pydantic Inc, talks about Pydantic AI - a type-safe framework for building structured AI agents in Python …
Shownotes
Summary
In this episode of the AI Engineering Podcast Samuel Colvin, creator of Pydantic and founder of Pydantic Inc, talks about Pydantic AI - a type-safe framework for building structured AI agents in Python. Samuel explains why he built Pydantic AI to bring FastAPI-like ergonomics and production-grade engineering to agents, focusing on strong typing, minimal abstractions, and reliability, observability, and stability. He explores the evolving agent ecosystem, patterns for single vs. many agents, graphs vs. durable execution, and how Pydantic AI approaches structured I/O, tool calling, and MCP with type safety in mind. Samuel also shares insights on design trade-offs, model-provider churn, schema unification, safe code execution, security gaps, and the importance of open standards and OpenTelemetry for observability.
Announcements- Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems
- When ML teams try to run complex workflows through traditional orchestration tools, they hit walls. Cash App discovered this with their fraud detection models - they needed flexible compute, isolated environments, and seamless data exchange between workflows, but their existing tools couldn't deliver. That's why Cash App rely on Prefect. Now their ML workflows run on whatever infrastructure each model needs across Google Cloud, AWS, and Databricks. Custom packages stay isolated. Model outputs flow seamlessly between workflows. Companies like Whoop and 1Password also trust Prefect for their critical workflows. But Prefect didn't stop there. They just launched FastMCP - production-ready infrastructure for AI tools. You get Prefect's orchestration plus instant OAuth, serverless scaling, and blazing-fast Python execution. Deploy your AI tools once, connect to Claude, Cursor, or any MCP client. No more building auth flows or managing servers. Prefect orchestrates your ML pipeline. FastMCP handles your AI tool infrastructure. See what Prefect and Fast MCP can do for your AI workflows at aiengineeringpodcast.com/prefect today.
- Your host is Tobias Macey and today I'm interviewing Samuel Colvin about the Pydantic AI framework for building structured AI agents
- Introduction
- How did you get involved in machine learning?
- Can you describe what Pydantic AI is and the story behind it?
- What are the core use cases and capabilities that you are focusing on with PydanticAI?
- The agent SDK landscape has been incredibly crowded and volatile since the introduction of LangChain and LlamaIndex. Can you give your summary of the current state of the ecosystem?
- What are the broad categories that you use when evaluating the various frameworks?
- Beyond the volatility of the frameworks, there is also a rapid pace of evolution in the different styles/patterns of agents. What are the patterns and integrations that Pydantic AI is best suited for?
- Can you describe the overall design/architecture of the Pydantic AI framework?
- How have the design and scope evolved since you first started working on it?
- For someone who wants to build a sophisticated, production-ready AI agent with Pydantic AI, what is your recommended path from idea to deployment?
- What are the elements of the framework that help engineers across those different stages of the lifecycle?
- What are some of the key learnings that you gained from all of your efforts on Pydantic that have been most helpful in developing and promoting Pydantic AI?
- What are some of the new and exciting failure modes that agentic applications introduce as compared to web/mobile/scientific/etc. applications?
- What are the most interesting, innovative, or unexpected ways that you have seen Pydantic AI used?
- What are the most interesting, unexpected, or challenging lessons that you have learned while working on Pydantic AI?
- When is Pydantic AI the wrong choice?
- What do you have planned for the future of Pydantic AI?
Parting Question- From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?
- Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
- Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
- If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.
- To help other people find the show please leave a review on iTunes and tell your friends and co-workers.
- Pydantic
- Pydantic AI
- Pydantic Inc
- Pydantic Logfire
- OpenAI Agents
- Google ADK
- LangChain
- LlamaIndex
- CrewAI
- Durable Execution
- Temporal
- MCP == Model Context Protocol
- Claude Code
- Typescript
- Gemini Structured Output
- OpenAI Structured Output
- Dottxt Outlines SDK
- smolagents
- LiteLLM
- OpenRouter
- OpenAI Responses API
- FastAPI
- SQLModel
- AI SDK JavaScript
- LangGraph
- NextJS
- Pyodide
- AI Elements frontend component library
From GPUs to Workloads: Flex AI’s Blueprint for Fast, Cost‑Efficient AI
Mit Brijesh Tripathi
SummaryIn this episode of the AI Engineering Podcast Brijesh Tripathi, CEO of Flex AI, talks about revolutionizing AI engineering by removing DevOps burdens through "workload as a service". Brijesh shares his …
Shownotes
Summary
In this episode of the AI Engineering Podcast Brijesh Tripathi, CEO of Flex AI, talks about revolutionizing AI engineering by removing DevOps burdens through "workload as a service". Brijesh shares his expertise from leading AI/HPC architecture at Intel and deploying supercomputers like Aurora, highlighting how access friction and idle infrastructure slow progress. He discusses Flex AI's innovative approach to simplifying heterogeneous compute, standardizing on consistent Kubernetes layers, and abstracting inference across various accelerators, allowing teams to iterate faster without wrestling with drivers, libraries, or cloud-by-cloud differences. Brijesh also shares insights into Flex AI's strategies for lifting utilization, protecting real-time workloads, and spanning the full lifecycle from fine-tuning to autoscaled inference, all while keeping complexity at bay.
Announcements- Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems
- When ML teams try to run complex workflows through traditional orchestration tools, they hit walls. Cash App discovered this with their fraud detection models - they needed flexible compute, isolated environments, and seamless data exchange between workflows, but their existing tools couldn't deliver. That's why Cash App rely on Prefect. Now their ML workflows run on whatever infrastructure each model needs across Google Cloud, AWS, and Databricks. Custom packages stay isolated. Model outputs flow seamlessly between workflows. Companies like Whoop and 1Password also trust Prefect for their critical workflows. But Prefect didn't stop there. They just launched FastMCP - production-ready infrastructure for AI tools. You get Prefect's orchestration plus instant OAuth, serverless scaling, and blazing-fast Python execution. Deploy your AI tools once, connect to Claude, Cursor, or any MCP client. No more building auth flows or managing servers. Prefect orchestrates your ML pipeline. FastMCP handles your AI tool infrastructure. See what Prefect and Fast MCP can do for your AI workflows at aiengineeringpodcast.com/prefect today.
- Your host is Tobias Macey and today I'm interviewing Brijesh Tripathi about FlexAI, a platform offering a service-oriented abstraction for AI workloads
- Introduction
- How did you get involved in machine learning?
- Can you describe what FlexAI is and the story behind it?
- What are some examples of the ways that infrastructure challenges contribute to friction in developing and operating AI applications?
- How do those challenges contribute to issues when scaling new applications/businesses that are founded on AI?
- There are numerous managed services and deployable operational elements for operationalizing AI systems. What are some of the main pitfalls that teams need to be aware of when determining how much of that infrastructure to own themselves?
- Orchestration is a key element of managing the data and model lifecycles of these applications. How does your approach of "workload as a service" help to mitigate some of the complexities in the overall maintenance of that workload?
- Can you describe the design and architecture of the FlexAI platform?
- How has the implementation evolved from when you first started working on it?
- For someone who is going to build on top of FlexAI, what are the primary interfaces and concepts that they need to be aware of?
- Can you describe the workflow of going from problem to deployment for an AI workload using FlexAI?
- One of the perennial challenges of making a well-integrated platform is that there are inevitably pre-existing workloads that don't map cleanly onto the assumptions of the vendor. What are the affordances and escape hatches that you have built in to allow partial/incremental adoption of your service?
- What are the elements of AI workloads and applications that you are explicitly not trying to solve for?
- What are the most interesting, innovative, or unexpected ways that you have seen FlexAI used?
- What are the most interesting, unexpected, or challenging lessons that you have learned while working on FlexAI?
- When is FlexAI the wrong choice?
- What do you have planned for the future of FlexAI?
Parting Question- From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?
- Flex AI
- Aurora Super Computer
- CoreWeave
- Kubernetes
- CUDA
- ROCm
- Tensor Processing Unit (TPU)
- PyTorch
- Triton
- Trainium
- ASIC == Application Specific Integrated Circuit
- SOC == System On a Chip
- Loveable
- FlexAI Blueprints
- Tenstorrent
Kapitel
Kapitel werden geladen …
Right-Sizing AI: Small Language Models for Real-World Production
Mit Steven Huels
SummaryIn this episode of the AI Engineering Podcast Steven Huels, Vice President of AI Engineering & Product Strategy at Red Hat, talks about the practical applications of small language models (SLMs) for …
Shownotes
Summary
In this episode of the AI Engineering Podcast Steven Huels, Vice President of AI Engineering & Product Strategy at Red Hat, talks about the practical applications of small language models (SLMs) for production workloads. He discusses how SLMs offer a pragmatic choice due to their ability to fit on single enterprise GPUs and provide model selection trade-offs. The conversation covers self-hosting vs using API providers, organizational capabilities needed for running production-grade LLMs, and the importance of guardrails and automated evaluation at scale. They also explore the rise of agentic systems and service-oriented approaches powered by smaller models, highlighting advances in customization and deployment strategies. Steven shares real-world examples and looks to the future of agent cataloging, continuous retraining, and resource efficiency in AI engineering.
Announcements- Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems
- When ML teams try to run complex workflows through traditional orchestration tools, they hit walls. Cash App discovered this with their fraud detection models - they needed flexible compute, isolated environments, and seamless data exchange between workflows, but their existing tools couldn't deliver. That's why Cash App rely on Prefect. Now their ML workflows run on whatever infrastructure each model needs across Google Cloud, AWS, and Databricks. Custom packages stay isolated. Model outputs flow seamlessly between workflows. Companies like Whoop and 1Password also trust Prefect for their critical workflows. But Prefect didn't stop there. They just launched FastMCP - production-ready infrastructure for AI tools. You get Prefect's orchestration plus instant OAuth, serverless scaling, and blazing-fast Python execution. Deploy your AI tools once, connect to Claude, Cursor, or any MCP client. No more building auth flows or managing servers. Prefect orchestrates your ML pipeline. FastMCP handles your AI tool infrastructure. See what Prefect and Fast MCP can do for your AI workflows at aiengineeringpodcast.com/prefect today.
- Your host is Tobias Macey and today I'm interviewing Steven Huels about the benefits of small language models for production workloads
- Introduction
- How did you get involved in machine learning?
- Language models are available in a wide range of sizes, measured both in terms of parameters and disk space. What are your heuristics for deciding what qualifies as a "small" vs. "large" language model?
- What are the corresponding heuristics for when to use a small vs. large model?
- The predominant use case for small models is in self-hosted contexts, which requires a certain amount of organizational sophistication. What are some helpful questions to ask yourself when determining whether to implement a model-serving stack vs. relying on hosted options?
- What are some examples of "small" models that you have seen used effectively?
- The buzzword right now is "agentic" for AI driven workloads. How do small models fit in the context of agent-based workloads?
- When and where should you rely on larger models?
- When speaking of small models, one of the common requirements for making them truly useful is to fine-tune them for your problem domain and organizational data. How has the complexity and difficulty of that operation changed over the past ~2 years?
- Serving models requires several operational capabilities beyond the raw inference serving. What are the other infrastructure and organizational investments that teams should be aware of as they embark on that path?
- What are the most interesting, innovative, or unexpected ways that you have seen small language models used?
- What are the most interesting, unexpected, or challenging lessons that you have learned while working on operationalizing inference and model customization?
- When is a small or self-hosted language model the wrong choice?
- What are your predictions for the near future of small language model capabilities/availability?
Parting Question- From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?
- Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
- Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
- If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.
- To help other people find the show please leave a review on iTunes and tell your friends and co-workers.
- RedHat AI Engineering
- Generative AI
- Predictive AI
- ChatGPT
- QLORA
- HuggingFace
- vLLM
- OpenShift AI
- Llama Models
- DeepSeek
- GPT-OSS
- Mistral
- Mixture of Experts (MoE)
- Qwen
- InstructLab
- SFT == Supervised Fine Tuning
- LORA
Kapitel
Kapitel werden geladen …
AI Agents and Identity Management
Mit Julianna Lamb
SummaryIn this episode of the AI Engineering Podcast Julianna Lamb, co-founder and CTO of Stytch, talks about the complexities of managing identity and authentication in agentic workflows. She explores the …
Shownotes
Summary
In this episode of the AI Engineering Podcast Julianna Lamb, co-founder and CTO of Stytch, talks about the complexities of managing identity and authentication in agentic workflows. She explores the evolving landscape of identity management in the context of machine learning and AI, highlighting the importance of flexible compute environments and seamless data exchange. The conversation covers implications of AI agents on identity management, including granular permissions, OAuth protocol, and adapting systems for agentic interactions. Julianna also discusses rate limiting, persistent identity, and evolving standards for managing identity in AI systems. She emphasizes the need to experiment with AI agents and prepare systems for integration to stay ahead in the rapidly advancing AI landscape.
Announcements- Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems
- When ML teams try to run complex workflows through traditional orchestration tools, they hit walls. Cash App discovered this with their fraud detection models - they needed flexible compute, isolated environments, and seamless data exchange between workflows, but their existing tools couldn't deliver. That's why Cash App rely on Prefect. Now their ML workflows run on whatever infrastructure each model needs across Google Cloud, AWS, and Databricks. Custom packages stay isolated. Model outputs flow seamlessly between workflows. Companies like Whoop and 1Password also trust Prefect for their critical workflows. But Prefect didn't stop there. They just launched FastMCP - production-ready infrastructure for AI tools. You get Prefect's orchestration plus instant OAuth, serverless scaling, and blazing-fast Python execution. Deploy your AI tools once, connect to Claude, Cursor, or any MCP client. No more building auth flows or managing servers. Prefect orchestrates your ML pipeline. FastMCP handles your AI tool infrastructure. See what Prefect and Fast MCP can do for your AI workflows at aiengineeringpodcast.com/prefect today.
- Your host is Tobias Macey and today I'm interviewing Julianna Lamb about the complexities of managing identity and auth in agentic workflows
- Introduction
- How did you get involved in machine learning?
- The term "identity" is very overloaded. Can you start by giving your definition in the context of technical systems?
- What are some of the different ways that AI agents intersect with identity?
- We have decades of experience and effort in building identity infrastructure for the internet, what are the most significant ways in which that is insufficient for agent-based use cases?
- I have heard anecdotal references to the ways in which AI agents lead to a proliferation of "identities". How would you characterize the magnitude of the difference in scale between human-powered identity, deterministic automation (e.g. bots or bot-nets), and AI agents?
- The other major element of establishing and verifying "identity" is how that intersects with permissions or authorization. What are the major shortcomings of our existing investment in managing and auditing access and control once you are within a system?
- How does that get amplified with AI agents?
- Typically authentication has been done at the perimeter of a system. How does that architecture change when accounting for AI agents?
- How does that get complicated by where the agent originates? (e.g external agents interacting with a third-party system vs. internal agents operated by the service provider)
- What are the concrete steps that engineering teams should be taking today to start preparing their systems for agentic use-cases (internal or external)?
- How do agentic capabilities change the means of protecting against malicious bots? (e.g. bot detection, defensive agents, etc.)
- What are the most interesting, innovative, or unexpected ways that you have seen authn/authz/identity addressed for AI use cases?
- What are the most interesting, unexpected, or challenging lessons that you have learned while working on identity/auth(n|z) systems?
- What are your predictions for the future of identity as adoption and sophistication of AI systems progresses?
Parting Question- From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?
- Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
- Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
- If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.
- To help other people find the show please leave a review on iTunes and tell your friends and co-workers.
- Stytch
- AI Agent
- Machine To Machine Authentication
- API Authentication
- MCP == Model Context Protocol
- OAuth
- Identity Provider
- OAuth Scopes
- OAuth 2.1
- Captcha
- RBAC == Role-Based Access Control
- ABAC == Attribute-Based Access Control
- ReBAC == Relationship-Based Access Control
- Google Zanzibar
- Idempotence
- Dynamic Client Registration
- Large Action Models
- Claude Code
Kapitel
Kapitel werden geladen …
Revolutionizing Production Systems: The Resolve AI Approach
Mit Spiros Xanthos
SummaryIn this episode of the AI Engineering Podcast, CEO of Resolve AI Spiros Xanthos shares his insights on building agentic capabilities for operational systems. He discusses the limitations of traditional …
Shownotes
Summary
In this episode of the AI Engineering Podcast, CEO of Resolve AI Spiros Xanthos shares his insights on building agentic capabilities for operational systems. He discusses the limitations of traditional observability tools and the need for AI agents that can reason through complex systems to provide actionable insights and solutions. The conversation highlights the architecture of Resolve AI, which integrates with existing tools to build a comprehensive understanding of production environments, and emphasizes the importance of context and memory in AI systems. Spiros also touches on the evolving role of AI in production systems, the potential for AI to augment human operators, and the need for continuous learning and adaptation to fully leverage these advancements.
Announcements- Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems
- Your host is Tobias Macey and today I'm interviewing Spiros Xanthos about architecting agentic capabilities for operational challenges with managing production systems.
- Introduction
- How did you get involved in machine learning?
- Can you describe what Resolve AI is and the story behind it?
- We have decades of experience as an industry in managing operational complexity. What are the critical failures in capabilities that you are addressing with the application of AI?
- Given the existing capabilities of dedicated platforms (e.g. Grafana, PagerDuty, Splunk, etc), what is your reasoning for building a new system vs. a new feature of existing operational product?
- Over the past couple of years the industry has developed a growing number of agent patterns. What was your approach in evaluating and selecting a particular approach for your product?
- One of the complications of building any platform that supports operational needs of engineering teams is the complexity of integrating with their technology stack. This is doubly true when building an AI system that needs rich context. What are the core primitives that you are relying on to build a robust offering?
- How are you managing the learning process for your systems to allow for iterative discovery and improvement?
- What are your strategies for personalizing those discoveries to a given customer and operating environment?
- One of the interesting challenges in agentic systems is managing the user experience for human-in-the-loop and machine to human handoffs in each direction. How are you thinking about that, especially given the criticality of the systems that you are interacting with?
- As more of the code that is running in production environments is co-developed with AI, what impact do you anticipate on the overall operational resilience of the systems being monitored?
- One of the challenges of working with LLMs is the cold start problem where every conversation starts from scratch. How are you approaching the overall problem of context engineering and ensuring that you are consistently providing the necessary information for the model to be effective in its role?
- What are the most interesting, innovative, or unexpected ways that you have seen Resolve AI used?
- What are the most interesting, unexpected, or challenging lessons that you have learned while working on Resolve AI?
- When is Resolve AI the wrong choice?
- What do you have planned for the future of Resolve AI?
Parting Question- From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?
- Thank you for listening! Don't forget to check out our other shows. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. The AI Engineering Podcast is your guide to the fast-moving world of building AI systems.
- Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
- If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.
- Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
- Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
- If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.
- To help other people find the show please leave a review on iTunes and tell your friends and co-workers.
The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0Kapitel
Kapitel werden geladen …
Designing Scalable AI Systems with FastMCP: Challenges and Innovations
Mit Jeremiah Lowin
SummaryIn this episode of the AI Engineering Podcast Jeremiah Lowin, founder and CEO of Prefect Technologies, talks about the FastMCP framework and the design of MCP servers. Jeremiah explains the evolution of …
Shownotes
Summary
In this episode of the AI Engineering Podcast Jeremiah Lowin, founder and CEO of Prefect Technologies, talks about the FastMCP framework and the design of MCP servers. Jeremiah explains the evolution of FastMCP, from its initial creation as a simpler alternative to the MCP SDK to its current role in facilitating the deployment of AI tools. The discussion covers the complexities of designing MCP servers, the importance of context engineering, and the potential pitfalls of overwhelming AI agents with too many tools. Jeremiah also highlights the importance of simplicity and incremental adoption in software design, and shares insights into the future of MCP and the broader AI ecosystem. The episode concludes with a look at the challenges of authentication and authorization in AI applications and the exciting potential of MCP as a protocol for the future of AI-driven business logic.
Announcements- Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems
- Your host is Tobias Macey and today I'm interviewing Jeremiah Lowin about the FastMCP framework and how to design and build your own MCP servers
- Introduction
- How did you get involved in machine learning?
- Can you start by describing what MCP is and its purpose in the ecosystem of AI applications?
- What is FastMCP and what motivated you to create it?
- Recognizing that MCP is relatively young, how would you characterize the landscape of MCP frameworks?
- What are some of the stumbling blocks on the path to building a well engineered MCP server?
- What are the potential ramifications of poorly designed and implemented MCP implementations?
- In the overall context of an AI-powered/agentic application, what are the tradeoffs of investing in the MCP protocol? (e.g. engineering effort, process isolation, tool creation, auth(n|z), etc.)
- In your experience, what are the architectural patterns that you see of MCP implementation and usage?
- There are a multitude of MCP servers available for a variety of use cases. What are the key factors that someone should be using to evaluate their viability for a production use case?
- Can you give an overview of the key characteristics of FastMCP and why someone might select it as their implementation target for a custom MCP server?
- How have the design, scope, and goals of the project evolved since you first started working on it?
- For someone who is using FastMCP as the framework for creating their own AI tools, what are some of the design considerations or best practices that they should be aware of?
- What are some of the ways that someone might consider integrating FastMCP into their existing Python-powered web applications (e.g. FastAPI, Django, Flask, etc.)
- As you continue to invest your time and energy into FastMCP, what is your overall goal for the project?
- What are the most interesting, innovative, or unexpected ways that you have seen FastMCP used?
- What are the most interesting, unexpected, or challenging lessons that you have learned while working on FastMCP?
- When is FastMCP the wrong choice?
- What do you have planned for the future of FastMCP?
Parting Question- From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?
- Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
- Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
- If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.
- To help other people find the show please leave a review on iTunes and tell your friends and co-workers.
- FastMCP
- FastMCP Cloud
- Prefect
- Model Context Protocol (MCP)
- AI Tools
- FastAPI
- Python Decorator
- Websockets
- SSE == Server-Sent Events
- Streamable HTTP
- OAuth
- MCP Gateway
- MCP Sampling
- Flask
- Django
- ASGI
- MCP Elicitation
- AuthKit
- Dynamic Client Registration
- smolagents
- Large Active Models
- A2A
Kapitel
Kapitel werden geladen …
Proactive Monitoring in Heavy Industry: The Role of AI and Human Curiosity
SummaryIn this episode of the AI Engineering Podcast Dr. Tara Javidi, CTO of KavAI, talks about developing AI systems for proactive monitoring in heavy industry. Dr. Javidi shares her background in mathematics …
Shownotes
Summary
In this episode of the AI Engineering Podcast Dr. Tara Javidi, CTO of KavAI, talks about developing AI systems for proactive monitoring in heavy industry. Dr. Javidi shares her background in mathematics and information theory, influenced by Claude Shannon's work, and discusses her approach to curiosity-driven AI that mimics human curiosity to improve data collection and predictive analytics. She explains how KavAI's platform uses generative AI models to enhance industrial monitoring by addressing informational blind spots and reducing reliance on human oversight. The conversation covers the architecture of KavAI's systems, integrating AI with existing workflows, building trust with operators, and the societal impact of AI in preventing environmental catastrophes, ultimately highlighting the future potential of information-centric AI models.
Announcements- Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems.
- Your host is Tobias Macey and today I'm interviewing Dr. Tara Javidi about building AI systems for proactive monitoring of physical environments for heavy industry
- Introduction
- How did you get involved in machine learning?
- Can you describe what KavAI is and the story behind it?
- What are some of the current state-of-the-art applications of AI/ML for monitoring and accident prevention in industrial environments?
- What are the shortcomings of those approaches?
- What are some examples of the types of harm that you are focused on preventing or mitigating with your platform?
- On your site it mentions that you have created a foundation model for physical awareness. What are some examples of the types of predictive/generative capabilities that your model provides?
- A perennial challenge when building any digital model of a physical system is the lack of absolute fidelity. What are the key sources of information acquisition that you rely on for your platform?
- In addition to your foundation model, what are the other systems that you incorporate to perform analysis and catalyze action?
- Can you describe the overall system architecture of your platform?
- What are some of the ways that you are able to integrate learnings across industries and environments to improve the overall capacity of your models?
- What are the most interesting, innovative, or unexpected ways that you have seen KavAI used?
- What are the most interesting, unexpected, or challenging lessons that you have learned while working on KavAI?
- When is KavAI/Physical AI the wrong choice?
- What do you have planned for the future of KavAI?
Parting Question- From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?
The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0Kapitel
Kapitel werden geladen …
Navigating the AI Landscape: Challenges and Innovations in Retail
Mit Shashank Kapadia
SummaryIn this episode of the AI Engineering Podcast machine learning engineer Shashank Kapadia explores the transformative role of generative AI in retail. Shashank shares his journey from an engineering …
Shownotes
Summary
In this episode of the AI Engineering Podcast machine learning engineer Shashank Kapadia explores the transformative role of generative AI in retail. Shashank shares his journey from an engineering background to becoming a key player in ML, highlighting the excitement of understanding human behavior at scale through AI. He discusses the challenges and opportunities presented by generative AI in retail, where it complements traditional ML by enhancing explainability and personalization, predicting consumer needs, and driving autonomous shopping agents and emotional commerce. Shashank elaborates on the architectural and operational shifts required to integrate generative AI into existing systems, emphasizing orchestration, safety nets, and continuous learning loops, while also addressing the balance between building and buying AI solutions, considering factors like data privacy and customization.
Announcements- Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems
- Your host is Tobias Macey and today I'm interviewing Shashank Kapadia about applications of generative AI in retail
- Introduction
- How did you get involved in machine learning?
- Can you summarize the main applications of generative AI that you are seeing the most benefit from in retail/ecommerce?
- What are the major architectural patterns that you are deploying for generative AI workloads?
- Working at an organization like WalMart, you already had a substantial investment in ML/MLOps. What are the elements of that organizational capability that remain the same, and what are the catalyzed changes as a result of generative models?
- When working at the scale of Walmart, what are the different types of bottlenecks that you encounter which can be ignored at smaller orders of magnitude?
- Generative AI introduces new risks around brand reputation, accuracy, trustworthiness, etc. What are the architectural components that you find most effective in managing and monitoring the interactions that you provide to your customers?
- Can you describe the architecture of the technical systems that you have built to enable the organization to take advantage of generative models?
- What are the human elements that you rely on to ensure the safety of your AI products?
- What are the most interesting, innovative, or unexpected ways that you have seen generative AI break at scale?
- What are the most interesting, unexpected, or challenging lessons that you have learned while working on AI?
- When is generative AI the wrong choice?
- What are your paying special attention to over the next 6 - 36 months in AI?
Parting Question- From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?
- Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
- Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
- If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.
- To help other people find the show please leave a review on iTunes and tell your friends and co-workers.
The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0Kapitel
Kapitel werden geladen …
The Anti-CRM CRM: How Spiro Uses AI to Transform Sales
SummaryIn this episode of the AI Engineering podcast Adam Honig, founder of Spiro AI, about using AI to automate CRM systems, particularly in the manufacturing sector. Adam shares his journey from running a …
Shownotes
Summary
In this episode of the AI Engineering podcast Adam Honig, founder of Spiro AI, about using AI to automate CRM systems, particularly in the manufacturing sector. Adam shares his journey from running a consulting company focused on Salesforce to founding Spiro, and discusses the challenges of traditional CRM systems where data entry is often neglected. He explains how Spiro addresses this issue by automating data collection from emails, phone calls, and other communications, providing a rich dataset for machine learning models to generate valuable insights. Adam highlights how Spiro's AI-driven CRM system is tailored to the manufacturing industry's unique needs, where sales are relationship-driven rather than funnel-based, and emphasizes the importance of understanding customer interactions and order histories to predict future business opportunities. The conversation also touches on the evolution of AI models, leveraging powerful third-party APIs, managing context windows, and platform dependencies, with Adam sharing insights into Spiro's future plans, including product recommendations and dynamic data modeling approaches.
Announcements- Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems
- Your host is Tobias Macey and today I'm interviewing Adam Honig about using AI to automate CRM maintenance
- Introduction
- How did you get involved in machine learning?
- Can you describe what Spiro is and the story behind it?
- What are the specific challenges posed by the manufacturing industry with regards to sales and customer interactions?
- How does the type of manufacturing and target customer influence the level of effort and communication involved in the sales and customer service cycles?
- Before we discuss the opportunities for automation, can you describe the typical interaction patterns and workflows involved in the care and feeding of CRM systems?
- Spiro has been around since 2014, long pre-dating the current era of generative models. What were your initial targets for improving efficiency and reducing toil for your customers with the aid of AI/ML?
- How have the generational changes of deep learning and now generative AI changed the ways that you think about what is possible in your product?
- Generative models reduce the level of effort to get a proof of concept for language-oriented workflows. How are you pairing them with more narrow AI that you have built?
- Can you describe the overall architecture of your platform and how it has evolved in recent years?
- While generative models are powerful, they can also become expensive, and the costs are hard to predict. How are you thinking about vendor selection and platform risk in the application of those models?
- What are the opportunities that you see for the adoption of more autonomous applications of language models in your product? (e.g. agents)
- What are the confidence building steps that you are focusing on as you investigate those opportunities?
- What are the most interesting, innovative, or unexpected ways that you have seen Spiro used?
- What are the most interesting, unexpected, or challenging lessons that you have learned while working on AI in the CRM space?
- When is AI the wrong choice for CRM workflows?
- What do you have planned for the future of Spiro?
Parting Question- From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?
- Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
- Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
- If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.
- To help other people find the show please leave a review on iTunes and tell your friends and co-workers.
The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0Kapitel
Kapitel werden geladen …
Unlocking AI Potential with AMD's ROCm Stack
Mit Anush Elangovan
SummaryIn this episode of the AI Engineering podcast Anush Elangovan, VP of AI software at AMD, discusses the strategic integration of software and hardware at AMD. He emphasizes the open-source nature of their …
Shownotes
Summary
In this episode of the AI Engineering podcast Anush Elangovan, VP of AI software at AMD, discusses the strategic integration of software and hardware at AMD. He emphasizes the open-source nature of their software, fostering innovation and collaboration in the AI ecosystem, and highlights AMD's performance and capability advantages over competitors like NVIDIA. Anush addresses challenges and opportunities in AI development, including quantization, model efficiency, and future deployment across various platforms, while also stressing the importance of open standards and flexible solutions that support efficient CPU-GPU communication and diverse AI workloads.
Announcements- Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems
- Your host is Tobias Macey and today I'm interviewing Anush Elangovan about AMD's work to expand the playing field for AI training and inference
- Introduction
- How did you get involved in machine learning?
- Can you describe what your work at AMD is focused on?
- A lot of the current attention on hardware for AI training and inference is focused on the raw GPU hardware. What is the role of the software stack in enabling and differentiating that underlying compute?
- CUDA has gained a significant amount of attention and adoption in the numeric computation space (AI, ML, scientific computing, etc.). What are the elements of platform risk associated with relying on CUDA as a developer or organization?
- The ROCm stack is the key element in AMD's AI and HPC strategy. What are the elements that comprise that ecosystem?
- What are the incentives for anyone outside of AMD to contribute to the ROCm project?
- How would you characterize the current competitive landscape for AMD across the AI/ML lifecycle stages? (pre-training, post-training, inference, fine-tuning)
- For teams who are focused on inference compute for model serving, what do they need to know/care about in regards to AMD hardware and the ROCm stack?
- What are the most interesting, innovative, or unexpected ways that you have seen AMD/ROCm used?
- What are the most interesting, unexpected, or challenging lessons that you have learned while working on AMD's AI software ecosystem?
- When is AMD/ROCm the wrong choice?
- What do you have planned for the future of ROCm?
Parting Question- From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?
- Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
- Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
- If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.
- To help other people find the show please leave a review on iTunes and tell your friends and co-workers.
- ImageNet
- AMD
- ROCm
- CUDA
- HuggingFace
- Llama 3
- Llama 4
- Qwen
- DeepSeek R1
- MI300X
- Nokia Symbian
- UALink Standard
- Quantization
- HIPIFY
- ROCm Triton
- AMD Strix Halo
- AMD Epyc
- Liquid Networks
- MAMBA Architecture
- Transformer Architecture
- NPU == Neural Processing Unit
- llama.cpp
- Ollama
- Perplexity Score
- NUMA == Non-Uniform Memory Access
- vLLM
- SGLang
Kapitel
Kapitel werden geladen …
Applying AI To The Construction Industry At Buildots
Mit Ori Silberberg
SummaryIn this episode of the Machine Learning Podcast Ori Silberberg, VP of Engineering at Buildots, talks about transforming the construction industry with AI. Ori shares how Buildots uses computer vision and …
Shownotes
Summary
In this episode of the Machine Learning Podcast Ori Silberberg, VP of Engineering at Buildots, talks about transforming the construction industry with AI. Ori shares how Buildots uses computer vision and AI to optimize construction projects by providing real-time feedback, reducing delays, and improving efficiency. Learn about the complexities of digitizing the construction industry, the technical architecture of Buildoz, and how its AI-driven solutions create a digital twin of construction sites. Ori emphasizes the importance of explainability and actionable insights in AI decision-making, highlighting the potential of generative AI to further enhance the construction process from planning to execution.
Announcements- Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems
- Your host is Tobias Macey and today I'm interviewing Ori Silberberg about applications of AI for optimizing building construction
- Introduction
- How did you get involved in machine learning?
- Can you describe what Buildotds is and the story behind it?
- What types of construction projects are you focused on? (e.g. residential, commercial, industrial, etc.)
- What are the main types of inefficiencies that typically occur on those types of job sites?
- What are the manual and technical processes that the industry has typically relied on to address those sources of waste and delay?
- In many ways the construction industry is as old as civilization. What are the main ways that the information age has transformed construction?
- What are the elements of the construction industry that make it resistant to digital transformation?
- Can you describe how you are applying AI to this complex and messy problem?
- What are the types of data that you are able to collect?
- How are you automating that data collection so that construction crews don't have to add extra work or distractions to their day?
- For construction crews that are using Buildots, can you talk through how it integrates into the overall process from site planning to project completion?
- Can you describe the technical architecture of the Buildots platform?
- Given the safety critical nature of construction, how does that influence the way that you think about the types of AI models that you use and where to apply them?
- What are the most interesting, innovative, or unexpected ways that you have seen Buildots used?
- What are the most interesting, unexpected, or challenging lessons that you have learned while working on Buildots?
- What do you have planned for the future of AI usage at Buildots?
Parting Question- From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?
- Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
- Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
- If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.
- To help other people find the show please leave a review on iTunes and tell your friends and co-workers.
The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0Kapitel
Kapitel werden geladen …
The Future of AI Systems: Open Models and Infrastructure Challenges
Mit Jamie De Guerre
SummaryIn this episode of the AI Engineering Podcast Jamie De Guerre, founding SVP of product at Together.ai, explores the role of open models in the AI economy. As a veteran of the AI industry, including his …
Shownotes
Summary
In this episode of the AI Engineering Podcast Jamie De Guerre, founding SVP of product at Together.ai, explores the role of open models in the AI economy. As a veteran of the AI industry, including his time leading product marketing for AI and machine learning at Apple, Jamie shares insights on the challenges and opportunities of operating open models at speed and scale. He delves into the importance of open source in AI, the evolution of the open model ecosystem, and how Together.ai's AI acceleration cloud is contributing to this movement with a focus on performance and efficiency.
Announcements- Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems
- Your host is Tobias Macey and today I'm interviewing Jamie de Guerre about the role of open models in the AI economy and how to operate them at speed and at scale
- Introduction
- How did you get involved in machine learning?
- Can you describe what Together AI is and the story behind it?
- What are the key goals of the company?
- The initial rounds of open models were largely driven by massive tech companies. How would you characterize the current state of the ecosystem that is driving the creation and evolution of open models?
- There was also a lot of argument about what "open source" and "open" means in the context of ML/AI models, and the different variations of licenses being attached to them (e.g. the Meta license for Llama models). What is the current state of the language used and understanding of the restrictions/freedoms afforded?
- What are the phases of organizational/technical evolution from initial use of open models through fine-tuning, to custom model development?
- Can you outline the technical challenges companies face when trying to train or run inference on large open models themselves?
- What factors should a company consider when deciding whether to fine-tune an existing open model versus attempting to train a specialized one from scratch?
- While Transformers dominate the LLM landscape, there's ongoing research into alternative architectures. Are you seeing significant interest or adoption of non-Transformer architectures for specific use cases?
- When might those other architectures be a better choice?
- While open models offer tremendous advantages like transparency, control, and cost-effectiveness, are there scenarios where relying solely on them might be disadvantageous?
- When might proprietary models or a hybrid approach still be the better choice for a specific problem?
- Building and scaling AI infrastructure is notoriously complex. What are the most significant technical or strategic challenges you've encountered at Together AI while enabling scalable access to open models for your users?
- What are the most interesting, innovative, or unexpected ways that you have seen open models/the TogetherAI platform used?
- What are the most interesting, unexpected, or challenging lessons that you have learned while working on powering AI model training and inference?
- Where do you see the open model space heading in the next 1-2 years? Any specific trends or breakthroughs you anticipate?
Parting Question- From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?
- Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
- Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
- If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.
- To help other people find the show please leave a review on iTunes and tell your friends and co-workers.
- Together AI
- Fine Tuning
- Post-Training
- Salesforce Research
- Mistral
- Agentforce
- Llama Models
- RLHF == Reinforcement Learning from Human Feedback
- RLVR == Reinforcement Learning from Verifiable Rewards
- Test Time Compute
- HuggingFace
- RAG == Retrieval Augmented Generation
- Google Gemma
- Llama 4 Maverick
- Prompt Engineering
- vLLM
- SGLang
- Hazy Research lab
- State Space Models
- Hyena Model
- Mamba Architecture
- Diffusion Model Architecture
- Stable Diffusion
- Black Forest Labs Flux Model
- Nvidia Blackwell
- PyTorch
- Rust
- Deepseek R1
- GGUF
- Pika Text To Video
Kapitel
Kapitel werden geladen …
Hier stehen die neuesten 40 von 82 Folgen. Ältere Folgen findest du im Feed des Podcasts.
Empfohlen von AI Engineering Podcast
Ähnliche Podcasts
Mehr Bildung-Podcasts
Ist das dein Podcast?
Bestätige es mit einem Eintrag in deinem Feed: Du bekommst hier einen Code, trägst ihn als podcast:txt in den Kanal deines Feeds ein, und wir prüfen ihn. Danach steht dein Podcast als „verifiziert“ bei SoundNode.