E18: Shawn Hymel on Edge AI - NPUs, Deployment Challenges, and the Future of Embedded ML

Ryan and Luca sit down with Shawn Hymel, an educator and course creator focused on edge AI and embedded systems. We explore what's changed in the past few years—from basic keyword spotting to full object detection on microcontrollers, thanks to integrated NPUs. Shawn walks us through the messy reality of deploying ML models to embedded hardware: vendor-specific toolchains, dependency hell, and the ongoing challenge of making edge AI accessible. We discuss what students and experienced engineers need to learn (or unlearn) to work effectively in this space, and look ahead to exciting developments like reinforcement learning on tiny devices and neuromorphic computing. It's a candid, technical conversation about where edge AI stands today and where it's headed.

Key Topics:

• [03:30] Why run ML on microcontrollers? Power, size, and application-specific advantages
• [06:00] Keyword spotting as the original killer app for edge AI
• [08:45] The game-changer: NPUs enabling full object detection on microcontrollers
• [13:20] Privacy benefits of on-device processing vs. cloud-based inference
• [16:00] How NPUs work under the hood and the vendor-specific deployment reality
• [24:30] The painful parts: dependency hell, graph compilers, and memory arena sizing
• [29:00] Using AI tools (LLMs) to navigate vendor documentation and generate code
• [33:45] What CS students and ECE students each need to learn for edge AI
• [40:15] Shifts in university enrollment: ECE rising, CS declining
• [44:00] When you don't need AI: PID loops and deterministic solutions still matter
• [47:30] Looking ahead: reinforcement learning on microcontrollers and neuromorphic computing

Notable Quotes:

"Five, six years ago, we didn't have full object detection on microcontrollers. Now with NPUs, we can do things like full YOLO on a 320x320 image—milliwatts of power, full object detection. That was not a thing five years ago." — Shawn Hymel

"Expect to spend a day or two getting inference to actually run. The docs are still new, the graph compilers are fairly new. You're going to end up in dependency hell—both on the Python side and when you bring it over to the embedded side." — Shawn Hymel

"If a PID loop solves your need, there is absolutely no reason to put AI on there. That is a solved problem. Don't do it—just use a PID loop." — Shawn Hymel

Resources Mentioned:

• Shawn Hymel's Website - Shawn's main site with links to free and paid courses on edge AI and embedded systems
• OpenMV - Computer vision platform for microcontrollers, including the new AE3 board with NPU support
• Edge Impulse - Platform that simplifies ML model deployment to embedded devices, supporting various NPUs
• Andrew Ng's Coursera ML Course - Foundational machine learning course recommended for understanding the math behind ML
• TensorFlow Lite for Microcontrollers (LiteRT) - Framework for running ML models on microcontrollers across different platforms