Meta and AMD Co-Design MI350X for Recommendation Model Training

This episode examines a Meta and AMD paper claiming that AMD's MI350X can train Meta's production recommendation models at roughly the throughput of NVIDIA's B200. It explains why recommendation models are a harder training target than LLMs: terabyte-scale sharded embedding tables, irregular gathers, heavy all-to-all communication, odd-shaped matrix multiplies, and atomic-update-heavy backward passes. It compares the two chips. Peak compute and memory bandwidth are close, the MI350X has about 1.6 times the HBM capacity, and the B200 has faster scale-up interconnect. The discussion defines "co-design" as adapting the model recipe to a purchased chip, for example by dropping activation checkpointing to use the extra memory, rather than building the platform as in Meta's earlier ZionEX work. It also raises fairness concerns about the headline parity: the recipe was changed for the AMD hardware, several co-authors are AMD engineers, and the MI350X ran at 1000 watts against the B200's 750-watt cap. The episode previews the paper's analytical GEMM tile-selection model and its agentic CUDA-to-AMD kernel porting. Sources: 1. Architecting for Hardware Diversity: Meta's End-to-End Co-Design for High-Performance AI Training on AMD GPUs — Haoyu Zhang, Arshad Rahman, Adnan Aziz, Ajith Prabhudev, Amin Farmahini, Ben Rhoads, Benson Ma, Brian Shi, Cen Zhao, CK Luk, Dingming Wu, Emad El-Haraty, Eric Chen, Frank Li, Gary Huang, Gefei Zuo, Haoran Zhang, Hiwot Kassa, Hongyi Zeng, Jack Taylor, Jeremy Hadidjojo, John Donati, Jonathan Sheffield, Julio Machado Silva, Karthik Ganesan, Kent Russell, KR Kishore, Lei Tian, Liu Ke, Muhammad Osama, Peng Chen, Rakesh Hemnani, Ryan Swann, Saleel Kudchadker, Shankar Yanamandram, Shobhit Kanaujia, Shucai Xiao, Srinath Bayareddy, Srinivas Vaidyanathan, Sudharssun Subramanian, Supadchaya Puangpontip, Taylor Robie, Tianhao Huang, Tyler Hart, Viswesh Sankaran, Weijun Jiang, Wenkai Du, Xiaodong Wang, Xiaohu Guo, Yao Rong, Yinbin Ma, Yue Dong, Yuzhen Huang, Yvonne Liu, Zhan Shu, Dan Rabinovitsj, Ajit Mathews, Chunqiang Tang, 2026 http://arxiv.org/abs/2610.22230 2. FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness — Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra, Christopher Ré, 2022 https://scholar.google.com/scholar?q=FlashAttention%3A+Fast+and+Memory-Efficient+Exact+Attention+with+IO-Awareness 3. Deep Learning Recommendation Model for Personalization and Recommendation Systems — Maxim Naumov, Dheevatsa Mudigere, et al. (Facebook AI), 2019 https://scholar.google.com/scholar?q=Deep+Learning+Recommendation+Model+for+Personalization+and+Recommendation+Systems 4. Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Models — Dheevatsa Mudigere, Yuchen Hao, Jianyu Huang, et al., 2022 https://scholar.google.com/scholar?q=Software-Hardware+Co-design+for+Fast+and+Scalable+Training+of+Deep+Learning+Recommendation+Models 5. DeepSeek-V3 Technical Report (and the follow-up 'Insights into DeepSeek-V3: Scaling Challenges and Reflections on Hardware for AI Architectures') — DeepSeek-AI, 2024-2025 https://scholar.google.com/scholar?q=DeepSeek-V3+Technical+Report+%28and+the+follow-up+%27Insights+into+DeepSeek-V3%3A+Scaling+Challenges+and+Reflections+on+Hardware+for+AI+Architectures%27%29 6. Ansor: Generating High-Performance Tensor Programs for Deep Learning — Lianmin Zheng, Chengfan Jia, Minmin Sun, et al., 2020 https://scholar.google.com/scholar?q=Ansor%3A+Generating+High-Performance+Tensor+Programs+for+Deep+Learning 7. A Learned Performance Model for Tensor Processing Units — Samuel J. Kaufman, Phitchaya Mangpo Phothilimthana, Yanqi Zhou, et al., 2021 https://scholar.google.com/scholar?q=A+Learned+Performance+Model+for+Tensor+Processing+Units 8. Roofline: An Insightful Visual Performance Model for Multicore Architectures — Samuel Williams, Andrew Waterman, David Patterson, 2009 https://scholar.google.com/scholar?q=Roofline%3A+An+Insightful+Visual+Performance+Model+for+Multicore+Architectures 9. Stream-K: Work-centric Parallel Decomposition for Dense Matrix-Matrix Multiplication on the GPU — Muhammad Osama, Duane Merrill, Cris Cecka, Michael Garland, John D. Owens, 2023 https://scholar.google.com/scholar?q=Stream-K%3A+Work-centric+Parallel+Decomposition+for+Dense+Matrix-Matrix+Multiplication+on+the+GPU 10. The Architectural Implications of Facebook's DNN-based Personalized Recommendation — Udit Gupta, Carole-Jean Wu, Xiaodong Wang, et al., 2020 https://scholar.google.com/scholar?q=The+Architectural+Implications+of+Facebook%27s+DNN-based+Personalized+Recommendation 11. Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations — Jiaqi Zhai, Lucy Liao, Xing Liu, et al., 2024 https://scholar.google.com/scholar?q=Actions+Speak+Louder+than+Words%3A+Trillion-Parameter+Sequential+Transducers+for+Generative+Recommendations 12. KernelBench: Can LLMs Write Efficient GPU Kernels? — Anne Ouyang, Simon Guo, Simran Arora, Alex L. Zhang, William Hu, Christopher Ré, Azalia Mirhoseini, 2025 https://scholar.google.com/scholar?q=KernelBench%3A+Can+LLMs+Write+Efficient+GPU+Kernels%3F 13. The AI CUDA Engineer: Agentic CUDA Kernel Discovery, Optimization and Composition — Robert Tjarko Lange, Aaditya Prasad, Qi Sun, Maxence Faldor, Yujin Tang, David Ha, 2025 https://scholar.google.com/scholar?q=The+AI+CUDA+Engineer%3A+Agentic+CUDA+Kernel+Discovery%2C+Optimization+and+Composition 14. Kevin: Multi-Turn RL for Generating CUDA Kernels — Carlo Baronio, Pietro Marsella, Ben Pan, Simon Guo, Silas Alberti, 2025 https://scholar.google.com/scholar?q=Kevin%3A+Multi-Turn+RL+for+Generating+CUDA+Kernels 15. Triton: An Intermediate Language and Compiler for Tiled Neural Network Computations — Philippe Tillet, H. T. Kung, David Cox, 2019 https://scholar.google.com/scholar?q=Triton%3A+An+Intermediate+Language+and+Compiler+for+Tiled+Neural+Network+Computations 16. tritonBLAS: Triton-based Analytical Approach for GEMM Kernel Parameter Selection — Ryan Swann, Muhammad Osama, Xiaohu Guo et al., 2025 https://scholar.google.com/scholar?q=tritonBLAS%3A+Triton-based+Analytical+Approach+for+GEMM+Kernel+Parameter+Selection 17. KernelEvolve: Scaling Agentic Kernel Coding for Heterogeneous AI Accelerators at Meta — Gang Liao et al., 2026 https://scholar.google.com/scholar?q=KernelEvolve%3A+Scaling+Agentic+Kernel+Coding+for+Heterogeneous+AI+Accelerators+at+Meta 18. Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations (HSTU) — Jiaqi Zhai et al., 2024 https://scholar.google.com/scholar?q=Actions+Speak+Louder+than+Words%3A+Trillion-Parameter+Sequential+Transducers+for+Generative+Recommendations+%28HSTU%29 19. Training Foundation Models on a Full-Stack AMD Platform: Compute, Networking, and System Design — Quentin Anthony et al. (Zyphra/AMD), 2025 https://scholar.google.com/scholar?q=Training+Foundation+Models+on+a+Full-Stack+AMD+Platform%3A+Compute%2C+Networking%2C+and+System+Design 20. FlashAttention-3: Fast and Accurate Attention with Asynchronicity and Low-precision — Jay Shah, Ganesh Bikshandi, Ying Zhang, Vijay Thakkar, Pradeep Ramani, Tri Dao, 2024 https://scholar.google.com/scholar?q=FlashAttention-3%3A+Fast+and+Accurate+Attention+with+Asynchronicity+and+Low-precision 21. Silent Data Corruptions at Scale — Harish Dixit et al. (Meta), 2021 https://scholar.google.com/scholar?q=Silent+Data+Corruptions+at+Scale 22. Meta's Second Generation AI Chip: Model-Chip Co-Design and Productionization Experiences (MTIA) — Joel Coburn et al., 2025 https://scholar.google.com/scholar?q=Meta%27s+Second+Generation+AI+Chip%3A+Model-Chip+Co-Design+and+Productionization+Experiences+%28MTIA%29 Interactive Visualization: Meta and AMD Co-Design MI350X for Recommendation Model Training