Adaptive Mixed-Precision Computing Speeds Up Nearest Neighbor Search

This episode examines ANNS-AMP, a June 2026 paper proposing adaptive mixed-precision computing for approximate nearest neighbor search. It starts with the IVF-PQ index pipeline: cluster locating, residual calculation, lookup-table construction, distance summation and top-k selection. The paper reports that over 99.9% of distance arithmetic in the cluster-locating and table-construction stages goes to vectors that are not neighbors. The proposed fix is a small runtime predictor that picks the bit-width for each cluster and each query, paired with a bit-serial, bit-plane near-memory accelerator so that fewer bits mean less compute and less memory traffic. The discussion also covers why existing index tuning and early-termination methods miss this redundancy. It questions whether flexible-precision hardware can succeed now that fixed INT8, FP8 and FP4 formats have taken over in neural networks. A running thread asks how much of the reported speed and energy gain comes from the precision idea and how much from the custom silicon. Sources: 1. ANNS-AMP: Accelerating Approximate Nearest Neighbor Search via Adaptive Mixed-Precision Computing — Mingkai Chen, Cheng Liu, Shengwen Liang, Lei Zhang, Xiaowei Li, Huawei Li, 2026 http://arxiv.org/abs/2606.07156 2. Mixed Precision Training — Paulius Micikevicius, Sharan Narang, Jonah Alben, Gregory Diamos, Erich Elsen, et al., 2018 https://scholar.google.com/scholar?q=Mixed+Precision+Training 3. HAQ: Hardware-Aware Automated Quantization with Mixed Precision — Kuan Wang, Zhijian Liu, Yujun Lin, Ji Lin, Song Han, 2019 https://scholar.google.com/scholar?q=HAQ%3A+Hardware-Aware+Automated+Quantization+with+Mixed+Precision 4. HAWQ: Hessian Aware Quantization of Neural Networks with Mixed-Precision — Zhen Dong, Zhewei Yao, Amir Gholami, Michael Mahoney, Kurt Keutzer, 2019 https://scholar.google.com/scholar?q=HAWQ%3A+Hessian+Aware+Quantization+of+Neural+Networks+with+Mixed-Precision 5. Stripes: Bit-Serial Deep Neural Network Computing — Patrick Judd, Jorge Albericio, Tayler Hetherington, Tor Aamodt, Andreas Moshovos, 2016 https://scholar.google.com/scholar?q=Stripes%3A+Bit-Serial+Deep+Neural+Network+Computing 6. Bit Fusion: Bit-Level Dynamically Composable Architecture for Accelerating Deep Neural Networks — Hardik Sharma, Jongse Park, Naveen Suda, Liangzhen Lai, Benson Chau, Vikas Chandra, Hadi Esmaeilzadeh, 2018 https://scholar.google.com/scholar?q=Bit+Fusion%3A+Bit-Level+Dynamically+Composable+Architecture+for+Accelerating+Deep+Neural+Networks 7. BISMO: A Scalable Bit-Serial Matrix Multiplication Overlay for Reconfigurable Computing — Yaman Umuroglu, Lahiru Rasnayake, Magnus Sjalander, 2018 https://scholar.google.com/scholar?q=BISMO%3A+A+Scalable+Bit-Serial+Matrix+Multiplication+Overlay+for+Reconfigurable+Computing 8. Product Quantization for Nearest Neighbor Search — Hervé Jégou, Matthijs Douze, Cordelia Schmid, 2011 https://scholar.google.com/scholar?q=Product+Quantization+for+Nearest+Neighbor+Search 9. Billion-scale similarity search with GPUs — Jeff Johnson, Matthijs Douze, Hervé Jégou, 2017 (IEEE Transactions on Big Data, 2019) https://scholar.google.com/scholar?q=Billion-scale+similarity+search+with+GPUs 10. Accelerating Large-Scale Inference with Anisotropic Vector Quantization — Ruiqi Guo, Philip Sun, Erik Lindgren, Quan Geng, David Simcha, Felix Chern, Sanjiv Kumar, 2020 https://scholar.google.com/scholar?q=Accelerating+Large-Scale+Inference+with+Anisotropic+Vector+Quantization 11. Co-design Hardware and Algorithm for Vector Search (FANNS) — Wenqi Jiang, Shigang Li, Yu Zhu, Johannes de Fine Licht, Zhenhao He, Runbin Shi, Cédric Renggli, Shuai Zhang, Theodoros Rekatsas, Torsten Hoefler, Gustavo Alonso, 2023 https://scholar.google.com/scholar?q=Co-design+Hardware+and+Algorithm+for+Vector+Search+%28FANNS%29 12. ANSMET: Approximate Nearest Neighbor Search with Near-Memory Processing and Hybrid Early Termination — Yiwei Li, Yuxin Jin, Boyu Tian, Huanchen Zhang, Mingyu Gao, 2025 https://scholar.google.com/scholar?q=ANSMET%3A+Approximate+Nearest+Neighbor+Search+with+Near-Memory+Processing+and+Hybrid+Early+Termination 13. ANNA: Specialized Architecture for Approximate Nearest Neighbor Search — Yejin Lee, Hyunji Choi, Sunhong Min, Hyunseung Lee, Sangwon Beak, Dawoon Jeong, Jae W. Lee, Tae Jun Ham, 2022 https://scholar.google.com/scholar?q=ANNA%3A+Specialized+Architecture+for+Approximate+Nearest+Neighbor+Search 14. JUNO: Optimizing High-Dimensional Approximate Nearest Neighbour Search with Sparsity-Aware Algorithm and Ray-Tracing Core Mapping — Zihan Liu, Wentao Ni, Jingwen Leng, Yu Feng, Cong Guo, Quan Chen, Chao Li, Minyi Guo, Yuhao Zhu, 2024 https://scholar.google.com/scholar?q=JUNO%3A+Optimizing+High-Dimensional+Approximate+Nearest+Neighbour+Search+with+Sparsity-Aware+Algorithm+and+Ray-Tracing+Core+Mapping 15. RaBitQ: Quantizing High-Dimensional Vectors with a Theoretical Error Bound for Approximate Nearest Neighbor Search — Jianyang Gao, Cheng Long, 2024 https://scholar.google.com/scholar?q=RaBitQ%3A+Quantizing+High-Dimensional+Vectors+with+a+Theoretical+Error+Bound+for+Approximate+Nearest+Neighbor+Search 16. High-Dimensional Approximate Nearest Neighbor Search: with Reliable and Efficient Distance Comparison Operations (ADSampling) — Jianyang Gao, Cheng Long, 2023 https://scholar.google.com/scholar?q=High-Dimensional+Approximate+Nearest+Neighbor+Search%3A+with+Reliable+and+Efficient+Distance+Comparison+Operations+%28ADSampling%29 17. Quicker ADC: Unlocking the Hidden Potential of Product Quantization with SIMD (and Fast Scan variants in Faiss) — Fabien André, Anne-Marie Kermarrec, Nicolas Le Scouarnec, 2019 https://scholar.google.com/scholar?q=Quicker+ADC%3A+Unlocking+the+Hidden+Potential+of+Product+Quantization+with+SIMD+%28and+Fast+Scan+variants+in+Faiss%29 Interactive Visualization: Adaptive Mixed-Precision Computing Speeds Up Nearest Neighbor Search