RaBitQ-SSD: Split Codes and Pipelined I/O for SSD Vector Search

This episode examines RaBitQ-SSD, a system for approximate nearest neighbor search over vectors stored on SSD, which the paper says can serve ten billion vectors on one drive at over 3,000 queries per second. It starts with the basics: why billion-scale embeddings are kept on SSD instead of DRAM, how recall at k is measured, how IVF partitions vectors into inverted lists, and why graph indexes such as DiskANN and PipeANN pay a disk read for every uncached hop. It then covers how RaBitQ's one-bit-per-dimension codes give each candidate a distance estimate with a probabilistic lower bound. That bound lets the system skip individual disk reads, and the paper reports about 38 pages read per query against roughly 870 for SPANN on Wikipedia embeddings. The hosts weigh the headline claims against a simple throughput model: once the SSD saturates at its measured 1.63 million reads per second, throughput depends on pages read per query. They also explain how the Split-RaBitQ design keeps only a prefix of each binary code in memory, rescaled so the estimate stays unbiased. Listeners get a clear account of the memory and blocking-I/O problems that make a direct port to SSD impractical. Sources: 1. RaBitQ-SSD: Split Codes and Pipelined I/O for SSD-Resident Vector Search — Yuexuan Xu, Jianyang Gao, Michael Norris, Alibek Zhakubayev, Pankaj Singh, Junjie Qi, Matthijs Douze, Cheng Long, 2026 http://arxiv.org/abs/2610.02652 2. RaBitQ: Quantizing High-Dimensional Vectors with a Theoretical Error Bound for Approximate Nearest Neighbor Search — Jianyang Gao, Cheng Long, 2024 https://scholar.google.com/scholar?q=RaBitQ%3A+Quantizing+High-Dimensional+Vectors+with+a+Theoretical+Error+Bound+for+Approximate+Nearest+Neighbor+Search 3. Practical and Asymptotically Optimal Quantization of High-Dimensional Vectors in Euclidean Space for Approximate Nearest Neighbor Search (Extended RaBitQ) — Jianyang Gao, Yutong Gou, Yuexuan Xu, Yongyi Yang, Cheng Long, Raymond Chi-Wing Wong, 2025 https://scholar.google.com/scholar?q=Practical+and+Asymptotically+Optimal+Quantization+of+High-Dimensional+Vectors+in+Euclidean+Space+for+Approximate+Nearest+Neighbor+Search+%28Extended+RaBitQ%29 4. DiskANN: Fast Accurate Billion-point Nearest Neighbor Search on a Single Node — Suhas Jayaram Subramanya, Fnu Devvrit, Harsha Vardhan Simhadri, Ravishankar Krishnaswamy, Rohan Kadekodi, 2019 https://scholar.google.com/scholar?q=DiskANN%3A+Fast+Accurate+Billion-point+Nearest+Neighbor+Search+on+a+Single+Node 5. SPANN: Highly-efficient Billion-scale Approximate Nearest Neighbor Search — Qi Chen, Bing Zhao, Haidong Wang, Mingqin Li, Chuanjie Liu, Zengzhong Li, Mao Yang, Jingdong Wang, 2021 https://scholar.google.com/scholar?q=SPANN%3A+Highly-efficient+Billion-scale+Approximate+Nearest+Neighbor+Search 6. PipeANN: Low-Latency and High-Throughput Billion-Scale Vector Search via Pipelined I/O and Compute (title as I recall it; verify) — Not confirmed from memory; verify against the citation in the paper (reference [22]), 2025 https://scholar.google.com/scholar?q=PipeANN%3A+Low-Latency+and+High-Throughput+Billion-Scale+Vector+Search+via+Pipelined+I%2FO+and+Compute+%28title+as+I+recall+it%3B+verify%29 7. PipeANN: Low-Latency Billion-Scale Vector Search via Speculative I/O Pipelining — Hao Guo, Youyou Lu, 2025 https://scholar.google.com/scholar?q=PipeANN%3A+Low-Latency+Billion-Scale+Vector+Search+via+Speculative+I%2FO+Pipelining 8. Starling: An I/O-Efficient Disk-Resident Graph Index Framework for High-Dimensional Vector Similarity Search on Data Segment — Mengzhao Wang et al., 2024 https://scholar.google.com/scholar?q=Starling%3A+An+I%2FO-Efficient+Disk-Resident+Graph+Index+Framework+for+High-Dimensional+Vector+Similarity+Search+on+Data+Segment 9. Product Quantization for Nearest Neighbor Search / Quicker ADC (FastScan) — Hervé Jégou, Matthijs Douze, Cordelia Schmid; André et al., 2011 / 2015 https://scholar.google.com/scholar?q=Product+Quantization+for+Nearest+Neighbor+Search+%2F+Quicker+ADC+%28FastScan%29 10. HELMSMAN (learned list selection with an SPDK I/O path for IVF on SSD) — As cited in the paper, 2025–2026 https://scholar.google.com/scholar?q=HELMSMAN+%28learned+list+selection+with+an+SPDK+I%2FO+path+for+IVF+on+SSD%29 11. AlayaLaser: Efficient Index Layout and Search Strategy for Large-scale High-dimensional Vector Similarity Search — As cited in the paper (ref [10]), 2025 https://scholar.google.com/scholar?q=AlayaLaser%3A+Efficient+Index+Layout+and+Search+Strategy+for+Large-scale+High-dimensional+Vector+Similarity+Search 12. VeloANN and SkipDisk — As cited in the paper (refs [73], [52]), 2025–2026 https://scholar.google.com/scholar?q=VeloANN+and+SkipDisk 13. Accelerating Large-Scale Inference with Anisotropic Vector Quantization (ScaNN) — Ruiqi Guo et al., 2020 https://scholar.google.com/scholar?q=Accelerating+Large-Scale+Inference+with+Anisotropic+Vector+Quantization+%28ScaNN%29 14. Filtered-DiskANN / FreshDiskANN — Gollapudi et al.; Singh et al., 2023 / 2021 https://scholar.google.com/scholar?q=Filtered-DiskANN+%2F+FreshDiskANN Interactive Visualization: RaBitQ-SSD: Split Codes and Pipelined I/O for SSD Vector Search