Genuine 1-Bit Post-Training Quantization for LLMs: All for 1-Bit

This episode examines "All for 1-Bit," a paper claiming to compress LLM linear weights to exactly one bit each, metadata included, without retraining. It builds on double binary factorization and challenges earlier "1-bit" post-training methods, which land at roughly 2.5 to 3.5 effective bits per weight once scales, masks and outlier indices are counted. The discussion covers why bandwidth-bound decoding motivates extreme quantization, how a product of two binary matrices adds capacity without adding bits, and how null-space reasoning and Shapley-value layer allocation are meant to spend a fixed rank budget well. Along the way it separates post-training from quantization-aware training and sets up the key skeptical test: whether "genuine" and "usable" both hold, given that a score near chance (about 25 on the zero-shot scale) means a broken model. Listeners get a clear explanation of what "bits per weight" really measures and a framework for judging low-bit quantization claims. Sources: 1. All for 1-Bit: Towards Genuine 1-Bit Post-Training Quantization for LLMs — Zhixiong Zhao, Zukang Xu, Guangyu Sun, Lifeng Liu, Dawei Yang, 2026 http://arxiv.org/abs/2609.06161 2. BiLLM: Pushing the Limit of Post-Training Quantization for LLMs — Wei Huang, Yangdong Liu, Haotong Qin, Ying Li, Shiming Zhang, Xianglong Liu, Michele Magno, Xiaojuan Qi, 2024 https://scholar.google.com/scholar?q=BiLLM%3A+Pushing+the+Limit+of+Post-Training+Quantization+for+LLMs 3. ARB-LLM: Alternating Refined Binarizations for Large Language Models — Zhiteng Li, Xianglong Yan, Tianao Zhang, Haotong Qin, Dong Xie, Jiang Tian, Zhongchao Shi, Linghe Kong, Yulun Zhang, Xiaokang Yang, 2024 https://scholar.google.com/scholar?q=ARB-LLM%3A+Alternating+Refined+Binarizations+for+Large+Language+Models 4. OneBit: Towards Extremely Low-bit Large Language Models — Yuzhuang Xu, Xu Han, Zonghan Yang, Shuo Wang, Qingfu Zhu, Zhiyuan Liu, Weidong Liu, Wanxiang Che, 2024 https://scholar.google.com/scholar?q=OneBit%3A+Towards+Extremely+Low-bit+Large+Language+Models 5. BitNet: Scaling 1-bit Transformers for Large Language Models / The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits — Hongyu Wang, Shuming Ma, Li Dong, Shaohan Huang, Huaijie Wang, Lingxiao Ma, Fan Yang, Ruiping Wang, Yi Wu, Furu Wei (BitNet); Shuming Ma et al. (b1.58), 2023 / 2024 https://scholar.google.com/scholar?q=BitNet%3A+Scaling+1-bit+Transformers+for+Large+Language+Models+%2F+The+Era+of+1-bit+LLMs%3A+All+Large+Language+Models+are+in+1.58+Bits 6. GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — Elias Frantar, Saleh Ashkboos, Torsten Hoefler, Dan Alistarh, 2022 https://scholar.google.com/scholar?q=GPTQ%3A+Accurate+Post-Training+Quantization+for+Generative+Pre-trained+Transformers 7. SVD-LLM: Truncation-aware Singular Value Decomposition for Large Language Model Compression — Xin Wang, Yu Zheng, Zhongwei Wan, Mi Zhang, 2024 https://scholar.google.com/scholar?q=SVD-LLM%3A+Truncation-aware+Singular+Value+Decomposition+for+Large+Language+Model+Compression 8. AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models — Junfeng Fang, Houcheng Jiang, Kun Wang, Yunshan Ma, Shi Jie, Xiang Wang, Xiangnan He, Tat-Seng Chua, 2024 https://scholar.google.com/scholar?q=AlphaEdit%3A+Null-Space+Constrained+Knowledge+Editing+for+Language+Models 9. Training Networks in Null Space of Feature Covariance for Continual Learning — Shipeng Wang, Xiaorong Li, Jian Sun, Zongben Xu, 2021 https://scholar.google.com/scholar?q=Training+Networks+in+Null+Space+of+Feature+Covariance+for+Continual+Learning 10. A Unified Approach to Interpreting Model Predictions (SHAP) — Scott M. Lundberg, Su-In Lee, 2017 https://scholar.google.com/scholar?q=A+Unified+Approach+to+Interpreting+Model+Predictions+%28SHAP%29 11. Data Shapley: Equitable Valuation of Data for Machine Learning — Amirata Ghorbani, James Zou, 2019 https://scholar.google.com/scholar?q=Data+Shapley%3A+Equitable+Valuation+of+Data+for+Machine+Learning 12. HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networks — Zhen Dong, Zhewei Yao, Yaohui Cai, Daiyaan Arfeen, Amir Gholami, Michael W. Mahoney, Kurt Keutzer, 2020 https://scholar.google.com/scholar?q=HAWQ-V2%3A+Hessian+Aware+trace-Weighted+Quantization+of+Neural+Networks 13. SqueezeLLM: Dense-and-Sparse Quantization — Sehoon Kim, Coleman Hooper, Amir Gholami, Zhen Dong, Xiuyu Li, Sheng Shen, Michael W. Mahoney, Kurt Keutzer, 2023 https://scholar.google.com/scholar?q=SqueezeLLM%3A+Dense-and-Sparse+Quantization 14. Addition is almost all you need: Compressing neural networks with double binary factorization — Vladimír Boža, Vladimír Macko, 2025 https://scholar.google.com/scholar?q=Addition+is+almost+all+you+need%3A+Compressing+neural+networks+with+double+binary+factorization 15. LittleBit: Ultra Low-Bit Quantization via Latent Factorization — Lee et al., 2026 https://scholar.google.com/scholar?q=LittleBit%3A+Ultra+Low-Bit+Quantization+via+Latent+Factorization 16. The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits (BitNet b1.58) — Shuming Ma et al., 2024 https://scholar.google.com/scholar?q=The+Era+of+1-bit+LLMs%3A+All+Large+Language+Models+are+in+1.58+Bits+%28BitNet+b1.58%29 17. FBI-LLM: Scaling Up Fully Binarized LLMs from Scratch via Autoregressive Distillation — Chuang Ma et al., 2024 https://scholar.google.com/scholar?q=FBI-LLM%3A+Scaling+Up+Fully+Binarized+LLMs+from+Scratch+via+Autoregressive+Distillation 18. HBLLM: Haar-based approach for accurate structured 1-bit quantized LLMs — Chen et al., 2025 https://scholar.google.com/scholar?q=HBLLM%3A+Haar-based+approach+for+accurate+structured+1-bit+quantized+LLMs 19. AQLM: Extreme Compression of LLMs via Additive Quantization — Egiazarian et al., 2024 https://scholar.google.com/scholar?q=AQLM%3A+Extreme+Compression+of+LLMs+via+Additive+Quantization 20. QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks — Tseng et al., 2024 https://scholar.google.com/scholar?q=QuIP%23%3A+Even+Better+LLM+Quantization+with+Hadamard+Incoherence+and+Lattice+Codebooks 21. QTIP: Quantization with Trellises and Incoherence Processing — Tseng et al., 2024 https://scholar.google.com/scholar?q=QTIP%3A+Quantization+with+Trellises+and+Incoherence+Processing 22. Extreme Compression of Large Language Models via Additive Quantization / PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression — Malinovskii et al., 2024 https://scholar.google.com/scholar?q=Extreme+Compression+of+Large+Language+Models+via+Additive+Quantization+%2F+PV-Tuning%3A+Beyond+Straight-Through+Estimation+for+Extreme+LLM+Compression 23. SVD-LLM: Truncation-aware Singular Value Decomposition for LLM Compression — Xin Wang et al., 2024 https://scholar.google.com/scholar?q=SVD-LLM%3A+Truncation-aware+Singular+Value+Decomposition+for+LLM+Compression 24. BitNet a4.8 / bitnet.cpp: Efficient edge inference for ternary LLMs — Microsoft Research, 2024-2025 https://scholar.google.com/scholar?q=BitNet+a4.8+%2F+bitnet.cpp%3A+Efficient+edge+inference+for+ternary+LLMs 25. ParetoQ: Scaling Laws in Extremely Low-bit LLM Quantization — Zechun Liu et al., 2025 https://scholar.google.com/scholar?q=ParetoQ%3A+Scaling+Laws+in+Extremely+Low-bit+LLM+Quantization Interactive Visualization: Genuine 1-Bit Post-Training Quantization for LLMs: All for 1-Bit