Adaptive multi-bit progressive quantization for stable training of binary neural networks

초록

To address the growing computational and memory demands of deep neural networks. Binary Neural Networks (BNNs) have emerged as an efficient alternative to full-precision models. However, BNNs suffer from severe information loss during quantization and limited expressive capacity. In this study, we propose an assistive teacher and self-knowledge distillation for binary Quantization-Aware Training (ASBQ). This novel progressive quantization framework combines multi-bit assistive teacher models with self-knowledge distillation to stabilize BNN training. To further enhance robustness across various neural network architectures, we integrate matching structured pruning with an asymmetric Binary Weight Network (BWN) scaling factor, thereby reducing quantization errors while maintaining hardware efficiency. Unlike previous approaches, ASBQ facilitates a smooth logit-based distillation transition from a full-precision teacher to a 1-bit student, minimizing accuracy loss through adaptive knowledge transfer. Extensive experiments demonstrate that ASBQ achieves state-of-the-art accuracy of 93.1% on the CIFAR-10 dataset and delivers performance comparable to full-precision teacher models, while maintaining lightweight efficiency. The code is available at https://github.com/Luadoo/ASBQ. © The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2026.

키워드