Accuracy-informed label smoothing and logit scaling for deep neural network calibration

초록

Addressing miscalibration in deep learning models is crucial for reliable confidence in safety-critical applica tions. Existing calibration methods often rely on smoothing-based techniques (e.g., label smoothing, Mixup, and their variants). However, they typically overlook the model's current accuracy and offer no principled way to determine the proper degree of smoothing. Thus, they often require costly hyperparameter searches or additional post-hoc retraining. We propose an accuracy-informed label smoothing and logit scaling (ALiS), which uses an accuracy-informed aligned confidence distribution to adaptively determine smoothing levels and stabilizes train ing via a single learnable logit-scaling parameter. Experimental results on the CIFAR-10, CIFAR-100, SVHN, and Tiny-ImageNet datasets demonstrate the effectiveness of the proposed method. Additionally, ALiS outperforms conventional calibration methods by integrating diverse data augmentations when faced with data distribution shifts.

키워드