On the Use of Adaptive Weights for the F_infty-Norm Support Vector Machine

Abstract

When the input features are generated by factors in a classification problem, it is more meaningful to identify important factors, rather than individual features. The $F_\infty$-norm support vector machine(SVM) has been developed to perform automatic factor selection in classification. However, the $F_\infty$-norm SVM may suffer from estimation inefficiency and model selection inconsistency because it applies the same amount of shrinkage to each factor without assessing its relative importance. To overcome such a limitation, we propose the adaptive $F_\infty$-norm ($\text{AF}_\infty$-norm) SVM, which penalizes the empirical hinge loss by the sum of the adaptively weighted factor-wise $L_\infty$-norm penalty. The $\text{AF}_\infty$-norm SVM computes the weights by the 2-norm SVM estimator and can be formulated as a linear programming(LP) problem which is similar to the one of the $F_\infty$-norm SVM. The simulation studies show that the proposed $\text{AF}_\infty$-norm SVM improves upon the $F_\infty$-norm SVM in terms of classification accuracy and factor selection performance.