Adaptive log-density estimation

Abstract

This study examines an adaptive log-density estimation method with an 1-type penalty. The proposed estimator is guaranteed to be a valid density in the sense that it is positive and integrates to one. The smoothness of the estimator is controlled in a data-adaptive way via 1 penalization. The advantages of the penalized log-density estimator are discussedwith an emphasis onwavelet estimators. Theoretical properties of the estimator are studied when the quality of fit is measured by theKullback-Leibler divergence (relative entropy). A nonasymptotic oracle inequality is obtained assuming a near orthogonality condition on the given dictionary. Based on the oracle inequality, selection consistency and minimax adaptivity are proved under some regularity conditions. The proposed method is implemented with a coordinate descent algorithm. Numerical illustrations based on the periodized Meyer wavelets are performed to demonstrate the finite sample performance of the proposed estimator.