Fast Laplace approximation for sparse Bayesian spike and slab models

摘要: We consider the application of Bayesian spike-and-slab models in high-dimensional feature selection problems. To do so, we propose a simple yet effective fast approximate inference algorithm based on Laplace's method. exploit two efficient optimization methods, GIST [Gong et al., 2013] and L-BFGS [Nocedal, 1980], to obtain mode posterior distribution. Then an ensemble Nystrom approach calculate diagonal inverse Hessian over marginals O(knp) time, k ≪ p. Furthermore, provide theoretical analysis about estimation consistency approximation error bounds. With model weights, use quadrature integration estimate marginal posteriors probabilities indicator variables for all features, which quantify uncertainty. Our method not only maintains benefits treatment (e.g., uncertainty quantification) but also possesses computational efficiency, oracle properties frequentist methods. Simulation shows that our estimates better or comparable than alternative methods such as VB EP, with less running time. Extensive experiments large real datasets demonstrate often improves prediction accuracy automatic relevance determination, L1 type