EfficientL 1 regularized logistic regression

作者: Pieter Abbeel , Andrew Y. Ng , Honglak Lee , Sun-In Lee

DOI:

关键词:

摘要: L1 regularized logistic regression is now a workhorse of machine learning: it widely used for many classification problems, particularly ones with features. requires solving convex optimization problem. However, standard algorithms problems do not scale well enough to handle the large datasets encountered in practical settings. In this paper, we propose an efficient algorithm regression. Our iteratively approximates objective function by quadratic approximation at current point, while maintaining constraint. each iteration, uses LARS (Least Angle Regression) solve resulting constrained theoretical results show that our guaranteed converge global optimum. experiments significantly outperforms problems. Moreover, four previously published were specifically designed

参考文章(13)
Peter McCullagh, John Ashworth Nelder, Generalized Linear Models ,(1983)
Joshua T. Goodman, Exponential priors for maximum entropy models north american chapter of the association for computational linguistics. pp. 305- 312 ,(2005)
James Theiler, Simon Perkins, Online feature selection using grafting international conference on machine learning. pp. 592- 599 ,(2003)
Michael R. Osborne, Brett Presnell, Berwin A. Turlach, On the LASSO and Its Dual Journal of Computational and Graphical Statistics. ,vol. 9, pp. 319- 337 ,(2000) , 10.2307/1390657
Andrew Y. Ng, Feature selection, L1 vs. L2 regularization, and rotational invariance Twenty-first international conference on Machine learning - ICML '04. pp. 78- ,(2004) , 10.1145/1015330.1015435
Robert Tibshirani, Trevor Hastie, Berwin A. Turlach, Bradley Efron, Jean Michel Loubes, Jean Michel Loubes, Hemant Ishwaran, Robert A. Stine, Keith Knight, Sanford Weisberg, Saharon Rosset, Saharon Rosset, Iain Johnstone, Pascal Massart, Pascal Massart, David Madigan, J. I. Zhu, Greg Ridgeway, Greg Ridgeway, Least angle regression Annals of Statistics. ,vol. 32, pp. 407- 499 ,(2004) , 10.1214/009053604000000067
C. L. Blake, UCI Repository of machine learning databases www.ics.uci.edu/〜mlearn/MLRepository.html. ,(1998)
Robert Tibshirani, Regression Shrinkage and Selection Via the Lasso Journal of the Royal Statistical Society: Series B (Methodological). ,vol. 58, pp. 267- 288 ,(1996) , 10.1111/J.2517-6161.1996.TB02080.X
V. Roth, The generalized LASSO IEEE Transactions on Neural Networks. ,vol. 15, pp. 16- 28 ,(2004) , 10.1109/TNN.2003.809398