Scalable Spectral k-Support Norm Regularization for Robust Low Rank Subspace Learning

作者: Yiu-ming Cheung , Jian Lou

DOI: 10.1145/2983323.2983738

关键词: Mathematical optimizationMatrix normNorm (mathematics)Convex relaxationComputer scienceLow-rank approximationScalabilitySubspace topologySparse matrixRegularization (mathematics)Computation

摘要: As a fundamental tool in the fields of data mining and computer vision, robust low rank subspace learning is to recover matrix under gross corruptions that are often modeled by another sparse matrix. Within this learning, we investigate spectral k-support norm, more appealing convex relaxation than popular nuclear as penalty paper. Despite better recovering performance, norm entails model difficult be optimized efficiently, which severely limits its scalability from practical perspective. Therefore, paper proposes scalable efficient algorithm considers dual objective original problem can take advantage computational linear oracle evaluated. Further, studying sub-gradient loss objective, line-search strategy adopted enable it adapt Holder smoothness. Experiments on various tasks demonstrate superior prediction performance computation efficiency proposed algorithm.

参考文章(33)
Ming Yuan, Yi Lin, Model selection and estimation in regression with grouped variables Journal of The Royal Statistical Society Series B-statistical Methodology. ,vol. 68, pp. 49- 67 ,(2006) , 10.1111/J.1467-9868.2005.00532.X
Emmanuel J. Candès, Xiaodong Li, Yi Ma, John Wright, Robust principal component analysis? Journal of the ACM. ,vol. 58, pp. 1- 37 ,(2011) , 10.1145/1970392.1970395
Rasmus Munk Larsen, Lanczos Bidiagonalization With Partial Reorthogonalization DAIMI Report Series. ,vol. 27, ,(1998) , 10.7146/DPB.V27I537.7070
Aleksandr Y. Aravkin, Dmitriy Drusvyatskiy, Michael P. Friedlander, James V. Burke, Scott Roy, Level-set methods for convex optimization arXiv: Optimization and Control. ,(2016)
Risheng Liu, Zhixun Su, Zhouchen Lin, Linearized Alternating Direction Method with Adaptive Penalty for Low-Rank Representation arXiv: Optimization and Control. ,(2011)
Andrew M. McDonald, Dimitris Stamos, Massimiliano Pontil, Fitting Spectral Decay with the $k$-Support Norm international conference on artificial intelligence and statistics. pp. 1061- 1069 ,(2016)
Quanming Yao, James T. Kwok, Wenliang Zhong, Fast Low-Rank Matrix Learning with Nonconvex Regularization 2015 IEEE International Conference on Data Mining. pp. 539- 548 ,(2015) , 10.1109/ICDM.2015.9
Cun-Hui Zhang, Nearly unbiased variable selection under minimax concave penalty Annals of Statistics. ,vol. 38, pp. 894- 942 ,(2010) , 10.1214/09-AOS729
Martin Jaggi, Revisiting Frank-Wolfe: Projection-Free Sparse Convex Optimization international conference on machine learning. pp. 427- 435 ,(2013)
Michael P. Friedlander, Kevin Murphy, Mark Schmidt, Ewout Van Den Berg, GROUP SPARSITY VIA LINEAR-TIME PROJECTION ,(2008)