Rank minimization via online learning

作者: Raghu Meka , Prateek Jain , Constantine Caramanis , Inderjit S. Dhillon

DOI: 10.1145/1390156.1390239

关键词: Convex optimizationConstraint (information theory)Multiplicative functionKernel (linear algebra)Feature (machine learning)Online machine learningActive learning (machine learning)Mathematical optimizationMathematicsRank (computer programming)

摘要: Minimum rank problems arise frequently in machine learning applications and are notoriously difficult to solve due the non-convex nature of objective. In this paper, we present first online approach for problem minimization matrices over polyhedral sets. particular, two algorithms - our algorithm is a multiplicative update method based on generalized experts framework, while second novel application convex programming framework (Zinkevich, 2003). latter, flip role decision maker by making search constraint space instead feasible points, as usually case programming. A salient feature that it allows us give provable approximation guarantees We demonstrate effectiveness methods synthetic examples, real-life low-rank kernel learning.

参考文章(18)
Dongmin Kim, Suvrit Sra, Inderjit S. Dhillon, Fast Newton-type Methods for the Least Squares Nonnegative Matrix Approximation Problem siam international conference on data mining. pp. 343- 354 ,(2007) , 10.1137/1.9781611972771.31
Elad Hazan, Adam Kalai, Satyen Kale, Amit Agarwal, Logarithmic Regret Algorithms for Online Convex Optimization Learning Theory. ,vol. 69, pp. 499- 513 ,(2006) , 10.1007/11776420_37
Maryam Fazel, Haitham Hindi, Stephen P Boyd, None, Log-det heuristic for matrix rank minimization with applications to Hankel and Euclidean distance matrices american control conference. ,vol. 3, pp. 2156- 2162 ,(2003) , 10.1109/ACC.2003.1243393
M. Fazel, H. Hindi, S.P. Boyd, A rank minimization heuristic with application to minimum order system approximation american control conference. ,vol. 6, pp. 4734- 4739 ,(2001) , 10.1109/ACC.2001.945730
Yoav Freund, Robert E Schapire, A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting conference on learning theory. ,vol. 55, pp. 119- 139 ,(1997) , 10.1006/JCSS.1997.1504
Kilian Q. Weinberger, Fei Sha, Lawrence K. Saul, Learning a kernel matrix for nonlinear dimensionality reduction international conference on machine learning. pp. 106- ,(2004) , 10.1145/1015330.1015345
N. Littlestone, M.K. Warmuth, The weighted majority algorithm foundations of computer science. pp. 256- 261 ,(1989) , 10.1109/SFCS.1989.63487
Edoardo Amaldi, Viggo Kann, On the approximability of minimizing nonzero variables or unsatisfied relations in linear systems Theoretical Computer Science. ,vol. 209, pp. 237- 260 ,(1998) , 10.1016/S0304-3975(97)00115-1
Alexander Barvinok, A Course in Convexity ,(2002)
Francis R Bach, Michael I Jordan, None, Predictive low-rank decomposition for kernel methods Proceedings of the 22nd international conference on Machine learning - ICML '05. pp. 33- 40 ,(2005) , 10.1145/1102351.1102356