Learning Algorithms for Enclosing Points in Bregmanian Spheres

作者: Koby Crammer , Yoram Singer

DOI: 10.1007/978-3-540-45167-9_29

关键词: Optimization problemBregman divergenceAlgorithmGeneralizationOnline algorithmEuclidean distanceDivergence (statistics)Kullback–Leibler divergenceExponential familyComputer science

摘要: We discuss the problem of finding a generalized sphere that encloses points originating from single source. The contained in such are within maximal divergence center point. divergences we study known as Bregman which include special case both Euclidean distance and relative entropy. cast learning task an optimization show it results simple dual form has interesting algebraic properties. then general algorithmic framework to solve problem. Our training algorithm employs auxiliary function bounds dual’s objective can be used with broad class functions. As specific application give detailed derivation for analyze generalization ability by adopting margin-style proof techniques. also describe two schemes online algorithms when radius is set advance.

参考文章(18)
Yair Al Censor, Stavros A. Zenios, Parallel Optimization: Theory, Algorithms, and Applications ,(1997)
Katy S. Azoury, M. K. Warmuth, Relative loss bounds for on-line density estimation with the exponential family of distributions uncertainty in artificial intelligence. ,vol. 43, pp. 31- 40 ,(1999) , 10.1023/A:1010896012157
J. Kivinen, M. K. Warmuth, Relative Loss Bounds for Multidimensional Regression Problems neural information processing systems. ,vol. 45, pp. 287- 293 ,(1997) , 10.1023/A:1017938623079
Michael Collins, Robert E. Schapire, Yoram Singer, Logistic Regression, AdaBoost and Bregman Distances conference on learning theory. ,vol. 48, pp. 158- 169 ,(2000) , 10.1023/A:1013912006537
Bernhard Schölkopf, Vladimir Vapnik, Chris Burges, Extracting support data for a given task knowledge discovery and data mining. pp. 252- 257 ,(1995)
Robert P. W. Duin, David M. J. Tax, Data domain description using support vectors. the european symposium on artificial neural networks. pp. 251- 256 ,(1999)
Robert E. Schapire, Yoav Freund, Peter Bartlett, Wee Sun Lee, Boosting the margin: a new explanation for the effectiveness of voting methods Annals of Statistics. ,vol. 26, pp. 1651- 1686 ,(1998) , 10.1214/AOS/1024691352
Robert E. Schapire, Yoram Singer, Improved boosting algorithms using confidence-rated predictions conference on learning theory. ,vol. 37, pp. 80- 91 ,(1998) , 10.1145/279943.279960
A. P. Dempster, N. M. Laird, D. B. Rubin, Maximum Likelihood from Incomplete Data Via theEMAlgorithm Journal of the Royal Statistical Society: Series B (Methodological). ,vol. 39, pp. 1- 22 ,(1977) , 10.1111/J.2517-6161.1977.TB01600.X