Discussion: Local Rademacher complexities and oracle inequalities in risk minimization

摘要: In this magnificent paper, Professor Koltchinskii offers general and powerful performance bounds for empirical risk minimization, a fundamental principle of statistical learning theory. Since the elegant pioneering work Vapnik Chervonenkis in early 1970s, various such have been known that relate minimizers to combinatorial geometrical features class over which minimization is performed. This area research has rich source motivation major field applications process The appearance advanced concentration inequalities 1990s, primarily thanks Talagrand’s influential work, provoked advances both theory led much deeper understanding some basic phenomena. discussed paper develops new methodology, iterative localization, which, with help inequalities, able explain most recent results go significantly beyond them many cases. main behind Koltchinskii’s based on classical problems as binary classification regression given sample (Xi ,Y i), i = 1 ,...,n , independent identically distributed pairs random variables (where Xi take their values feature space X Yi are, say, real-valued), goal find function f : → R whose risk, defined terms expected value an appropriately chosen loss function, small possible. remaining part discussion we point out how can be used study seemingly different model, motivated by nonparametric ranking problems, received increasing attention machine literature. Indeed, several applications, search engine problem or credit screening, learn rank—or score—observations rather than just classify them. case, measures involve observations, seen, instance, AUC (Area Under ROC Curve) criterion.