Performance Guarantees for Regularized Maximum Entropy Density Estimation

摘要: We consider the problem of estimating an unknown probability distribution from samples using principle maximum entropy (maxent). To alleviate overfitting with a very large number features, we propose applying maxent relaxed constraints on expectations features. By convex duality, this turns out to be equivalent finding Gibbs minimizing regularized version empirical log loss. prove non-asymptotic bounds showing that, respect true underlying distribution, produces density estimates that are almost as good best possible. These in terms deviation feature averages relative their expectations, can bounded standard uniform-convergence techniques. In particular, leads drop quickly samples, and depend moderately or complexity also derive convergence for both sequential-update parallel-update algorithms. Finally, briefly describe experiments data relevant modeling species geographical distributions.