Graphical Models, Exponential Families, and Variational Inference

摘要: The formalism of probabilistic graphical models provides a unifying framework for capturing complex dependencies among random variables, and building large-scale multivariate statistical models. Graphical have become focus research in many statistical, computational mathematical fields, including bioinformatics, communication theory, physics, combinatorial optimization, signal image processing, information retrieval machine learning. Many problems that arise specific instances — the key computing marginals modes probability distributions are best studied general setting. Working with exponential family representations, exploiting conjugate duality between cumulant function entropy families, we develop variational representations likelihoods, marginal probabilities most probable configurations. We describe how wide variety algorithms them sum-product, cluster methods, expectation-propagation, mean field max-product linear programming relaxation, as well conic relaxations can all be understood terms exact or approximate forms these representations. approach complementary alternative to Markov chain Monte Carlo source approximation methods inference