Distributed Learning of Gaussian Graphical Models via Marginal Likelihoods

作者: Ami Wiesel , Zhaoshi Meng , Alfred O. Hero , Dennis L. Wei

DOI:

关键词:

摘要: We consider distributed estimation of the inverse covariance matrix, also called concentration in Gaussian graphical models. Traditional centralized often requires iterative and expensive global inference is therefore difficult large networks. In this paper, we propose a general framework for based on maximum marginal likelihood (MML) approach. Each node independently computes local estimate by maximizing defined with respect to data collected from its neighborhood. Due non-convexity MML problem, derive solving convex relaxation. The estimates are then combined into without need message-passing between neighborhoods. prove that relaxed estimator asymptotically consistent. Through numerical experiments several synthetic real-world sets, demonstrate two-hop version proposed significantly better than one-hop version, nearly closes gap many situations.

参考文章(20)
Martin J. Wainwright, Alan S. Willsky, Tommi S. Jaakkola, Tree-reweighted belief propagation algorithms and approximate ML estimation by pseudo-moment matching. international conference on artificial intelligence and statistics. ,(2003)
Nir Friedman, Daniel L. Koller, Probabilistic graphical models : principles and techniques The MIT Press. ,(2009)
James Franklin, The elements of statistical learning : data mining, inference,and prediction The Mathematical Intelligencer. ,vol. 27, pp. 83- 85 ,(2005) , 10.1007/BF02985802
S. Milgram, The Small World Problem Psychology today. ,vol. 1, pp. 60- 67 ,(1967)
Adam J. Rothman, Elizaveta Levina, Peter J. Bickel, Ji Zhu, Sparse permutation invariant covariance estimation Electronic Journal of Statistics. ,vol. 2, pp. 494- 515 ,(2008) , 10.1214/08-EJS176
Ami Wiesel, Alfred O. Hero, Distributed Covariance Estimation in Gaussian Graphical Models IEEE Transactions on Signal Processing. ,vol. 60, pp. 211- 220 ,(2012) , 10.1109/TSP.2011.2172430
Pradeep Ravikumar, Martin J. Wainwright, Garvesh Raskutti, Bin Yu, High-dimensional covariance estimation by minimizing ℓ1-penalized log-determinant divergence Electronic Journal of Statistics. ,vol. 5, pp. 935- 980 ,(2011) , 10.1214/11-EJS631
Duncan J. Watts, Steven H. Strogatz, Collective dynamics of small-world networks Nature. ,vol. 393, pp. 440- 442 ,(1998) , 10.1038/30918
Ali Jalali, Pradeep Ravikumar, Christopher C. Johnson, High-dimensional Sparse Inverse Covariance Estimation using Greedy Methods Journal of Machine Learning Research. ,vol. 22, pp. 574- 582 ,(2012)
Joachim Dahl, Lieven Vandenberghe, Vwani Roychowdhury, Covariance selection for nonchordal graphs via chordal embedding Optimization Methods & Software. ,vol. 23, pp. 501- 520 ,(2008) , 10.1080/10556780802102693