From neural networks to graphical models: a brief introduction

作者: Toshiyuki Tanaka

DOI:

关键词: Graphical modelConditional probabilityPattern recognitionBayesian networkProbabilistic neural networkInformation processingArtificial neural networkProbability distributionBasis (linear algebra)Computer scienceArtificial intelligenceTheoretical computer science

摘要: One of the most important and interesting aspects neural networks is that they exhibit collective information-processing capabilities by connecting processing elements (“neurons”), each which performs a very simple limited information processing. The central issue network research thus to explore how why such system with exhibits complex functionalities. Being looked at from another direction, studying problem, we are asking about global characterization performed network, on basis local (i.e., neuron), as well interconnected. Essentially same problem also arises in study graphical models (Bayesian networks). main objective this section give brief review models, some emphasis their relation networks. In one class an N -dimensional random vector x characterized set conditional probabilities form: p(xi|x\i), where x\i denotes all except xi. If, for i, xi independent xj , j > then defines probability distribution

参考文章(10)
Toshiyuki Tanaka, Information Geometry of Mean Field Theory IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences. ,vol. 79, pp. 709- 715 ,(1996)
Y Kabashima, D Saad, Belief propagation vs. TAP for decoding corrupted messages EPL. ,vol. 44, pp. 668- 674 ,(1998) , 10.1209/EPL/I1998-00524-7
David H. Ackley, Geoffrey E. Hinton, Terrence J. Sejnowski, A learning algorithm for boltzmann machines Cognitive Science. ,vol. 9, pp. 147- 169 ,(1985) , 10.1207/S15516709COG0901_7
R.J. McEliece, D.J.C. MacKay, Jung-Fu Cheng, Turbo decoding as an instance of Pearl's "belief propagation" algorithm IEEE Journal on Selected Areas in Communications. ,vol. 16, pp. 140- 152 ,(1998) , 10.1109/49.661103
D.J.C. MacKay, Good error-correcting codes based on very sparse matrices international symposium on information theory. ,vol. 45, pp. 399- 431 ,(1997) , 10.1109/18.748992
Shiro Ikeda, Toshiyuki Tanaka, Shun-ichi Amari, Stochastic reasoning, free energy, and information geometry Neural Computation. ,vol. 16, pp. 1779- 1810 ,(2004) , 10.1162/0899766041336477
Geoffrey E. Hinton, J. Sejnowski, OPTIMAL PERCEPTUAL INFERENCE ,(1983)
M.J. Wainwright, T.S. Jaakkola, A.S. Willsky, Tree-based reparameterization framework for analysis of sum-product and related algorithms IEEE Transactions on Information Theory. ,vol. 49, pp. 1120- 1146 ,(2003) , 10.1109/TIT.2003.810642
J.S. Yedidia, W.T. Freeman, Y. Weiss, Constructing free-energy approximations and generalized belief propagation algorithms IEEE Transactions on Information Theory. ,vol. 51, pp. 2282- 2312 ,(2005) , 10.1109/TIT.2005.850085