On-line Variational Bayesian Learning
作者： H. Valpola , Antti Honkela
DOI:
关键词:
摘要: Variational Bayesian learning is an approximation to the exact where true posterior approximated with a simpler distribution. In this paper we present on-line variant of variational learning. The method based on collecting likelihood information as training samples are processed one at time and decaying old information. decay or forgetting very important since otherwise system would get stuck first reasonable solution it finds. tested simple linear independent component analysis (ICA) problem but can easily be applied other more difficult problems.
aalto.fi
ntt.co.jp 
hiit.fi 参考文章(19)
H. Attias, ICA, graphical models and variational methods Independent Component Analysis. pp. 95- 112 ,(2001) , 10.1017/CBO9780511624148.004
Harri Valpola, J. Karhunen, T. Östman, Nonlinear Independent Factor Analysis by Hierarchical Models ,(2003)
W.D. Penny, S.J. Roberts, R.M. Everson, ICA: model order selection and dynamic source models Cambridge University Press. pp. 299- 314 ,(2001) , 10.1017/CBO9780511624148.013
J. W. Miskin, D.J.C. MacKay, Ensemble Learning for blind source separation Independent Component Analysis. pp. 209- 233 ,(2001) , 10.1017/CBO9780511624148.009
Zoubin Ghahramani, Matthew J Beal, None, Graphical Models and Variational Methods In: Saad, D and Opper, M, (eds.) Advanced Mean Field Methods: Theory and Practice. (pp. 161-177). MIT Press: Cambridge, MA, USA. (2001). ,(2001)
Michael I Jordan, Zoubin Ghahramani, Tommi S Jaakkola, Lawrence K Saul, None, An introduction to variational methods for graphical models Machine Learning. ,vol. 37, pp. 105- 161 ,(1999) , 10.1023/A:1007665907178
Harri Valpola, T. Raiko, J. Karhunen, Building Blocks for Hierarchical Latent Variable Models ,(2001)
David J. C. MacKay, Developments in Probabilistic Modelling with Neural Networks — Ensemble Learning SNN Symposium on Neural Networks. pp. 191- 198 ,(1995) , 10.1007/978-1-4471-3087-1_37
Geoffrey E. Hinton, Drew van Camp, Keeping the neural networks simple by minimizing the description length of the weights conference on learning theory. pp. 5- 13 ,(1993) , 10.1145/168304.168306
Harri Valpola, Markus Harva, Juha Karhunen, Hierarchical models of variance sources Signal Processing. ,vol. 84, pp. 267- 282 ,(2004) , 10.1016/J.SIGPRO.2003.10.014