Modeling pixel means and covariances using factorized third-order boltzmann machines

作者: Marc'Aurelio Ranzato , Geoffrey E. Hinton

DOI: 10.1109/CVPR.2010.5539962

关键词: Feature extractionGenerative modelPixelArtificial intelligenceCovarianceStatistical modelMathematicsBoltzmann machinePattern recognitionCovariance matrixProbabilistic logic

摘要: Learning a generative model of natural images is useful way extracting features that capture interesting regularities. Previous work on learning such models has focused methods in which the latent are used to determine mean and variance each pixel independently, or hidden units covariance matrix zero-mean Gaussian distribution. In this work, we propose probabilistic combines these two approaches into single framework. We represent image using one set binary image-specific separate mean. show approach provides framework for widely simple-cell complex-cell architecture, it produces very realistic samples extracts yield state-of-the-art recognition accuracy challenging CIFAR 10 dataset.

参考文章(34)
Aude Oliva, Antonio Torralba, Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope International Journal of Computer Vision. ,vol. 42, pp. 145- 175 ,(2001) , 10.1023/A:1011139631724
Geoffrey Hinton, Radford M. Neal, Bayesian learning for neural networks ,(1995)
Terrence J. Sejnowski, Higher‐order Boltzmann machines AIP Conference Proceedings 151, Neural Networks for Computimg. ,vol. 151, pp. 398- 403 ,(2008) , 10.1063/1.36246
Yan Karklin, Michael S. Lewicki, Emergence of complex cell properties by learning to generalize in natural scenes Nature. ,vol. 457, pp. 83- 86 ,(2009) , 10.1038/NATURE07481
Pascal Vincent, Hugo Larochelle, Yoshua Bengio, Pierre-Antoine Manzagol, Extracting and composing robust features with denoising autoencoders Proceedings of the 25th international conference on Machine learning - ICML '08. pp. 1096- 1103 ,(2008) , 10.1145/1390156.1390294
Geoffrey E Hinton, Ruslan R Salakhutdinov, Reducing the Dimensionality of Data with Neural Networks Science. ,vol. 313, pp. 504- 507 ,(2006) , 10.1126/SCIENCE.1127647
Bruno A. Olshausen, David J. Field, Sparse Coding with an Overcomplete Basis Set: A Strategy Employed by V1 ? Vision Research. ,vol. 37, pp. 3311- 3325 ,(1997) , 10.1016/S0042-6989(97)00169-7
Geoffrey Hinton, Simon Osindero, Max Welling, Yee-Whye Teh, Unsupervised Discovery of Nonlinear Structure Using Contrastive Backpropagation Cognitive Science. ,vol. 30, pp. 725- 731 ,(2006) , 10.1207/S15516709COG0000_76
Nicolas Heess, Christopher K.I. Williams, Geoffrey E. Hinton, Learning generative texture models with extended Fields-of-Experts british machine vision conference. pp. 1- 11 ,(2009) , 10.5244/C.23.115
Urs Köster, Aapo Hyvärinen, A Two-Layer ICA-Like Model Estimated by Score Matching Lecture Notes in Computer Science. pp. 798- 807 ,(2007) , 10.1007/978-3-540-74695-9_82