Context-dependent representation in recurrent neural networks
作者： Gilles Wainrib
DOI:
关键词:
摘要: In order to assess the short-term memory performance of non-linear random neural networks, we introduce a measure quantify dependence representation upon past context. We study this both numerically and theoretically using mean-field theory for showing existence an optimal level synaptic weights heterogeneity. further investigate influence network topology, in particular symmetry reciprocal connections, on context dependence, revealing importance considering interplay between non-linearities connectivity structure.
harvard.edu
arxiv.org参考文章(37)
B. Cessac, M. Samuelides, From Neuron to Neural Networks dynamics European Physical Journal-special Topics. ,vol. 142, pp. 7- 88 ,(2007) , 10.1140/EPJST/E2007-00058-2
Olivier Faugeras, James Maclaurin, Asymptotic description of stochastic neural networks. I. Existence of a large deviation principle Comptes Rendus Mathematique. ,vol. 352, pp. 841- 846 ,(2014) , 10.1016/J.CRMA.2014.08.018
Olivier Faugeras, James MacLaurin, A large deviation principle for networks of rate neurons with correlated synaptic weights BMC Neuroscience. ,vol. 14, pp. 252- ,(2013) , 10.1186/1471-2202-14-S1-P252
Albert Compte, Nicolas Brunel, Patricia S Goldman-Rakic, Xiao-Jing Wang, Synaptic Mechanisms and Network Dynamics Underlying Spatial Working Memory in a Cortical Network Model Cerebral Cortex. ,vol. 10, pp. 910- 923 ,(2000) , 10.1093/CERCOR/10.9.910
Olivia L. White, Daniel D. Lee, Haim Sompolinsky, Short-term memory in orthogonal neural networks. Physical Review Letters. ,vol. 92, pp. 148102- 148300 ,(2004) , 10.1103/PHYSREVLETT.92.148102
Hugh R. Wilson, Jack D. Cowan, Excitatory and Inhibitory Interactions in Localized Populations of Model Neurons Biophysical Journal. ,vol. 12, pp. 1- 24 ,(1972) , 10.1016/S0006-3495(72)86068-5
Michiel Hermans, Benjamin Schrauwen, Memory in reservoirs for high dimensional input international joint conference on neural network. pp. 1- 7 ,(2010) , 10.1109/IJCNN.2010.5596884
H. Sompolinsky, A. Crisanti, H. J. Sommers, Chaos in random neural networks. Physical Review Letters. ,vol. 61, pp. 259- 262 ,(1988) , 10.1103/PHYSREVLETT.61.259
Mustafa C. Ozturk, Dongming Xu, José C. Príncipe, Analysis and design of echo state networks Neural Computation. ,vol. 19, pp. 111- 138 ,(2007) , 10.1162/NECO.2007.19.1.111
Qingsong Song, Zuren Feng, Effects of connectivity structure of complex echo state network on its prediction performance for nonlinear time series Neurocomputing. ,vol. 73, pp. 2177- 2185 ,(2010) , 10.1016/J.NEUCOM.2010.01.015