Learning to control fast-weight memories: an alternative to dynamic recurrent networks
DOI: 10.1162/NECO.1992.4.1.131
关键词: Net (mathematics) 、 Computer science 、 Control (management) 、 Temporal information 、 Class (computer programming) 、 Storage efficiency 、 Sequence learning 、 Machine learning 、 Temporary variable 、 Artificial intelligence 、 Feed forward
摘要: Previous algorithms for supervised sequence learning are based on dynamic recurrent networks. This paper describes an alternative class of gradient-based systems consisting two feedforward nets that learn to deal with temporal sequences using fast weights: The first net learns produce context-dependent weight changes the second whose weights may vary very quickly. method offers potential STM storage efficiency: A single (instead a full-fledged unit) be sufficient storing information. Various methods derived. Two experiments unknown time delays illustrate approach. One experiment shows how system can used adaptive temporary variable binding.
acm.org
core.ac.uk
mitpressjournals.org参考文章(8)
Jürgen Schmidhuber, Learning Algorithms for Networks with Internal and External Feedback Connectionist Models#R##N#Proceedings of the 1990 Summer School. pp. 52- 61 ,(1991) , 10.1016/B978-1-4832-1448-1.50012-3
JURGEN SCHMIDHUBER, A Local Learning Algorithm for Dynamic Feedforward and Recurrent Networks Connection Science. ,vol. 1, pp. 403- 412 ,(1989) , 10.1080/09540098908915650
RONALD J. WILLIAMS, DAVID ZIPSER, Experimental Analysis of the Real-time Recurrent Learning Algorithm Connection Science. ,vol. 1, pp. 87- 111 ,(1989) , 10.1080/09540098908915631
Jürgen Schmidhuber, A fixed size storage O(n 3 ) time complexity learning algorithm for fully recurrent continually running networks Neural Computation. ,vol. 4, pp. 243- 248 ,(1992) , 10.1162/NECO.1992.4.2.243
Barak A. Pearlmutter, Learning state space trajectories in recurrent neural networks Neural Computation. ,vol. 1, pp. 263- 269 ,(1989) , 10.1162/NECO.1989.1.2.263
D. E. Rumelhart, G. E. Hinton, R. J. Williams, Learning internal representations by error propagation Parallel distributed processing: explorations in the microstructure of cognition, vol. 1. ,vol. 1, pp. 318- 362 ,(1986)
J. Schmidhuber, Learning to generate sub-goals for action sequences Artificial Neural Networks. pp. 967- 972 ,(1991)
P. Werbos, Beyond regression : new fools for prediction and analysis in the behavioral sciences PhD thesis, Harvard University. ,(1974)