Large Scale Incremental Learning
作者： Yue Wu , Yinpeng Chen , Lijuan Wang , Yuancheng Ye , Zicheng Liu
关键词: Linear model 、 Layer (object-oriented design) 、 Forgetting 、 Machine learning 、 Artificial intelligence 、 Scale (descriptive set theory) 、 Training set 、 Incremental learning 、 Face (geometry) 、 Deep learning 、 Missing data 、 Computer science
摘要: Modern machine learning suffers from \textit{catastrophic forgetting} when new classes incrementally. The performance dramatically degrades due to the missing data of old classes. Incremental methods have been proposed retain knowledge acquired classes, by using distilling and keeping a few exemplars However, these struggle \textbf{scale up large number classes}. We believe this is because combination two factors: (a) imbalance between (b) increasing visually similar Distinguishing an particularly challenging, training unbalanced. propose simple effective method address issue. found that last fully connected layer has strong bias towards can be corrected linear model. With parameters, our performs remarkably well on datasets: ImageNet (1000 classes) MS-Celeb-1M (10000 classes), outperforming state-of-the-art algorithms 11.1\% 13.2\% respectively.
computer.org
sci-hub.se 参考文章(26)
Michael McCloskey, Neal J. Cohen, Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem Psychology of Learning and Motivation. ,vol. 24, pp. 109- 165 ,(1989) , 10.1016/S0079-7421(08)60536-8
Karen Simonyan, Andrew Zisserman, Very Deep Convolutional Networks for Large-Scale Image Recognition computer vision and pattern recognition. ,(2014)
Geoffrey Hinton, Oriol Vinyals, Jeff Dean, Distilling the Knowledge in a Neural Network arXiv: Machine Learning. ,(2015)
T. Mensink, J. Verbeek, F. Perronnin, G. Csurka, Distance-Based Image Classification: Generalizing to New Classes at Near-Zero Cost IEEE Transactions on Pattern Analysis and Machine Intelligence. ,vol. 35, pp. 2624- 2637 ,(2013) , 10.1109/TPAMI.2013.83
, Generative Adversarial Nets neural information processing systems. ,vol. 27, pp. 2672- 2680 ,(2014) , 10.3156/JSOFT.29.5_177_2
Robi Polikar, Lalita Upda, Satish S Upda, Vasant Honavar, Learn++: an incremental learning algorithm for supervised neural networks systems man and cybernetics. ,vol. 31, pp. 497- 508 ,(2001) , 10.1109/5326.983933
Tomaso Poggio, Gert Cauwenberghs, Incremental and Decremental Support Vector Machine Learning neural information processing systems. ,vol. 13, pp. 409- 415 ,(2000)
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, Li Fei-Fei, ImageNet Large Scale Visual Recognition Challenge International Journal of Computer Vision. ,vol. 115, pp. 211- 252 ,(2015) , 10.1007/S11263-015-0816-Y
Tianjun Xiao, Jiaxing Zhang, Kuiyuan Yang, Yuxin Peng, Zheng Zhang, Error-Driven Incremental Learning in Deep Convolutional Neural Network for Large-Scale Image Classification acm multimedia. pp. 177- 186 ,(2014) , 10.1145/2647868.2654926
Ilja Kuzborskij, Francesco Orabona, Barbara Caputo, From N to N+1: Multiclass Transfer Incremental Learning computer vision and pattern recognition. pp. 3358- 3365 ,(2013) , 10.1109/CVPR.2013.431