A study on the randomness reduction effect of extreme learning machine with ridge regression

作者: Meng Joo Er , Zhifei Shao , Ning Wang

DOI: 10.1007/978-3-642-39065-4_21

关键词: Machine learningReduction (complexity)Feedforward neural networkGeneralizationRidge (differential geometry)Computer scienceRegressionPower (physics)RandomnessArtificial intelligenceExtreme learning machine

摘要: In recent years, Extreme Learning Machine (ELM) has attracted comprehensive attentions as a universal function approximator. Comparing to other single layer feedforward neural networks, its input parameters of hidden neurons can be randomly generated rather than tuned, and thereby saving huge amount computational power. However, it been pointed out that the randomness ELM would result in fluctuating performances. this paper, we intensively investigate reduction effect by using regularized version ELM, named Ridge (RELM). Previously, RELM shown achieve generally better generalization original ELM. Furthermore, try demonstrate also greatly reduce performance with 12 real world regression tasks. An insight into is given.

参考文章(17)
Guang-Bin Huang, Dian Hui Wang, Yuan Lan, Extreme learning machines: a survey International Journal of Machine Learning and Cybernetics. ,vol. 2, pp. 107- 122 ,(2011) , 10.1007/S13042-011-0019-Y
Zhan-Li Sun, Tsan-Ming Choi, Kin-Fan Au, Yong Yu, Sales forecasting using extreme learning machine with applications in fashion retailing decision support systems. ,vol. 46, pp. 411- 419 ,(2008) , 10.1016/J.DSS.2008.07.009
Yuan Lan, Yeng Chai Soh, Guang-Bin Huang, Constructive hidden nodes selection of extreme learning machine for regression Neurocomputing. ,vol. 73, pp. 3191- 3199 ,(2010) , 10.1016/J.NEUCOM.2010.05.022
Guang-Bin Huang, Hongming Zhou, Xiaojian Ding, Rui Zhang, Extreme Learning Machine for Regression and Multiclass Classification systems man and cybernetics. ,vol. 42, pp. 513- 529 ,(2012) , 10.1109/TSMCB.2011.2168604
Qin-Yu Zhu, A.K. Qin, P.N. Suganthan, Guang-Bin Huang, Rapid and brief communication: Evolutionary extreme learning machine Pattern Recognition. ,vol. 38, pp. 1759- 1763 ,(2005) , 10.1016/J.PATCOG.2005.03.028
Arthur E. Hoerl, Robert W. Kennard, Ridge regression: biased estimation for nonorthogonal problems Technometrics. ,vol. 42, pp. 80- 86 ,(2000) , 10.2307/1271436
Yuan Lan, Yeng Chai Soh, Guang-Bin Huang, Two-stage extreme learning machine for regression Neurocomputing. ,vol. 73, pp. 3028- 3038 ,(2010) , 10.1016/J.NEUCOM.2010.07.012
S. Suresh, S. Saraswathi, N. Sundararajan, Performance enhancement of extreme learning machine for multi-category sparse data classification problems Engineering Applications of Artificial Intelligence. ,vol. 23, pp. 1149- 1157 ,(2010) , 10.1016/J.ENGAPPAI.2010.06.009
Yuan Lan, Yeng Chai Soh, Guang-Bin Huang, Letters: Ensemble of online sequential extreme learning machine Neurocomputing. ,vol. 72, pp. 3391- 3395 ,(2009) , 10.1016/J.NEUCOM.2009.02.013
Guorui Feng, Guang-Bin Huang, Qingping Lin, R. Gay, Error Minimized Extreme Learning Machine With Growth of Hidden Nodes and Incremental Learning IEEE Transactions on Neural Networks. ,vol. 20, pp. 1352- 1357 ,(2009) , 10.1109/TNN.2009.2024147