An error-entropy minimization algorithm for supervised training of nonlinear adaptive systems

作者: D. Erdogmus , J.C. Principe

DOI: 10.1109/TSP.2002.1011217

关键词:

摘要: The paper investigates error-entropy-minimization in adaptive systems training. We prove the equivalence between minimization of error's Renyi (1970) entropy order /spl alpha/ and a Csiszar (1981) distance measure densities desired system outputs. A nonparametric estimator for Renyi's is presented, it shown that global minimum this same as actual entropy. performance criterion compared with mean-square-error-minimization short-term prediction chaotic time series nonlinear identification.

参考文章(22)
J. C. Príncipe, W. Curt Lefebvre, Neil R. Euliano, Neural and adaptive systems : fundamentals through simulations Wiley. ,(2000)
David E. Rumelhart, Geoffrey E. Hinton, Ronald J. Williams, Learning representations by back-propagating errors Nature. ,vol. 323, pp. 696- 699 ,(1988) , 10.1038/323533A0
Deniz Erdogmus, Jose C. Principe, Information Theoretic Learning Information Theoretic Learning by Jose C. Principe. pp. 902- 909 ,(2010) , 10.4018/978-1-59904-849-9.CH133
Jose C. Principe, Dongxin Xu, Qun Zhao, John W. Fisher III, Learning from Examples with Information Theoretic Criteria signal processing systems. ,vol. 26, pp. 61- 77 ,(2000) , 10.1023/A:1008143417156
S. Haykin, J. Principe, Making sense of a complex world [chaotic events modeling] IEEE Signal Processing Magazine. ,vol. 15, pp. 66- 81 ,(1998) , 10.1109/79.671132
CE Shennon, Warren Weaver, A mathematical theory of communication Bell System Technical Journal. ,vol. 27, pp. 379- 423 ,(1948) , 10.1002/J.1538-7305.1948.TB01338.X