Optimal bandwidth selection for re-substitution entropy estimation

作者: Yu-Lin He , James N.K. Liu , Xi-Zhao Wang , Yan-Xing Hu

DOI: 10.1016/J.AMC.2012.08.056

关键词: Entropy estimationEstimatorDifferential entropyMaximum entropy spectral estimationMaximum entropy probability distributionMathematical optimizationDensity estimationEntropy (information theory)AlgorithmMathematicsBinary entropy function

摘要: Abstract A new fusion approach of selecting an optimal bandwidth for re-substitution entropy estimator (RE) is presented in this study. When approximating the continuous with density estimation, two types errors will be generated: estimation error (type-I error) and (type-II error). These are all strongly dependent on undetermined bandwidths. Firstly, experimental conclusion based 24 typical probability distributions demonstrated that there some inconsistency between bandwidths associated these errors. Secondly, different measures type-I type-II derived. trade-off a fundamental potential property our proposed method called RE I + II . Thus, conducted solved. Finally, comparisons carried out to verify performance strategy. The discretization deemed necessary preprocessing technology calculation traditionally. So, nine mostly used unsupervised methods introduced give comparison their computational performances And, five most popular estimators approximation also plugged into comparisons: splitting data (SDE), cross-validation (CVE), m-spacing (mSE), m n -spacing (mnSE), nearest neighbor distance (NNDE). simulation studies show can obtain better among involved methods. Meanwhile, behaviors revealed comparative results. empirical analysis demonstrates more insensitive generalizable way entropy. makes it possible handy derived from given dataset.

参考文章(58)
Hung-Ju Huang, Tzu-Tsung Wong, Why Discretization Works for Naive Bayesian Classifiers international conference on machine learning. pp. 399- 406 ,(2000)
Oldrich Vasicek, A Test for Normality Based on Sample Entropy Journal of the Royal Statistical Society: Series B (Methodological). ,vol. 38, pp. 54- 59 ,(1976) , 10.1111/J.2517-6161.1976.TB01566.X
John Lafferty, Kamal Nigam, Andrew McCallum, Using Maximum Entropy for Text Classification ,(1999)
Geoffrey I. Webb, Ying Yang, Weighted proportional k-interval discretization for naive-Bayes classifiers knowledge discovery and data mining. pp. 501- 512 ,(2003) , 10.5555/1760894.1760960
Randy Kerber, ChiMerge: discretization of numeric attributes national conference on artificial intelligence. pp. 123- 128 ,(1992)
Geoffrey I. Webb, Ying Yang, Non-Disjoint Discretization for Naive-Bayes Classifiers international conference on machine learning. pp. 666- 673 ,(2002)
J. Catlett, On changing continuous attributes into ordered discrete attributes Lecture Notes in Computer Science. pp. 164- 178 ,(1991) , 10.1007/BFB0017012
Luís Torgo, João Gama, Search-Based Class Discretization european conference on machine learning. pp. 266- 273 ,(1997) , 10.1007/3-540-62858-4_91
James Dougherty, Ron Kohavi, Mehran Sahami, Supervised and Unsupervised Discretization of Continuous Features Machine Learning Proceedings 1995. pp. 194- 202 ,(1995) , 10.1016/B978-1-55860-377-6.50032-3
Chun-Nan Hsu, Hung-Ju Huang, Tzu-Tsung Wong, Implications of the Dirichlet Assumption for Discretization of Continuous Variables in Naive Bayesian Classifiers Machine Learning. ,vol. 53, pp. 235- 263 ,(2003) , 10.1023/A:1026367023636