Optimal bandwidth selection for re-substitution entropy estimation

详细信息    查看全文

• 作者：Yu-Lin He^a ; ^{csylhe@gmail.com} ; ^{fuchengrenyulin@126.com} ; James N.K. Liu^b ; ^{csnkliu@inet.polyu.edu.hk} ; Xi-Zhao Wang^a ; ^{xizhaowang@ieee.org} ; Yan-Xing Hu^b ; ^{huyanxing@gmail.com}
• 关键词：Information entropy ; Re-substitution entropy estimator ; Probability density estimation ; Optimal bandwidth ; Integrated mean square error ; Discretization
• 刊名：Applied Mathematics and Computation
• 出版年：2012
• 出版时间：15 December, 2012
• 年：2012
• 卷：219
• 期：8
• 页码：3425-3460
• 全文大小：2575 K

文摘

A new fusion approach of selecting an optimal bandwidth for re-substitution entropy estimator (RE) is presented in this study. When approximating the continuous entropy with density estimation, two types of errors will be generated: entropy estimation error (type-I error) and density estimation error (type-II error). These two errors are all strongly dependent on the undetermined bandwidths. Firstly, an experimental conclusion based on 24 typical probability distributions is demonstrated that there is some inconsistency between the optimal bandwidths associated with these two errors. Secondly, two different error measures for type-I and type-II errors are derived. A trade-off between type-I and type-II errors is a fundamental and potential property of our proposed method called . Thus, the fusion of these two errors is conducted and an optimal bandwidth for is solved. Finally, the experimental comparisons are carried out to verify the estimation performance of our proposed strategy. The discretization method is deemed to be the necessary preprocessing technology for the calculation of continuous entropy traditionally. So, the nine mostly used unsupervised discretization methods are introduced to give comparison of their computational performances with that of . And, five most popular estimators for entropy approximation are also plugged into our comparisons: splitting data estimator (SDE), cross-validation estimator (CVE), m-spacing estimator (mSE), -spacing estimator (m_nSE), and nearest neighbor distance estimator (NNDE). The simulation studies on 24 different typical density distributions show that can obtain the better estimation performance among the involved methods. Meanwhile, the estimation behaviors of different entropy estimation methods are also revealed based on the comparative results. The empirical analysis demonstrates that is more insensitive to data and a better generalizable way for the estimation of continuous entropy. makes it possible for a handy optimal bandwidth to be derived from a given dataset.