基于粒子群优化算法的最大相关最小冗余混合式特征选择方法
|
摘要
在分析粒子群优化(PSO)算法和简化PSO算法的基础上,提出一种基于PSO的最大相关最小冗余的Filter-Wrapper混合式特征选择方法.Filter模型是基于互信息和特征的相关冗余综合测度,Wrapper模型是基于改进的简化粒子群算法.在PSO搜索过程中,引入相关冗余度量标准来选择特征子集,将Filter融合在Wrapper中,利用Filter的高效率和Wrapper的高精度提高搜索的速度和性能.最后以支持向量机(SVM)为分类器,在公共数据集UCI上进行实验,实验结果表明了所提出算法的可行性和有效性.
A Filter-Wrapper hybrid feature selection approach with maximum relevance and minimum redundancy based on particle swarm optimization(PSO) algorithm is proposed on the analysis of PSO algorithm and simplified PSO algorithm. The Filter is based on mutual information and the composite measure of feature relevance and redundancy,while the Wrapper is based on a simply modified PSO algorithm.The relevance and redundancy criterion is introduced to select features in the PSO's searching procedure.Meantime,the Filter is fused into the Wrapper.The speed and performance of the search are improved with the higher efficiency of the Filter and the greater accuracy of the Wrapper.The experiment results based on UCI data sets with support vector machine(SVM) as the classifier show the effectiveness and feasibility of the algorithm.
A Filter-Wrapper hybrid feature selection approach with maximum relevance and minimum redundancy based on particle swarm optimization(PSO) algorithm is proposed on the analysis of PSO algorithm and simplified PSO algorithm. The Filter is based on mutual information and the composite measure of feature relevance and redundancy,while the Wrapper is based on a simply modified PSO algorithm.The relevance and redundancy criterion is introduced to select features in the PSO's searching procedure.Meantime,the Filter is fused into the Wrapper.The speed and performance of the search are improved with the higher efficiency of the Filter and the greater accuracy of the Wrapper.The experiment results based on UCI data sets with support vector machine(SVM) as the classifier show the effectiveness and feasibility of the algorithm.
引文
[1]Yu L,Liu H.Efficient feature selection via analysis of relevance and redundancy[J].J of Machine Learning Research,2004,5(1):1205-1224.
[2]Dash M,Liu H.Feature selection for classification[J]. Intelligent Data Analysis,1997,1(2):131-156.
[3]Liu Jihong,Wang Guoxiong.A hybrid Feature Selection Method for Data Sets of thousands of variables[C].The 2nd Int Conf on Advanced Computer Control.Shenyang,2010, 2:288-291.
[4]Sarojini Llango B,Ramaraj N.A hybrid prediction model with F-score feature selection for type II diabetes databases[C].Proc of the 1st Amrita ACM-W Celebration on Women in Computing.India:ACM,2010:1-4.
[5]刘杰,金弟,杜慧君,等.一种新的混合特征选择方法RRK[J].吉林大学学报,2009,39(2):419-423. (Liu J,Jin D,Du H J,et al.New hybrid feature selection method[J].J of Jilin University,2009,39(2):419-423.)
[6]毛俐旻,姚淑萍,胡昌振.一种新型混合特征选择方法及其在入侵检测中的应用[J].北京理工大学学报,2008, 28(3):218-221. (Mao L M,Yao S P,Hu C Z.A new hybrid attribute selection method and its application in intrusion detection[JJ.Trans of Beijing Institute of Technology, 2008,28(3):218-221.)
[7]陈鑫,梁海洁,廖腾峰.基于TSVM分类器和混合型特征选择方法的入侵检测研究[J].微电子学与计算机, 2010,27(8):242-244. (Chen X,Liang H J,Liao T F.Intrusion detection based on TSVM and feature selection[J].Microelectronics and Computer,2010,27(8):242-244.)
[8]Kennedy J,Eberhart R.Particle swarm optimization[C]. Proc of IEEE Int Conf on Neural Networks.Perth:IEEE Press,1995:1942-1948.
[9]HU Wang,LI Zhi-shu.A simpler and more effective particle swarm optimization algorithm[J].J of Software, 2007,18(4):861-868.
[10]John G H,Kohavi R,Pfleger K.Irrelevant feature and the subset selection problem[C].Proc of the 11th Int Conf on Machine Learning.New Jersey:Morgan Kaufmann Publishers,1994:121-129.
[11]Bonnlander B V,Weigend A S.Selecting input variables using mutual information and nonparametric density evaluation[C].Proc of the 1994 Int Symposium on Artificial Neural Networks.Tainan,1994:42-50.
[12]Kwak N,Choi C H.Input feature selection by mutual information based on Parzen window[J].IEEE Trans on Pattern Analysis and Machine Intelligence,2002,24(12): 1667-1671.
[13]Hanchuan Peng,Fuhui Long,Chris Ding.Feature selection based on mutual information:Criteria of max-dependency, max-relevance,and min-redundancy[J].IEEE Trans on Pattern Analysis and Machine Intelligence,2005,27(8): 1226-1238.
[14]Estevez P A,Michel Tesmer,Claudio A Perez,et al. Normalized mutual information feature selection[J].IEEE Trans on Neural Networks,2009,20(2):189-201.
[15]La The Vinh,Sungyoung Lee,Young-Tack Park,et al.A novel feature selection method based on normalized mutual information[J].Applied Intelligence,2012,37(1):100- 120.
[16]Pedrycz W.Identifying core sets of discriminatory features using particle swarm optimization[J].Expert Systems with Applications,2009,36(5):4610-4616.
[17]Unler A,Murat A,Chinnam R B,et al.A maximum relevance minimum redundancy feature selection method based on swarm intelligence for support vector machine classification[J].Information Science,2011,181(20): 4625-4641.
[18]Li-Yeh Chuang,Cheng-Hong Yang,Jung-Chike Li. Chaotic maps based on binary particle swarm optimization for feature selection[J].Applied Soft Computing,2011, 11(4):239-248.
[19]刘一民.基于改进进化规划方法的电力系统无功优化研究[D].武汉:华中科技大学,2007. (Liu Y M.Study on reactive power system opti-mization based on improved evolutionary programming methodfD]. Wuhan:Huazhong University of Science and Technology, 2007.)
[20]Hettich S,Bay S D.The UCI KDD Archive[DB/OL]. http://kdd.ics.uci.edu/,1999.
[21]高岳林,任子晖.带有变异算子的自适应粒子群优化算法[J].计算机工程与应用,2007,43(25):43-47. (Gao Y L,Ren Z H.Adaptive particle swarm optimization algorithm with mutation operator[J].Computer Engineering and Applications,2007,43(25):43-47.)
[2]Dash M,Liu H.Feature selection for classification[J]. Intelligent Data Analysis,1997,1(2):131-156.
[3]Liu Jihong,Wang Guoxiong.A hybrid Feature Selection Method for Data Sets of thousands of variables[C].The 2nd Int Conf on Advanced Computer Control.Shenyang,2010, 2:288-291.
[4]Sarojini Llango B,Ramaraj N.A hybrid prediction model with F-score feature selection for type II diabetes databases[C].Proc of the 1st Amrita ACM-W Celebration on Women in Computing.India:ACM,2010:1-4.
[5]刘杰,金弟,杜慧君,等.一种新的混合特征选择方法RRK[J].吉林大学学报,2009,39(2):419-423. (Liu J,Jin D,Du H J,et al.New hybrid feature selection method[J].J of Jilin University,2009,39(2):419-423.)
[6]毛俐旻,姚淑萍,胡昌振.一种新型混合特征选择方法及其在入侵检测中的应用[J].北京理工大学学报,2008, 28(3):218-221. (Mao L M,Yao S P,Hu C Z.A new hybrid attribute selection method and its application in intrusion detection[JJ.Trans of Beijing Institute of Technology, 2008,28(3):218-221.)
[7]陈鑫,梁海洁,廖腾峰.基于TSVM分类器和混合型特征选择方法的入侵检测研究[J].微电子学与计算机, 2010,27(8):242-244. (Chen X,Liang H J,Liao T F.Intrusion detection based on TSVM and feature selection[J].Microelectronics and Computer,2010,27(8):242-244.)
[8]Kennedy J,Eberhart R.Particle swarm optimization[C]. Proc of IEEE Int Conf on Neural Networks.Perth:IEEE Press,1995:1942-1948.
[9]HU Wang,LI Zhi-shu.A simpler and more effective particle swarm optimization algorithm[J].J of Software, 2007,18(4):861-868.
[10]John G H,Kohavi R,Pfleger K.Irrelevant feature and the subset selection problem[C].Proc of the 11th Int Conf on Machine Learning.New Jersey:Morgan Kaufmann Publishers,1994:121-129.
[11]Bonnlander B V,Weigend A S.Selecting input variables using mutual information and nonparametric density evaluation[C].Proc of the 1994 Int Symposium on Artificial Neural Networks.Tainan,1994:42-50.
[12]Kwak N,Choi C H.Input feature selection by mutual information based on Parzen window[J].IEEE Trans on Pattern Analysis and Machine Intelligence,2002,24(12): 1667-1671.
[13]Hanchuan Peng,Fuhui Long,Chris Ding.Feature selection based on mutual information:Criteria of max-dependency, max-relevance,and min-redundancy[J].IEEE Trans on Pattern Analysis and Machine Intelligence,2005,27(8): 1226-1238.
[14]Estevez P A,Michel Tesmer,Claudio A Perez,et al. Normalized mutual information feature selection[J].IEEE Trans on Neural Networks,2009,20(2):189-201.
[15]La The Vinh,Sungyoung Lee,Young-Tack Park,et al.A novel feature selection method based on normalized mutual information[J].Applied Intelligence,2012,37(1):100- 120.
[16]Pedrycz W.Identifying core sets of discriminatory features using particle swarm optimization[J].Expert Systems with Applications,2009,36(5):4610-4616.
[17]Unler A,Murat A,Chinnam R B,et al.A maximum relevance minimum redundancy feature selection method based on swarm intelligence for support vector machine classification[J].Information Science,2011,181(20): 4625-4641.
[18]Li-Yeh Chuang,Cheng-Hong Yang,Jung-Chike Li. Chaotic maps based on binary particle swarm optimization for feature selection[J].Applied Soft Computing,2011, 11(4):239-248.
[19]刘一民.基于改进进化规划方法的电力系统无功优化研究[D].武汉:华中科技大学,2007. (Liu Y M.Study on reactive power system opti-mization based on improved evolutionary programming methodfD]. Wuhan:Huazhong University of Science and Technology, 2007.)
[20]Hettich S,Bay S D.The UCI KDD Archive[DB/OL]. http://kdd.ics.uci.edu/,1999.
[21]高岳林,任子晖.带有变异算子的自适应粒子群优化算法[J].计算机工程与应用,2007,43(25):43-47. (Gao Y L,Ren Z H.Adaptive particle swarm optimization algorithm with mutation operator[J].Computer Engineering and Applications,2007,43(25):43-47.)