神经网络集成及其在分类和回归问题中的应用研究
|
摘要
神经网络集成是一种可以显著地提高神经网络系统泛化能力的方法,它通过简单地训练多个神经网络并将其结果进行合成来实现。神经网络集成实现方法的研究主要集中在两个方面,即怎样将多个神经网络的输出结论进行结合以及如何生成集成中的个体网络,包括如何有效地应用有限的样本集。选择性集成可以有效地降低神经网络集成学习的泛化误差。本文结合实际问题,针对现有方法的不足提出了动态选择性集成和模糊核聚类集成的新方法,并利用核主元分析对高维故障样本进行特征提取,进一步提高了汽轮机故障诊断的精确度和稳定性。最后,提出了一种基于熵值法加权的神经网络集成方法,进一步提高了风电场风速预测的精确度。
Neural network ensemble can significantly improve generalization of neural network systems by training several neural networks simply and combining their results. The research of ensemble learning is focused on two aspects, namely, how to combine the results of multiple neural networks and how to generate individual neural network, including how to effectively use the limited sample set. Selective ensemble can effectively reduce the generalization error of ensemble learning. In this paper, against inadequacy of existing methods, a dynamic selective ensemble method and a kernel fuzzy c-means clustering ensemble method are proposed which combined kernel principal components analysis for features extraction of high dimension fault data to further improve the accuracy and stability of classification for fault diagnosis of steam turbine. Lastly, an entropy weighted ensemble method is proposed to further improve the accuracy of prediction for wind speed.
Neural network ensemble can significantly improve generalization of neural network systems by training several neural networks simply and combining their results. The research of ensemble learning is focused on two aspects, namely, how to combine the results of multiple neural networks and how to generate individual neural network, including how to effectively use the limited sample set. Selective ensemble can effectively reduce the generalization error of ensemble learning. In this paper, against inadequacy of existing methods, a dynamic selective ensemble method and a kernel fuzzy c-means clustering ensemble method are proposed which combined kernel principal components analysis for features extraction of high dimension fault data to further improve the accuracy and stability of classification for fault diagnosis of steam turbine. Lastly, an entropy weighted ensemble method is proposed to further improve the accuracy of prediction for wind speed.
引文
[1]朱大奇.人工神经网络研究现状及其展望.江南大学学报,2004,3(1):103～110
[2]董军,胡上序.混沌神经网络研究进展和展望.信息与制,1997,26(5):360～368
[3]Hansen L K, Salamon P. Neural network ensembles. IEEE Trans. Pattern Analysis and Machine Intelligence, 1990, 12(10): 993～1001
[4]Zhou Z H, Wu J X, Tang W. Ensembling neural networks: many could be better than all. Artificial Intelligence, 2002, 137(222): 239～263
[5]Krogh A,Vedelsby J.Neural network ensembles,cross validation,and active learning. Cambridge: MIT Press, 1995: 231～238
[6]傅强.选择性神经网络集成算法研究:[博士学位论文].杭州:浙江大学, 2007
[7]Nilsson N J. Learning Machines: Foundations of trainable pattern- classifying. New York: McGraw Hill, 1965
[8]Kanal L. Patterns in pattern recognition. IEEE Trans. Information Theory, 1974, 20(6): 697～722
[9]Minsky M. Logical versus analogical or symbolic versus connectionist or neat versus scruffy. AI Magazine, 1991, 12(2): 34～51
[10]Sollich P, Krogh A. Learning with ensembles: How over-fitting can be useful. Cambridge: MIT Press, 1996
[11]Schapire R E. The strength of weak learnability. Machine learning, 1990, 5(2): 197～227
[12]Perrone M P, Cooper L N. When networks disagree: Ensemble method for neural networks. London: Chapman-hall, 1993, 126～142
[13]Breiman L. Bagging Predictors. Machine Learning, 1996,24(2): 123～140
[14]Freund Y, Schapire R E. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Science, 1997, 55(1): 119～139
[15]Yao X, Liu Y. Making use of population information in evolutionary artificial neural networks. IEEE Trans. Systems, Man and Cybernetics -Part B: Cybernetics, 1998, 28(3): 417～425
[16]Jimenez D. Dynamically weighted ensemble neural networks for classification. In: Proceedings of the IEEE International Joint Conference on neural networks, Los Alamitos, CA: IEEE, 1998, Vol.1, 753～756
[17]Liu Y, Yao X. Ensemble learning via negative correlation. Neural Networks, 1999, 12(10): 1399～1404
[18] Lazarevic A, Obradoric Z. Effeetive Pruning of neural network classifier ensembles. In: Proc. Iniemation Joint Conference on Neural Networks, 2001. 796～801
[19]Islam M M,Yao X,Murase K. A constructive algorithm for training cooperative neural network ensembles. IEEE Transactions on Neural Networks,2003,14(4): 820～834
[20]王正群,陈世福,陈兆乾.并行学习神经网络集成方法.计算机学报,2005, 28(3): 402～408
[21]王正群,陈世福,陈兆乾.一种主动学习神经网络集成方法.计算机研究与发展, 2005, 42(3): 375～380
[22]Valiant L G. A theory of the learnable. Artificial Intelligence and Language Processing, 1984, 27(11): 1134～1142
[23]Kearns M, Valiant L G. Learning boolean formulae or factoring. Cambridge: Havard University Aiken Computation Laboratory, 1988
[24]肖健华.智能模式识别方法.广州:华南理工大学出版社,2006
[25]Opitz D W, Shavlik J W. Actively searching for an effective neural network ensemble. Connection Science, 1996, 8(3-4): 337～353
[26]Scholkopf B,Smola A,Müller K R.Nonlinear Component Analysis as a Kernel Eigenvalue Problem.Neural Computation,1998,(10):1299～1319
[27]凌维业,贾民平.粗糙集神经网络故障诊断系统的优化方法研究.中国电机工程学报,2003,23(5):98～102
[28]陈果.粗糙集-遗传算法-神经网络集成分类器及其在转子故障诊断中的应用研究.中国机械工程,2008,19(1):85～90
[29]侯国莲,孙晓刚,张建华,等.基于非线性主元分析和概率神经网络的凝汽器故障诊断方法研究.中国电机工程学报,2005,25(18):104～108
[30]焦卫东,钱苏翔,杨世锡,等.基于非线性主分量分析—多层感知器网络的变压器油中溶解气体数据分析.中国电机工程学报,2007,27(6):72～76
[31]郝志华,林田,马孝江.基于经验模式分解的神经网络模型及其在转子系统故障诊断中的应用.中国电机工程学报,2006,26(20):149～153
[32]魏海坤著.神经网络结构设计的理论与方法.北京:国防工业出版社,2005:25～30
[33]郑建军,甘仞初,贺跃,等.神经网络分类器动态集成方法.北京理工大学学报, 2005, 25(12): 1062～1065
[34]张道强.基于核的联想记忆及聚类算法的研究与应用:[博士学位论文].南京:南京航空航天大学,2004
[35]普运伟,金炜东,朱明,等.核模糊C均值算法的聚类有效性研究.计算机科学,2007,34(2):207～210
[36]罗会兰,孔繁胜,李一啸.聚类集成中的差异性度量研究.计算机学报,2007,30(8):1315～1324
[37]李凯,黄厚宽.一种基于聚类技术的选择性神经网络集成方法.计算机研究与发展,2005,42(4):594～598
[38]杨秀媛,肖洋,陈树勇.风电场风速和发电功率预测研究.中国电机工程学报,2005,25(11):1～5
[39]Mehmet B,Besir S,Abdulkadir Y.Application of artificial neural networks for the wind speed prediction of target station using reference stations data.Renewable energy,2007,32(14):2350～2360
[40]蔡凯,谭伦农,李春林,等.时间序列与神经网络法相结合的短期风速预测.电网技术,2008,32(8):82～85
[41]Takens F.Detecting strange attractors in turbulence.Lecture notes in Math,1981,898:366～381
[42]冬雷,高爽,廖晓钟,等.风力发电系统发电容量时间序列的混沌属性分析.太阳能学报,2007,28(11):1290～1294
[43]Kim H S, Eykholt R, Salas J D.Nonlinear dynamics,delay times and embedding windows.Physica D,1999,127(1-2):48～60
[44]Grassberger P , Procaccia I . Measuring the strangeness of strange attractors.Physica D,1983,9:189～208
[45]Rosenstein M T, Collins J J, De L C J.A practical method for calculating largest Lyapunov exponents from small data sets.Physica D,1993,65:117～134
[46]Wolf A, Swift J B, Swinney H L, et al.Determining Lyapunov exponents from a time series.Physica D,1985,16(33):285～317
[47]吕金虎,陆君安,陈士华著.混沌时间序列分析及其应用.武汉:武汉大学出版社,2002:107～108
[48]雷绍兰.基于电力负荷时间序列混沌特性的短期负荷预测方法研究: [博士学位论文].重庆:重庆大学,2005:33～74
[49]陈华友.组合预测方法有效性理论及其应用.北京:科学出版社,2008:50～54
[50]王永生,孙瑾,王昌金,等.变参数混沌时间序列的神经网络预测研究.物理学报,2008,57(10):6120～6131
[2]董军,胡上序.混沌神经网络研究进展和展望.信息与制,1997,26(5):360～368
[3]Hansen L K, Salamon P. Neural network ensembles. IEEE Trans. Pattern Analysis and Machine Intelligence, 1990, 12(10): 993～1001
[4]Zhou Z H, Wu J X, Tang W. Ensembling neural networks: many could be better than all. Artificial Intelligence, 2002, 137(222): 239～263
[5]Krogh A,Vedelsby J.Neural network ensembles,cross validation,and active learning. Cambridge: MIT Press, 1995: 231～238
[6]傅强.选择性神经网络集成算法研究:[博士学位论文].杭州:浙江大学, 2007
[7]Nilsson N J. Learning Machines: Foundations of trainable pattern- classifying. New York: McGraw Hill, 1965
[8]Kanal L. Patterns in pattern recognition. IEEE Trans. Information Theory, 1974, 20(6): 697～722
[9]Minsky M. Logical versus analogical or symbolic versus connectionist or neat versus scruffy. AI Magazine, 1991, 12(2): 34～51
[10]Sollich P, Krogh A. Learning with ensembles: How over-fitting can be useful. Cambridge: MIT Press, 1996
[11]Schapire R E. The strength of weak learnability. Machine learning, 1990, 5(2): 197～227
[12]Perrone M P, Cooper L N. When networks disagree: Ensemble method for neural networks. London: Chapman-hall, 1993, 126～142
[13]Breiman L. Bagging Predictors. Machine Learning, 1996,24(2): 123～140
[14]Freund Y, Schapire R E. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Science, 1997, 55(1): 119～139
[15]Yao X, Liu Y. Making use of population information in evolutionary artificial neural networks. IEEE Trans. Systems, Man and Cybernetics -Part B: Cybernetics, 1998, 28(3): 417～425
[16]Jimenez D. Dynamically weighted ensemble neural networks for classification. In: Proceedings of the IEEE International Joint Conference on neural networks, Los Alamitos, CA: IEEE, 1998, Vol.1, 753～756
[17]Liu Y, Yao X. Ensemble learning via negative correlation. Neural Networks, 1999, 12(10): 1399～1404
[18] Lazarevic A, Obradoric Z. Effeetive Pruning of neural network classifier ensembles. In: Proc. Iniemation Joint Conference on Neural Networks, 2001. 796～801
[19]Islam M M,Yao X,Murase K. A constructive algorithm for training cooperative neural network ensembles. IEEE Transactions on Neural Networks,2003,14(4): 820～834
[20]王正群,陈世福,陈兆乾.并行学习神经网络集成方法.计算机学报,2005, 28(3): 402～408
[21]王正群,陈世福,陈兆乾.一种主动学习神经网络集成方法.计算机研究与发展, 2005, 42(3): 375～380
[22]Valiant L G. A theory of the learnable. Artificial Intelligence and Language Processing, 1984, 27(11): 1134～1142
[23]Kearns M, Valiant L G. Learning boolean formulae or factoring. Cambridge: Havard University Aiken Computation Laboratory, 1988
[24]肖健华.智能模式识别方法.广州:华南理工大学出版社,2006
[25]Opitz D W, Shavlik J W. Actively searching for an effective neural network ensemble. Connection Science, 1996, 8(3-4): 337～353
[26]Scholkopf B,Smola A,Müller K R.Nonlinear Component Analysis as a Kernel Eigenvalue Problem.Neural Computation,1998,(10):1299～1319
[27]凌维业,贾民平.粗糙集神经网络故障诊断系统的优化方法研究.中国电机工程学报,2003,23(5):98～102
[28]陈果.粗糙集-遗传算法-神经网络集成分类器及其在转子故障诊断中的应用研究.中国机械工程,2008,19(1):85～90
[29]侯国莲,孙晓刚,张建华,等.基于非线性主元分析和概率神经网络的凝汽器故障诊断方法研究.中国电机工程学报,2005,25(18):104～108
[30]焦卫东,钱苏翔,杨世锡,等.基于非线性主分量分析—多层感知器网络的变压器油中溶解气体数据分析.中国电机工程学报,2007,27(6):72～76
[31]郝志华,林田,马孝江.基于经验模式分解的神经网络模型及其在转子系统故障诊断中的应用.中国电机工程学报,2006,26(20):149～153
[32]魏海坤著.神经网络结构设计的理论与方法.北京:国防工业出版社,2005:25～30
[33]郑建军,甘仞初,贺跃,等.神经网络分类器动态集成方法.北京理工大学学报, 2005, 25(12): 1062～1065
[34]张道强.基于核的联想记忆及聚类算法的研究与应用:[博士学位论文].南京:南京航空航天大学,2004
[35]普运伟,金炜东,朱明,等.核模糊C均值算法的聚类有效性研究.计算机科学,2007,34(2):207～210
[36]罗会兰,孔繁胜,李一啸.聚类集成中的差异性度量研究.计算机学报,2007,30(8):1315～1324
[37]李凯,黄厚宽.一种基于聚类技术的选择性神经网络集成方法.计算机研究与发展,2005,42(4):594～598
[38]杨秀媛,肖洋,陈树勇.风电场风速和发电功率预测研究.中国电机工程学报,2005,25(11):1～5
[39]Mehmet B,Besir S,Abdulkadir Y.Application of artificial neural networks for the wind speed prediction of target station using reference stations data.Renewable energy,2007,32(14):2350～2360
[40]蔡凯,谭伦农,李春林,等.时间序列与神经网络法相结合的短期风速预测.电网技术,2008,32(8):82～85
[41]Takens F.Detecting strange attractors in turbulence.Lecture notes in Math,1981,898:366～381
[42]冬雷,高爽,廖晓钟,等.风力发电系统发电容量时间序列的混沌属性分析.太阳能学报,2007,28(11):1290～1294
[43]Kim H S, Eykholt R, Salas J D.Nonlinear dynamics,delay times and embedding windows.Physica D,1999,127(1-2):48～60
[44]Grassberger P , Procaccia I . Measuring the strangeness of strange attractors.Physica D,1983,9:189～208
[45]Rosenstein M T, Collins J J, De L C J.A practical method for calculating largest Lyapunov exponents from small data sets.Physica D,1993,65:117～134
[46]Wolf A, Swift J B, Swinney H L, et al.Determining Lyapunov exponents from a time series.Physica D,1985,16(33):285～317
[47]吕金虎,陆君安,陈士华著.混沌时间序列分析及其应用.武汉:武汉大学出版社,2002:107～108
[48]雷绍兰.基于电力负荷时间序列混沌特性的短期负荷预测方法研究: [博士学位论文].重庆:重庆大学,2005:33～74
[49]陈华友.组合预测方法有效性理论及其应用.北京:科学出版社,2008:50～54
[50]王永生,孙瑾,王昌金,等.变参数混沌时间序列的神经网络预测研究.物理学报,2008,57(10):6120～6131