基于改进粒子群算法的模糊神经网络研究
|
摘要
近年来,模糊神经网络正日益引起学术界的重视和关注,模糊逻辑具有模拟人类大脑推理的能力,可广泛用于模式识别、专家系统、故障诊断、系统辨识以及非线性系统的控制。神经网络则具有从数据中学习的能力、并行处理能力、容错以及泛化能力。模糊神经网络结合了两者的优点,克服了神经网络从输入到输出的“黑箱”式非线性映射,又克服了人为选取模糊规则时存在的主观性。很多专家预测模糊神经网络技术有望成为21世纪智能控制领域的核心技术,并且其学习算法也引起了高度的重视。
目前使用的最多的学习算法仍然是基于梯度下降的BP算法和遗传算法。然而基于梯度下降的BP网络存在收敛速度慢、易陷入局部极小的缺陷。而遗传算法需要设置很多参数。为了解决BP算法和遗传算法的缺点,一些学者把粒子群算法用于对模糊神经网络权值的训练。
PSO的优势在于简单容易实现并且没有许多参数需要调整。目前已广泛应用于函数优化,神经网络训练,模糊系统控制等领域。
针对模糊神经网络权值训练的非线性、复杂过程,惯性权重线性递减的线性粒子群算法往往不能反映非线性优化搜索过程,动态粒子群算法虽然能较好的实现非线性的搜索,但是更容易陷入局部最优。
因此本文提出了基于禁忌搜索的动态粒子群算法。改进的粒子群算法引入了禁忌搜索的思想,来解决动态粒子群算法容易陷入局部最优的问题;并对禁忌公式进行了修改,使其不仅可以解决极小值最优问题,而且可以解决极大值最优问题。实验结果表明,改进的粒子群算法在非线性的搜索极值过程中收敛速度和最终结果都要优于惯性权重线性递减的线性粒子群算法和动态粒子群算法;且改进的粒子群算法在模糊神经网络权值的训练中收敛速度和跳出局部最优的能力都要比BP算法更优。
Recently, fuzzy neural networks attract more and more attentions from academic circle. Fuzzy logic has the ability of mimicking human reasoning capabilities, and it is widely used in pattern identification, expert systems、fault diagnosis、system identification and in the control of nonlinear systems. Neural networks have a few advantages, such as adaptive learning, parallelism, fault tolerance, and generalization. Fuzzy neural networks combine the advantages of the above two approaches, overcoming the“black-box”nonlinear mapping from input to output, and also, the subjectivity of selecting fuzzy rules by human. It is predicted by many experts that fuzzy neural networks would become the core technique in the region of intelligent control in 21 century. The study algorithm of fuzzy neural network has being thought much of.
The known study algorithms which are used to be Fuzzy Neural Network parameter study algorithms are BP algorithm with gradient descent and inheritance algorithm. However, BP network with gradient descent has some defects such as low convergence speed, fall in local minima. And inheritance algorithm has too many parameters to be set. So Particle Swarm Optimization has been introduced to train the weigh of Fuzzy Neural Network.
PSO has the advantage of simple and easy to achieve and not many parameters need to be adjusted. Has been widely used in function optimization, training neural networks, fuzzy systems control, and other fields.
For FNN weight training in non-linear, complex process, Linear Particle Swarm Optimization algorithm which makes the inertia weightωreduction linearly often fails to reflect the actual optimized search process. Dynamic particle swarm algorithm can be used to achieve the nonlinear search, but it is easy to fall into local optimization.
So Tabu Search based Dynamic particle swarm algorithm was presented. The algorithm was introduced to settle local optimization of Dynamic particle swarm algorithm. And carried on a modification to Tabu Search’s formula, make it can solve the problems both the minimum optimal and the maximum optimal. According to experiment result, improved algorithm has better result and faster optimal speed than Linear Particle Swarm Optimization and Dynamic particle swarm algorithm. And, in FNN weight training, improved PSO in the convergence rate and the ability to jump out to local optimum algorithm is better than BP.
目前使用的最多的学习算法仍然是基于梯度下降的BP算法和遗传算法。然而基于梯度下降的BP网络存在收敛速度慢、易陷入局部极小的缺陷。而遗传算法需要设置很多参数。为了解决BP算法和遗传算法的缺点,一些学者把粒子群算法用于对模糊神经网络权值的训练。
PSO的优势在于简单容易实现并且没有许多参数需要调整。目前已广泛应用于函数优化,神经网络训练,模糊系统控制等领域。
针对模糊神经网络权值训练的非线性、复杂过程,惯性权重线性递减的线性粒子群算法往往不能反映非线性优化搜索过程,动态粒子群算法虽然能较好的实现非线性的搜索,但是更容易陷入局部最优。
因此本文提出了基于禁忌搜索的动态粒子群算法。改进的粒子群算法引入了禁忌搜索的思想,来解决动态粒子群算法容易陷入局部最优的问题;并对禁忌公式进行了修改,使其不仅可以解决极小值最优问题,而且可以解决极大值最优问题。实验结果表明,改进的粒子群算法在非线性的搜索极值过程中收敛速度和最终结果都要优于惯性权重线性递减的线性粒子群算法和动态粒子群算法;且改进的粒子群算法在模糊神经网络权值的训练中收敛速度和跳出局部最优的能力都要比BP算法更优。
Recently, fuzzy neural networks attract more and more attentions from academic circle. Fuzzy logic has the ability of mimicking human reasoning capabilities, and it is widely used in pattern identification, expert systems、fault diagnosis、system identification and in the control of nonlinear systems. Neural networks have a few advantages, such as adaptive learning, parallelism, fault tolerance, and generalization. Fuzzy neural networks combine the advantages of the above two approaches, overcoming the“black-box”nonlinear mapping from input to output, and also, the subjectivity of selecting fuzzy rules by human. It is predicted by many experts that fuzzy neural networks would become the core technique in the region of intelligent control in 21 century. The study algorithm of fuzzy neural network has being thought much of.
The known study algorithms which are used to be Fuzzy Neural Network parameter study algorithms are BP algorithm with gradient descent and inheritance algorithm. However, BP network with gradient descent has some defects such as low convergence speed, fall in local minima. And inheritance algorithm has too many parameters to be set. So Particle Swarm Optimization has been introduced to train the weigh of Fuzzy Neural Network.
PSO has the advantage of simple and easy to achieve and not many parameters need to be adjusted. Has been widely used in function optimization, training neural networks, fuzzy systems control, and other fields.
For FNN weight training in non-linear, complex process, Linear Particle Swarm Optimization algorithm which makes the inertia weightωreduction linearly often fails to reflect the actual optimized search process. Dynamic particle swarm algorithm can be used to achieve the nonlinear search, but it is easy to fall into local optimization.
So Tabu Search based Dynamic particle swarm algorithm was presented. The algorithm was introduced to settle local optimization of Dynamic particle swarm algorithm. And carried on a modification to Tabu Search’s formula, make it can solve the problems both the minimum optimal and the maximum optimal. According to experiment result, improved algorithm has better result and faster optimal speed than Linear Particle Swarm Optimization and Dynamic particle swarm algorithm. And, in FNN weight training, improved PSO in the convergence rate and the ability to jump out to local optimum algorithm is better than BP.
引文
[1] Jang J.R. ANFIS: Adaptive-Network-Based Fuzzy Inference System [C].IEEE Transactions on Systems, Man, and Cybernetics, 23:665-684,1993.
[2]孙海蓉.模糊神经网络的研究及应用[D].河北:华北电力大,2006.
[3] LTran.Haoi, S.Osowski. Neuro-fuzzy TSK network for approximation of static and dynamic functions[J]. Control and Cybernetics, 31(2):309-326,2002.
[4] C.J.Lin and C.T.Lin. An ART-based fuzzy adaptive learning control network[C]. IEEE Trans. Fuzzy Syst.,1997,5(4):477-496.
[5]白江斌,金慰刚,张建.基于粒子群的模糊神经网络[J].华北电力大学,2007,(02),16-19.
[6]王世卫,李爱国.粒子群优化算法训练模糊神经网络[J].仪器仪表学报,2004,(08),938-939.
[7]鲁小帆,郭嗣琮,董超.基于遗传算法的神经网络学习算法研究[C].中国计量协会冶金分会,2007.
[8] C.T.Lin, C.S.G.Lee. Neural-network-based fuzzy logic control and decision system[C]. IEEE Trans.Comput.,1991,40(12):1320-1336.
[9] Buckley J J, Hayashi Yoichi. Fuzzy neural networks: a survey[J]. Fuzzy Sets and Systems, 1994:1-13.
[10]应行仁,曾南.采用BP神经网络模糊控制器的研究[J].自动化学报,1997,(17):63-67.
[11]王科俊,金鸿章,李国斌等.神经网络模糊控制器的研究[J].黑龙江自动化技术与应用,1993,12(3):47-50.
[12]徐冬玲,方建安,邵世煌.交通系统的模糊控制及其神经网络实现[J].信息与控制,1992,21(2):74-78.
[13]顾毅.智能控制发展综述[J].信息技术,2000,(06):39-40.
[14]张选平,杜玉平,秦国强,覃征.一种动态改变惯性权重的自适应粒子群算法[J].西安交通大学学报,Vol,39,No,10,10,2005,P.1039-1042.
[15] Zadeh L.A. Fuzzy Set[J]. Information and Control, 1965, 8 (3):338-358.
[16] Mamadani E. H., Application of fuzzy algorithms for simple dynamic plant, In Proc[J]. IEEE, 1994,121 (12):1585-1588
[17]王爽,朱栋华,王家凯.模糊神经网络的理论与应用[J],江苏环境科技,2007,(S2).
[18]王振雷.模糊神经网络理论及其在复杂系统中的应用研究[D].沈阳:东北大学,2003.
[19] W. S. McCulloch,W. Pitts. A logic calculus of the ideas imminent in neurons activity[J]. Bulletin of Math.Bio, 1993,No.5,115-133.
[20] Eberhart RC, Kennedy J. Swarm Intelligence[J]. Morgan Kanfmans,2001.
[21] J.J.Hopfield.Neural networks and physical system with emergent collective computational abilities[J]. Proc. Acad. Sci USA, 1982, Vol. 79, 2554-2558
[22] G.E.Hinton, T.J.ejnowski, D.H.Ackley. Boltzmann machine: constraint satisfaction networks that learn[C]. CMV-CS-84-119, Carneie-Mellon Uni,1984.
[23] D.H. Ackley, G E. Hinton, T. J. Sejnowski. Learning algorithm for bolt zmann machines[J]. Cognitive Science 1988,Vol.9,147-169.
[24] D.E.Rumelhart, J.L.McClelland. Parallel Distributed Processing[D]. America:MIT,1986.
[25]焦李成.神经网络的应用与实现[M].西安:西安电子科技大学出版社,1993.
[26]莫宏伟,金鸿章.人工免疫系统、人工神经网络、进化算法的比较[C] 2003中国控制与决策学术年会论文集.
[27]刘士容.神经模糊系统得若干问题研究[D].上海:华东理工大学,2000.
[28]吴涛.自适应增强型模糊神经网络及其在建模控制中的应用[D].上海:上海交通大学,1999.
[29] Fredric M.Ham,Ivica Kostanic.神经计算原理[M].北京:机械工业出版社,2003.
[30]蒋宗礼.人工神经网络导论.北京:高等教育出版社,2002.
[31] Mamadani E. H.Application of fuzzy algorithms for simple dynamic plant, In Proc[J]. IEEE, 1974,121 (12):1585-1588.
[32]徐丽娜编著.神经网络控制[M].北京:电子工业出版社,2002.
[33] Eberhart RC, Kennedy J, Swarm Intelligence, Morgan Kanfmans,2001.
[34]周文.粒子群优化算法及其参数设置的研究[J].湖北职业技术学院学报,2006,(12),No 4,Vol 9:93-96.
[35] Tseng C S, Chen B S, Uang H J. Fuzzy tracking control design for nonlinear dynamic systems via T-S fuzzy model[J ]. IEEE Trans on Fuzzy System , 2001, 9 (3) :381-392.
[36] D.H. Ackley,G E. Hinton&T. J. Sejnowski. Learning algorithm for bolt zmann machines. Cognitive Science 1988,Vol.9,147-169.
[37]黄显明,易继锴.基于Rough集理论的模糊神经网络构造方法[J].中国工程科学,2004年4月第6卷第4期.
[2]孙海蓉.模糊神经网络的研究及应用[D].河北:华北电力大,2006.
[3] LTran.Haoi, S.Osowski. Neuro-fuzzy TSK network for approximation of static and dynamic functions[J]. Control and Cybernetics, 31(2):309-326,2002.
[4] C.J.Lin and C.T.Lin. An ART-based fuzzy adaptive learning control network[C]. IEEE Trans. Fuzzy Syst.,1997,5(4):477-496.
[5]白江斌,金慰刚,张建.基于粒子群的模糊神经网络[J].华北电力大学,2007,(02),16-19.
[6]王世卫,李爱国.粒子群优化算法训练模糊神经网络[J].仪器仪表学报,2004,(08),938-939.
[7]鲁小帆,郭嗣琮,董超.基于遗传算法的神经网络学习算法研究[C].中国计量协会冶金分会,2007.
[8] C.T.Lin, C.S.G.Lee. Neural-network-based fuzzy logic control and decision system[C]. IEEE Trans.Comput.,1991,40(12):1320-1336.
[9] Buckley J J, Hayashi Yoichi. Fuzzy neural networks: a survey[J]. Fuzzy Sets and Systems, 1994:1-13.
[10]应行仁,曾南.采用BP神经网络模糊控制器的研究[J].自动化学报,1997,(17):63-67.
[11]王科俊,金鸿章,李国斌等.神经网络模糊控制器的研究[J].黑龙江自动化技术与应用,1993,12(3):47-50.
[12]徐冬玲,方建安,邵世煌.交通系统的模糊控制及其神经网络实现[J].信息与控制,1992,21(2):74-78.
[13]顾毅.智能控制发展综述[J].信息技术,2000,(06):39-40.
[14]张选平,杜玉平,秦国强,覃征.一种动态改变惯性权重的自适应粒子群算法[J].西安交通大学学报,Vol,39,No,10,10,2005,P.1039-1042.
[15] Zadeh L.A. Fuzzy Set[J]. Information and Control, 1965, 8 (3):338-358.
[16] Mamadani E. H., Application of fuzzy algorithms for simple dynamic plant, In Proc[J]. IEEE, 1994,121 (12):1585-1588
[17]王爽,朱栋华,王家凯.模糊神经网络的理论与应用[J],江苏环境科技,2007,(S2).
[18]王振雷.模糊神经网络理论及其在复杂系统中的应用研究[D].沈阳:东北大学,2003.
[19] W. S. McCulloch,W. Pitts. A logic calculus of the ideas imminent in neurons activity[J]. Bulletin of Math.Bio, 1993,No.5,115-133.
[20] Eberhart RC, Kennedy J. Swarm Intelligence[J]. Morgan Kanfmans,2001.
[21] J.J.Hopfield.Neural networks and physical system with emergent collective computational abilities[J]. Proc. Acad. Sci USA, 1982, Vol. 79, 2554-2558
[22] G.E.Hinton, T.J.ejnowski, D.H.Ackley. Boltzmann machine: constraint satisfaction networks that learn[C]. CMV-CS-84-119, Carneie-Mellon Uni,1984.
[23] D.H. Ackley, G E. Hinton, T. J. Sejnowski. Learning algorithm for bolt zmann machines[J]. Cognitive Science 1988,Vol.9,147-169.
[24] D.E.Rumelhart, J.L.McClelland. Parallel Distributed Processing[D]. America:MIT,1986.
[25]焦李成.神经网络的应用与实现[M].西安:西安电子科技大学出版社,1993.
[26]莫宏伟,金鸿章.人工免疫系统、人工神经网络、进化算法的比较[C] 2003中国控制与决策学术年会论文集.
[27]刘士容.神经模糊系统得若干问题研究[D].上海:华东理工大学,2000.
[28]吴涛.自适应增强型模糊神经网络及其在建模控制中的应用[D].上海:上海交通大学,1999.
[29] Fredric M.Ham,Ivica Kostanic.神经计算原理[M].北京:机械工业出版社,2003.
[30]蒋宗礼.人工神经网络导论.北京:高等教育出版社,2002.
[31] Mamadani E. H.Application of fuzzy algorithms for simple dynamic plant, In Proc[J]. IEEE, 1974,121 (12):1585-1588.
[32]徐丽娜编著.神经网络控制[M].北京:电子工业出版社,2002.
[33] Eberhart RC, Kennedy J, Swarm Intelligence, Morgan Kanfmans,2001.
[34]周文.粒子群优化算法及其参数设置的研究[J].湖北职业技术学院学报,2006,(12),No 4,Vol 9:93-96.
[35] Tseng C S, Chen B S, Uang H J. Fuzzy tracking control design for nonlinear dynamic systems via T-S fuzzy model[J ]. IEEE Trans on Fuzzy System , 2001, 9 (3) :381-392.
[36] D.H. Ackley,G E. Hinton&T. J. Sejnowski. Learning algorithm for bolt zmann machines. Cognitive Science 1988,Vol.9,147-169.
[37]黄显明,易继锴.基于Rough集理论的模糊神经网络构造方法[J].中国工程科学,2004年4月第6卷第4期.