摘要
非线性系统辨识是控制理论研究的难点和热点。利用对角递归神经网络对复杂的非线性系统的模型进行辨识时,可通过调整内部神经元的权值来实现非线性系统的动态映射,表现出了很强的动态映射能力,同时使得网络权值调节的计算量很小。但是由于现有的用于训练对角递归神经网络的学习算法存在收敛速度慢、辨识精度不够理想的缺点,因此本文针对这一问题进行了探讨和研究。
本文首先针对在对角递归神经网络中广泛采用的DBP算法的辨识误差较大且收敛速度慢的缺点,分别采用了Lyapunov函数算法和遗传算法作为改进算法训练对角递归神经网络。并对DBP算法、Lyapunov函数算法及遗传算法的辨识效果进行了比较,仿真结果表明基于Lyapunov函数算法的辨识误差和辨识精度都要优于DBP算法和遗传算法,其辨识误差最小,且收敛速度最快。
其次,为了避免在Lyapunov函数算法中梯度大小对网络权值改变的影响,本文首次将一种局部自适应学习算法RPROP算法用于训练对角递归神经网络,该算法不受梯度大小对权值调整的影响,而只是决定网络权值的调整方向,该算法辨识精度高,能加速收敛,并在一定程度上克服了局部最小问题。
然后,针对神经网络隐层节点数任要靠经验选取的问题,本文提出将RPROP算法与支持向量回归算法相结合的混合算法——RPROP-SVR算法,用于对角递归神经网络的训练,其中利用SVR算法自动确定网络隐层节点数,利用RPROP算法训练网络权值,并将这一新算法用于非线性系统辨识,取得了很好的辨识效果。
最后,将基于RPROP算法与RPROP-SVR混合算法的辨识效果进行了对比,仿真结果表明,两种算法的辨识误差、辨识精度及收敛速度基本相同,这就证明了利用RPROP-SVR混合算法自动确定网络隐层节点数的有效性,说明该方法可替代人工试取的方式一次性自动确定出神经网络结构,从而可节省大量因人工试取隐层节点数而耗费的试验时间。
The identification of nonlinear system is always the difficulty and the focus of control theory study. When identifying the complex nonlinear system model based on the diagonal recurrent neural network, dynamic mapping for nonlinear system is available via adjusting internal neurons weight. The results show the highly dynamic mapping capability as well as the little regulation for network weight. But due to existing learning algorithm of the diagonal recurrent neural network being defects of slow convergence and bad identification inaccuracy, so this paper discussed and researched the topic.
As the large identification errors and the slow convergence rate for dynamic back propagation (DBP) algorithm in the diagonal recurrent neural network, Lyapunov function algorithm (LFA) and genetic algorithm (GA) here are developed as an improved algorithm to train the diagonal recurrent neural network. The identification results are fully compared to demonstrate that both the identification errors and the convergence rate for LFA are better than that of DBP algorithm and GA, the identification residual is the smallest, and the convergence rate is the most fast.
Secondly, in order to avoid the gradient affection for network weight in Lyapunov function, a local adaptive learning algorithm, namely resilient back-propagation (RPROP) algorithm, is applied to train the diagonal recurrent neural network, which can almost neglected the gradient variation but determine the adjustment direction for the network weight. Therefore, RPROP algorithm has the advanced characteristics as high identification precision, convergence-acceleration capacity and appropriate solution for local minimum problem.
According to problem of selecting the number of hidden neural network nodes by experience, a hybrid algorithm named RPROP-SVR is proposed by integrating RPROP algorithm and vector support regression (SVR) algorithm to train the diagonal recurrent neural network. Detailedly, the number of hidden neural network nodes is selected by SVR algorithm, as well as that network weight is trained by RPROP algorithm. Furthermore, this new algorithm shows that expected identification effect has been obtained with its application in a nonlinear identification system.
Finally, the identification effects for RPROP algorithm and RPROP-SVR algorithm, respectively, are fully compared basing on the simulation results that the identification errors, the identification precision and the convergence speed are almost the same. Therefore, RPROP-SVR hybrid algorithm is demonstrated to automatically effectively determine the number of hidden neural network nodes. Interestingly, this technique can be an alternative way of artificial test to automatically build the structure of a neural network at a time, which improves the efficiency a lot without artificial test of the number of hidden neural network nodes.
引文
[1]侯媛彬.汪梅.王立琦.系统辨识及其MATLAB仿真.北京:科学出版社,2004:
[2]徐丽娜.神经网络控制[M].哈尔滨:哈尔滨工业大学出版社,1999
[3]史天运.贾利民.基于进化策略的动态递归神经网络建模与辨识[J].控制与决策.2000,15(4):439
[4]宋轶民.余越庆.张策等.动态递归网络及其在机敏机构辨识中的应用[J].机械科学与技术.2001,20(4):515
[5]Gao F R.Wang F L.A simple nonlinear controller with diagonal recurrent neural network[J].Chem Eng Sci.2000,55:1283
[6]Rafael P H.Simple recurrent neural networks towards prediction and modeling of dynamical system[J].Neurocomputing.1998,23:277
[7]Aussem A.Dynamical recurrent neural networks towards prediction and modeling of dynamical system[J].Neurocomputing.1999,28:207
[8]田景文.高美娟.人工神经网络算法研究及应用.北京理工大学出版社,2006:24
[9]韩力群.人工神经网络教程.北京邮电大学出版社,2006:14-15
[10]Hayakawa T.Haddad W M.Hovakim yan N.et al.Neural network adaptive control for nonlinear non-negative dynamical systems[J].IEEE Transactions on Neural Networks.2005,16(2):399-413.
[11]Xiong Z H.Zhang J.A batch-to-batch iterative op-timal control strategy based on recurrent neural net-work models[J].Journal of Process Control.2005,15(1):11-21.
[12]Ge Hong-wei.Liang Yan-chun.Evolutionary Elman neural network model and identification for non-line-at systems[J].Journal of Jilin University(Engineering and technology Edition).2005,35(5):511-519.
[13](美)哈根著.戴葵译.神经网络设计.北京:机械工业出社.2002,PP:1-255
[14]韦巍.一种回归神经网络的快速在线学习算法.自动化学报.1998,vol.24(5),PP:616-621
[15]Gao F R.Wang F L.A simple nonlinear controller with diagonal recurrent neural network[J].Chem Eng Sei,2000(3),pp:55-58
[16]李鸿儒.顾树生.邓长辉.递归神经网络的RPE算法及其在非线性动态系统建模中的应用.东北大学学报.2000,vol.21(6),pp:590-593
[17]段慧达.郑德玲.刘聪.基于对角递归神经网络的建模及应用.北京科技大学学报.2004,vol.26(1),pp:103-106
[18]邹高峰.王正欧.基于回归神经网络的非线性时变系统辨识.控制与决策.2002(9):517-521
[19]张丽红.王艳.基于回归神经网络自适应快速BP算法.计算机测量与控制.2004.12(5):480-482
[20]邹政达.孙雅明.张智晟.基于蚁群优化算法递归神经网络的短期负荷预测.电网技术.2005.2:59-62
[21]杜福银.徐扬.基于递归神经网络的预测模糊控制.西南交通大学学报.2006,12:733-736
[22]徐戎.一种改进的递归神经网络盲均衡算法.电子科技大学学报.2007,4:210-212
[23]胡云安.左斌.李静.退火递归神经网络极值搜索算法及其在无人机紧密编队飞行控制中的应用.控制理论与应用.2008,10:879-882
[24]刘金琨.智能控制.电子工业出版社.2006:135
[25]Cong Shuang.Dai Yi.Structure of recurrent neural networks.Computer Application.2004,24(8):18-27
[26]陈平.裘丽华.王占林.基于对角同归网络的非线性系统建模.北京航空航天大学.2003,3(29):248-251
[27]王振雷.王建辉.顾树生.一种新的对角回归神经网络快速学习算法.控制与决策.2002.5(17):346-348
[28]Ku Chao-Chee,Lee Kwang Y.Diagonal recurrent neural networks for dynamic system control[J].IEEE Trans.Neural Networks,1995,6(1):144-156
[29]段慧达.王建南.白晶.基于动态对角递归网络的变压器故障诊断.2007,23:214-215
[30]于海波.马翠红.基于对角递归神经网络系统辨识及应用.微计算机信息.2007,23:216-217
[31]Du Yan chum Li Yi bin.Wang Gui yue.Application of Genetic Diagonal Recurrent Neural Network to Servo System.Proceedings of the 26~(th) Chinese Control Conference July 26-31,2007,Zhangjiajie,Hunan,China
[32]Yah chun Du.Yi bin Li.Gui yue Wang.Mobile Robot Behavior Controller Based on Genetic Diagonal Recurrent Neural Network.Proceedings of the IEEE International Conference on Automation and Logistics August 18-21,2007,Jinan,China
[33]Liang Deng.Yong hong Tan.Diagonal recurrent neural network with modi(?)ed backlash operators for modeling of rate-dependent hysteresis in piezoelectric actuators.Sensors and Actuators A.2008,148:259-270
[34]张欣.基于DRNN的传感器的建模研究.2008,12:61-71
[35]Ying zheng Han.Ju min Zhao.Deng Ao Li.Research of Adaptive Equalization Algorithm Based on Diagonal Recurrent Neural Network.Fourth International Conference on Natural Computation.IEEE 2008,352-355
[36]LiYi Zhang.Rui Lu.Huahui Wang.DingGuo Sha.Analysis of the Neural Network Blind Equalization.Journal of Electronic Measurement and Instrument.2002,27(6):1867-1875
[37]Zhou Zhen-xiong.Yang Jian-dong.Qu Yong-yin.The Position Controller with DRNN Compensator for PMLSM.2008 Chinese Control and Decision Conference.IEEE 2008:3156-3160
[38]M.Anthony,P.Bartlett.Learning in Neural Networks:Theoretical Foundations[M].Cambridge University Press.1999
[39]J.Shawe-Taylor,N.Cristianini.Margin Distribution and Soft Margin[M].In A.Smola,P.Bartlett,B.Scholkopf,and D.Schuurmans,editors,Advances in Large Margin Classifiers,MIT Press,Cambridge,MA,2000:349-358
[40]J.Weston.Leave-One-Out Support Vector Machines[A].IJCAI[C]1999:727-733
[41]A.Smola,B.Scholkopf,K.R.Muller.Convex Cost Functions for Support Vector Regression[A].In L.Niklasson,M.Bodtn,and T.Ziemke,editors,Proceedings of the Eighth International Conference on Artificial Neural Networks[C],Perspectives in Neural Computing,Berlin Springer-Verlag.1998
[42]Tong S,Koller D.Support vector machine active learning with applications to text classification.Journal of Machine Learning Research,2002,2(1):45-66
[43]黄启春.刘仰光.何钦铭.基于支持向量机的增量式算法.浙江大学学报.2008,12:2121-2126
[44]Suykens J.A.K.,Vandewalls J.,De Moor.Optimal control by least squares support vector machines.Neural Networks.2001,14:23-35
[45]肖健华.吴今培.杨叔子.基于SVM的综合评价方法研究.计算机工程.2002,28(8):28-30
[46]李元诚.方廷健.基于粗糙集理论的支撑向量机预测方法研究.数据采集与处理.2003,18(2):199-203
[47]Ling Wei.Jian-jun Qi.Wen-xie Zhang.Knowledge discovery of decision table based on support vector machine.Proc.of the second international conference on machine learning and cybernetics.Xi'an.November 2-5,2003
[48]F.Mclgani,L.Bruzzone.Classificaton of hyperspectral remote sensing Images with support vector machines.IEEE Transactions on Geoscience and Remote Sensing.2004,42(8):1778-1790
[49]M.M.Chi,L.Bruzzone.Semisupervised classification of hyperspectral images by SVMs optimized in the primal.Transactions on Geoscience and Remote Sensing,2007,45(6):1870-1880
[50]Song-ming Jiao.Pu Han.Li-hui Zhou.Jiao-bo Li.Recurrent neural network applied in dynamitic process identification based on RPROP and chaos optimization coupling algorithm.Proceedings of the Third International Conference Machine Learning and Cybernetics,Shanghai,August 2004:26-29
[51]王艳.秦玉平.张志强.唐政.刘伟江.一种改进的Elman神经网络算法.渤海大学学报(自然科学版)[J],2007,28(4):377-380
[52]方崇智.萧德云.过程辨识.第一版.清华大学出版社,1998.2
[53]余远俊 神经网络在非线性系统辨识中的应用 西南交通大学硕士学位论文2005:24
[54]Li-Hui Zhou,Pu Han,Song-Ming Jiao,Bi-Hua Lin.Feedforward neural networks using RPROP algorithm and its application in system identification.Proceedings of the First International Conference on Machine Learning and Cybernetics,Beijing,4-5 November 2002
[55]Martin Riedmiller,Heinrich Braun.A Direct Adaptive Method for Faster Backpropagation Learning:The RPROP Algorithm[C].Ruspini H,editor.Proceedings of the IEEE International Conference on Neural Networks (ICNN).San Francisco:1993:586-591
[56]V.Vapnik and Chervonenkis A Y.On the Uniform Convergence of Relative Frequencies of Events to their Probalities.Theory Probab Appl.1971,16920:264-280.
[57]V.Cherkassky and F.Mulier.Learning from Data:Concepts,Theory and Methods.NY:John Viley & Sons,1997.
[58]V.Vapnik.The Nature of Statistical Learning Theory.New York:Springer-Verlag,1995.
[59]V.VaPnik.Statistical Learning Theory.New York:John Wiley&Sons,Inc.,1998.