基于系统辨识的神经网络学习算法研究
|
摘要
本文对神经网络用于系统辨识的参数学习算法进行了系统的研究，简要介绍了神经网络系统辨识的基本原理，分析了线性神经网络在系统辨识中的应用，并将其用于直流调速系统建模。
BP网络是应用最广的一种网络，本文对BP算法进行了详细的推导，分析了其存在的缺陷；改进的BP算法虽然能在一定程度上提高学习速度，但对改善收敛效果，提高收敛精度，甚至摆脱局部极小，没有太大的积极作用。因此，作者将非线性规划的平行切线算法用于神经网络的权值学习，提出了一种具有快速学习算法的神经网络。
本文分析了一种动态补偿神经网络模型，模型的训练利用反向传播原理实现。采用该模型对非线性动态系统进行建模时，能显著提高建模精度，特别是在网络模型工作时，对新出现的输出误差具有动态补偿能力。
分析了动态递归神经网络系统辨识的参数学习算法。
直流调速系统应用范围广泛，传统的建模方法采用近似的线性化方法，忽略了各环节的非线性因素，这将使模型存在一定的误差，导致其所应用领域的系统性能降低。为了改变这一状况，有必要研究调速系统的非线性特性。本文采用基于RPE算法的神经网络对速度环进行动态辨识，仿真结果表明，神经网络能满意地建立直流调速系统的动态模型。
This paper makes a systematic research on parameter learning algorithm, which applies the neural network to system identification and modeling of D. C. speed-adjustable system. Besides, the basic principle of system identification using neural network and the application of linear neural network in it are briefly mentioned.
BP network is one of the most widely used ones. Standard BP algorithm is derived in details here, and its existent defects are also analyzed. Though modified BP algorithm can enhance learning rate to some extent, its active function is far from satisfactory on improving convergence effect and raising convergence accuracy and even geting rid of local extreme minimum value. This paper applies parallel tangents of nonlinear programming during the weights training of neural network and puts forward a neural network in view of fast learning algorithm.
A neural network model with dynamical compensating capability is analyzed. During the training of this network model, we apply the principle of dynamic error back-propagation. Using this model in nonlinear of dynamic systems modeling, the modeling accuracy can be significantly raised and the dynamic error can be effectively reduced especially during working of this network model.
The parameter learning algorithm of dynamic recurrent neural network based on system identification is analyzed.
D. C. speed-adjustable system is widely adopted. The traditional modeling approach adopts approximately linearized method and neglects nonlinear factors of links, which causes the model to produce certain error and leads to the decreasing of system performance in its usage. In this sense, it is advisable to study the nonlinear nature of D.C. speed-adjustable system. In this paper, the speed loop is identified dynamically
using the neural network based on RPE algorithm. The simulation results imply that the neural network can develop the dynamic model of D.C. speed-adjustable satisfactorily.
BP网络是应用最广的一种网络，本文对BP算法进行了详细的推导，分析了其存在的缺陷；改进的BP算法虽然能在一定程度上提高学习速度，但对改善收敛效果，提高收敛精度，甚至摆脱局部极小，没有太大的积极作用。因此，作者将非线性规划的平行切线算法用于神经网络的权值学习，提出了一种具有快速学习算法的神经网络。
本文分析了一种动态补偿神经网络模型，模型的训练利用反向传播原理实现。采用该模型对非线性动态系统进行建模时，能显著提高建模精度，特别是在网络模型工作时，对新出现的输出误差具有动态补偿能力。
分析了动态递归神经网络系统辨识的参数学习算法。
直流调速系统应用范围广泛，传统的建模方法采用近似的线性化方法，忽略了各环节的非线性因素，这将使模型存在一定的误差，导致其所应用领域的系统性能降低。为了改变这一状况，有必要研究调速系统的非线性特性。本文采用基于RPE算法的神经网络对速度环进行动态辨识，仿真结果表明，神经网络能满意地建立直流调速系统的动态模型。
This paper makes a systematic research on parameter learning algorithm, which applies the neural network to system identification and modeling of D. C. speed-adjustable system. Besides, the basic principle of system identification using neural network and the application of linear neural network in it are briefly mentioned.
BP network is one of the most widely used ones. Standard BP algorithm is derived in details here, and its existent defects are also analyzed. Though modified BP algorithm can enhance learning rate to some extent, its active function is far from satisfactory on improving convergence effect and raising convergence accuracy and even geting rid of local extreme minimum value. This paper applies parallel tangents of nonlinear programming during the weights training of neural network and puts forward a neural network in view of fast learning algorithm.
A neural network model with dynamical compensating capability is analyzed. During the training of this network model, we apply the principle of dynamic error back-propagation. Using this model in nonlinear of dynamic systems modeling, the modeling accuracy can be significantly raised and the dynamic error can be effectively reduced especially during working of this network model.
The parameter learning algorithm of dynamic recurrent neural network based on system identification is analyzed.
D. C. speed-adjustable system is widely adopted. The traditional modeling approach adopts approximately linearized method and neglects nonlinear factors of links, which causes the model to produce certain error and leads to the decreasing of system performance in its usage. In this sense, it is advisable to study the nonlinear nature of D.C. speed-adjustable system. In this paper, the speed loop is identified dynamically
using the neural network based on RPE algorithm. The simulation results imply that the neural network can develop the dynamic model of D.C. speed-adjustable satisfactorily.
引文
[1] 王伟．人工神经网络原理——入门与应用．北京航空航天大学出版社，1995
[2] 刘以成．21世纪的公路交通-智能车路系统．中国公路学报，1995，8(3)：15-17
[3] Mahmassani H.S., Pecta S., et al. Dynamic traffic assignment with multiple user classes for real-time ATIS/ATMS applications in large urban systems[A] Procedings of the Advanced Traffic Management Conference, Federal Highway Administration, VS Department of Transportation, Washington, D.C., 1993,91-114
[4] Merchant D.K. and Nemhauser G.L. A model and an algorithm for the dynamic traffic assignment problems. Transportation Science, 1978 a, 23(12): 183-199
[5] Merchant D.K. and Nemhauser G.L.. Optimality conditions for a dynamic traffic assigment model[J]. Transportation Science, 1978b, 23(12):200-207
[6] 宋轶民，张策，马文贵．基于Hopfield神经网络的线性系统参数辨识．控制理论与应用，2000，17(1)：121-124
[7] 丛爽．典型人工神经网络的结构、功能及其在智能系统中的应用．信息与控制，2001，30(2)：97-103
[8] R.P. Lippman. An introduction to computing with neural net. IEEE ASSP Magazine, 1989,4:4-22
[9] Sarkav D. Methods to Speed up Error BP Learning Algorithm. 计算机学报，1996，11:209-213
[10] A.G. Parlos, B. Fernandez, A.F. Atiyz & W.K. Tsai, An accelerated Learning Algorithm for Multilayer Perception Networks, IEEE Trans. On NN, 1994,5(3):493-497
[11] 邓志东，孙增圻．利用线性再励的自适应变步长快速BP算法．模式识别与人工智能，1993，6(4)：319-323
[12] Q.Hong & H.Zhenya. Variable Step BP Algorithm which Pruns Away Redundant Connections Dynamically. IJCNN, 1992,2:441-445
[13] U.V. Phansalkar & P. S. Sastry. Analysis of the Back-Propagation Algorithm with Momentim. IEEE Trans, 1994,5(3):505-506
[14] S. Roy & J. J. Shynk. Analysis of the Momentum LMS Algorithm. IEEE ISCAS, 1992,4(3):423-426
[15]王科俊，王克成．神经网络建模、预报与控制．哈尔滨工程大学出版社，1996
[16] R. Vitthal, P. Sunthar & C.H.D. Rao. The Generalized Proportional Integral Derivative Gradient Descent Back Propagation Algorithm. Neural Network, 1995,8(4):705-709
[17]邓志东，孙增圻，刘建伟．BP网络的PID型二阶快速学习算法．自动化学报，1995，21(1)：67-73
[18]何玉彬，李新忠．神经网络技术及应用．科学出版社，2000
[19]陈增强，袁著祉．神经网络建模在热膨胀螺栓形变测量中的应用．控制与决策，2001，16(1)，117-119
[20]王立红．具有快速学习算法的神经网络及其在系统辨识中的应用．辽宁工学院学报，2001，21(6)：24-25
[21]焦李成．神经网络计算．西安电子科技大学出版社，1993
[22]胡守仁．神经网络应用技术．国防科技大学出版社，1993
[23] K. P. Venugopal & A.S. Pandya. Alopex Algorithm for Training Multilayer Neural Networks. IEEE Int. Conf. On NN, 1991,21(2): 196-201
[24] D. J. Montana & L. Davis. Training Feedforward Network Using Genetic Algorithm. Int. J. Conf. On AI ,1989,32(5):762-767
[25] M. R. Azimi-Sadjadi & R. J. Lion. Fast Learning Process of Multilayer Neural Networks Using Resursive Least Squares Method. IEEE Trans. On SP, 1992,40(3):446-450
[26]谭永红．多层前向神经网络的RLS训练算法及其在辨识中的应用．控制理论与应用，1994，11(5)：594-599
[27]王正欧，林晨．一种前向神经网络快速学习算法及其在系统辨识中的应用．自动化学报，1997，23(6)；728-733
[28]阎平凡，张长水．人工神经网络与模拟进化计算。清华大学出版社，2000
[29]王旭东，邵惠鹤．RBF神经网络在非线性系统建模中的应用．控制理论与应用，1997,14(1)：59-66
[30]刘妹琴，沈轶，廖晓昕．一类新的RBF神经网络在非线性系统建模中的应用．控制与决策，2001，16(3)：278-281
[31]方崇智，萧德云．过程辨识．清华大学出版社，1998
[32]袁曾任．人工神经元网络及其应用。清华大学出版社，1999
[33]高小榕，杨福生．采用同伦BP算法进行多层前向神经网络的训练．计算机学报，1996，11：209-213
[34] D. E. Rumelhart, et al. Learning interal representation by error propagation in Paraller Distributed Processing. Cambridge, MA, 1986,1(1): 13-16
[35]杨大力，刘泽民．多层前向神经网络中BP算法的误差分析及改进的算法．电子学报，1995，23(1)：117-120
[36]刘培玉．应用最优控制。大连理工大学出版社，1990
[37] Hunt, K. J., et al. Neural networks for Control Systems-A Survey. Automatica, 1992,28:1083-1112
[38]张星昌．具有动态补偿能力的神经网络模型及其在动态系统建模中的应用．控制理论与应用，1996，13(6)：823-826
[39] Box, G. E. P. and Jenkins, G. M.. Time Series Analysis Forecasting and Control. Holden-Day inc, 1976
[40]陈伯时．电力拖动自动控制系统．机械工业出版社，1992
[41] Billing S.A.,et al. A Comparison of the Back-propagation and Resursive Prediction Error Algorithms for Training Neural Networks. Mechanical Systems and Signal Processing, 1991,233-255
[2] 刘以成．21世纪的公路交通-智能车路系统．中国公路学报，1995，8(3)：15-17
[3] Mahmassani H.S., Pecta S., et al. Dynamic traffic assignment with multiple user classes for real-time ATIS/ATMS applications in large urban systems[A] Procedings of the Advanced Traffic Management Conference, Federal Highway Administration, VS Department of Transportation, Washington, D.C., 1993,91-114
[4] Merchant D.K. and Nemhauser G.L. A model and an algorithm for the dynamic traffic assignment problems. Transportation Science, 1978 a, 23(12): 183-199
[5] Merchant D.K. and Nemhauser G.L.. Optimality conditions for a dynamic traffic assigment model[J]. Transportation Science, 1978b, 23(12):200-207
[6] 宋轶民，张策，马文贵．基于Hopfield神经网络的线性系统参数辨识．控制理论与应用，2000，17(1)：121-124
[7] 丛爽．典型人工神经网络的结构、功能及其在智能系统中的应用．信息与控制，2001，30(2)：97-103
[8] R.P. Lippman. An introduction to computing with neural net. IEEE ASSP Magazine, 1989,4:4-22
[9] Sarkav D. Methods to Speed up Error BP Learning Algorithm. 计算机学报，1996，11:209-213
[10] A.G. Parlos, B. Fernandez, A.F. Atiyz & W.K. Tsai, An accelerated Learning Algorithm for Multilayer Perception Networks, IEEE Trans. On NN, 1994,5(3):493-497
[11] 邓志东，孙增圻．利用线性再励的自适应变步长快速BP算法．模式识别与人工智能，1993，6(4)：319-323
[12] Q.Hong & H.Zhenya. Variable Step BP Algorithm which Pruns Away Redundant Connections Dynamically. IJCNN, 1992,2:441-445
[13] U.V. Phansalkar & P. S. Sastry. Analysis of the Back-Propagation Algorithm with Momentim. IEEE Trans, 1994,5(3):505-506
[14] S. Roy & J. J. Shynk. Analysis of the Momentum LMS Algorithm. IEEE ISCAS, 1992,4(3):423-426
[15]王科俊，王克成．神经网络建模、预报与控制．哈尔滨工程大学出版社，1996
[16] R. Vitthal, P. Sunthar & C.H.D. Rao. The Generalized Proportional Integral Derivative Gradient Descent Back Propagation Algorithm. Neural Network, 1995,8(4):705-709
[17]邓志东，孙增圻，刘建伟．BP网络的PID型二阶快速学习算法．自动化学报，1995，21(1)：67-73
[18]何玉彬，李新忠．神经网络技术及应用．科学出版社，2000
[19]陈增强，袁著祉．神经网络建模在热膨胀螺栓形变测量中的应用．控制与决策，2001，16(1)，117-119
[20]王立红．具有快速学习算法的神经网络及其在系统辨识中的应用．辽宁工学院学报，2001，21(6)：24-25
[21]焦李成．神经网络计算．西安电子科技大学出版社，1993
[22]胡守仁．神经网络应用技术．国防科技大学出版社，1993
[23] K. P. Venugopal & A.S. Pandya. Alopex Algorithm for Training Multilayer Neural Networks. IEEE Int. Conf. On NN, 1991,21(2): 196-201
[24] D. J. Montana & L. Davis. Training Feedforward Network Using Genetic Algorithm. Int. J. Conf. On AI ,1989,32(5):762-767
[25] M. R. Azimi-Sadjadi & R. J. Lion. Fast Learning Process of Multilayer Neural Networks Using Resursive Least Squares Method. IEEE Trans. On SP, 1992,40(3):446-450
[26]谭永红．多层前向神经网络的RLS训练算法及其在辨识中的应用．控制理论与应用，1994，11(5)：594-599
[27]王正欧，林晨．一种前向神经网络快速学习算法及其在系统辨识中的应用．自动化学报，1997，23(6)；728-733
[28]阎平凡，张长水．人工神经网络与模拟进化计算。清华大学出版社，2000
[29]王旭东，邵惠鹤．RBF神经网络在非线性系统建模中的应用．控制理论与应用，1997,14(1)：59-66
[30]刘妹琴，沈轶，廖晓昕．一类新的RBF神经网络在非线性系统建模中的应用．控制与决策，2001，16(3)：278-281
[31]方崇智，萧德云．过程辨识．清华大学出版社，1998
[32]袁曾任．人工神经元网络及其应用。清华大学出版社，1999
[33]高小榕，杨福生．采用同伦BP算法进行多层前向神经网络的训练．计算机学报，1996，11：209-213
[34] D. E. Rumelhart, et al. Learning interal representation by error propagation in Paraller Distributed Processing. Cambridge, MA, 1986,1(1): 13-16
[35]杨大力，刘泽民．多层前向神经网络中BP算法的误差分析及改进的算法．电子学报，1995，23(1)：117-120
[36]刘培玉．应用最优控制。大连理工大学出版社，1990
[37] Hunt, K. J., et al. Neural networks for Control Systems-A Survey. Automatica, 1992,28:1083-1112
[38]张星昌．具有动态补偿能力的神经网络模型及其在动态系统建模中的应用．控制理论与应用，1996，13(6)：823-826
[39] Box, G. E. P. and Jenkins, G. M.. Time Series Analysis Forecasting and Control. Holden-Day inc, 1976
[40]陈伯时．电力拖动自动控制系统．机械工业出版社，1992
[41] Billing S.A.,et al. A Comparison of the Back-propagation and Resursive Prediction Error Algorithms for Training Neural Networks. Mechanical Systems and Signal Processing, 1991,233-255