复杂系统的神经网络建模及仿真研究
|
摘要
神经网络对于辨识和逼近复杂的非线性系统有优越的性能，已经在工程领域得到广泛而成功的应用。国内外许多学者采用神经网络来建立生产过程的模型，以达到预报或优化的目的。
本文介绍了近年来得到广泛应用的两种前馈神经网络：BP网络和RBF网络，并给出了学习算法。文中归纳了前馈型神经网络具有逼近任意非线性映射能力的理论性的结论。简单介绍了钢材退火生产工艺，给出了基于神经网络的钢材退火生产过程的建模与优化模型。最后，作了用BP网络和RBF网络逼近非线性函数的仿真研究，并作了比较。针对BP网络学习速度慢，容易陷入局部极小点等问题，给出了几种改进方法，取得了较好的效果。文中对前馈网络中的隐层神经元个数的确定，学习样本的选取以及神经网络的泛化能力作了较详细的讨论。最后，采用一种直接方法步长加速法来寻找最优值。
Neural network is very effective to identify and approximate nonlinear dynamical system. It is applied widely and successfully in engineering area. Many researchers model process based neural network to predict or optimize yield.
This paper presents two kinds of forward neural network: BP (Back-Propagation) and RBF (Radial Basis Function), and the learning algorithms are also introduced . Then the concept of steel annealing is expounded. And neural-based solution of modeling and optimization of steel annealing process is given. Nonlinear function approximation using BP neural network and RBF neural network is simulated. We propose some modifications of the back-propagation algorithm to speed up the convergence rate and increase the possibility to escape local minima. In this paper, we discuss detailedly about how to determine the number of hidden layer of neural network, about how to select learning samples, about generalization of neural network. At last, the algorithm to find the optimal value is given.
本文介绍了近年来得到广泛应用的两种前馈神经网络：BP网络和RBF网络，并给出了学习算法。文中归纳了前馈型神经网络具有逼近任意非线性映射能力的理论性的结论。简单介绍了钢材退火生产工艺，给出了基于神经网络的钢材退火生产过程的建模与优化模型。最后，作了用BP网络和RBF网络逼近非线性函数的仿真研究，并作了比较。针对BP网络学习速度慢，容易陷入局部极小点等问题，给出了几种改进方法，取得了较好的效果。文中对前馈网络中的隐层神经元个数的确定，学习样本的选取以及神经网络的泛化能力作了较详细的讨论。最后，采用一种直接方法步长加速法来寻找最优值。
Neural network is very effective to identify and approximate nonlinear dynamical system. It is applied widely and successfully in engineering area. Many researchers model process based neural network to predict or optimize yield.
This paper presents two kinds of forward neural network: BP (Back-Propagation) and RBF (Radial Basis Function), and the learning algorithms are also introduced . Then the concept of steel annealing is expounded. And neural-based solution of modeling and optimization of steel annealing process is given. Nonlinear function approximation using BP neural network and RBF neural network is simulated. We propose some modifications of the back-propagation algorithm to speed up the convergence rate and increase the possibility to escape local minima. In this paper, we discuss detailedly about how to determine the number of hidden layer of neural network, about how to select learning samples, about generalization of neural network. At last, the algorithm to find the optimal value is given.
引文
[1] Chen T, Chen H. Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems, IEEE Trans. Neural Networks, 1995, 6(4):911-917.
[2] 王向东，基于神经网络的集成电路芯片生产过程建模与优化，中国科学院半导体研究所博士论文，2001
[3] 范建东，王唯一，荣亦诚，龚幼民，RBF神经网络应用于连铸漏钢预报，上海大学学报，2001，Vol．7 No．5，391～393
[4] 王中杰，柴天佑，邵诚，基于RBF神经网络的加热炉钢温预报模型，系统仿真学报，1999，Vol．11 No．3，181～185
[5] 柴天佑，谢书明，杜斌，任德祥，基于RBF神经网络的转炉炼钢终点预报，中国有色金属学报，1999，Vol．9 No．4，868～872
[6] W.S.McCulloch and W. Pitts, Bull. Math. Biophys, 1943, 5,115～133
[7] D.O.Hebb, The Organization of Behavior, Wiley, New York, 1949
[8] B.Widrow and M.Hoff, 1960 IRE WESCON Conr. Record, 96～104, Aug, 1960
[9] M.Minsky and S.Papert, Perceptrons, MIT Press, 1969
[10] Hopfield JJ. Neural networks and physical systems with emergent collective computing abilites, Press. NatL. Acad. Sci. USA, 1982, 79:2554～2558
[11] 黄德双，神经网络模式识别系统理论，第一版，北京：电子工业出版社，1996
[12] Peter L. Bartlett and Shai Ben-David, Hardness results for neural network approximation problems, Theoretical Computer Science, 2002, No.284, pp.53～66.
[13] J.Q.Gong, Bin Yao, Neural network adaptive robust control of nonlinear systems in semi-strict feedback form, Automatica, 2001, No.37, 1149～1160
[14] Jaime Arriagada, Pernilla Olausson, Azra Selimovic, Artificial neural network simulator for SOFC performance prediction, Journal of Power Sources , 2002, No. 112, 54～60
[15] 王守觉，多值和多阈值神经元及其网络组合与应用，电子学报，1996，Vol．24 No．5，1～6
[16] 王守觉，陈向东，曾玉娟，王向东，王戌靖，通用前馈网络及排序学习前向掩蔽模型在模式识别中的应用，电子学报，1998，Vol．26 No．8，1～6
[17] 张承福，对当前神经网络研究的几点看法，力学进展，1994，Vol．24 No．2，181～186
[18] 阎平凡，对多层前向神经网络研究的几点看法，自动化学报，1997，Vol．23 No．1，129～135
[19] Chen T, Chen H, Liu R W. Approximation capability in C(R) by multilayer feedforward networks and related problems, IEEE Trans. Neural Networks, 1995,6(1):25-30.
[20] Horinik K, Stinchcombe M, White H. Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks, Neural Networks, 1990, 3:551-560.
[21] Williamson R C, Helmke U. Existence and uniquences results for neural network approximation, IEEE Trans. Neural Networks, 1995, 6(1):2-13.
[22] Hornik K., Stinchcombe, M. and White H. Multi-layer feedforward networks are universal approximators. Neural Networks, vol. 2, pp. 359-366, 1989.
[23] Hornik K., Approximation capabilities of multilayer feedforward networks. Neural Networks, vol. 4 pp. 251-257, 1991.
[24] 阎平凡，对多层前向神经网络研究的进一步看法，电子学报，1999，Vol．27 No．5，82～85
[25] 张立明，人工神经网络的模型及其应用，第1版，上海：复旦大学出版社，1993，46～50
[26] 徐秉铮，张百灵，韦岗，神经网络理论与应用，第1版，广州：华南理工大学出版社，1994，250～350
[27] Mark Kon, Complexity of Neural Network Approximation with Limited Information: A Worst Case Approach, Journal of Complexity, 2001, No.17, 345～365
[28] 阎平凡，人工神经网络的容量、学习与计算复杂性，电子学报，1995，Vol．23 No．4，63～67
[29] 廖宁放，高稚允，BP神经网络用于函数逼近的最佳隐层结构，北京理工大学学报，1998，Vol．18 No．4，476～480
[30] 张立明，人工神经网络的模型及其应用，第1版，上海：复旦大学出版社，1993，46～50
[31] Jasmina Arifovic and Ramazan Gencay, Using genetic algorithms to select architecture of a feedforward artificial neural network, Physica, 2001, No.289, 574～594.
[32] 丛爽，宋瑞祥，钱镇，魏衡华，改进型B样条模糊神经网络，控制理论与应用，2001，Vol．18，No．2，277～281
[33] 朱明星，张德龙．RBF网络基函数中心选取算法的研究[J]．安徽大学学报(自然科学版)，2000，24(1)：72～78
[34] 王洪斌，杨香兰，王洪瑞，一种改进的RBF神经网络学习算法，系统工程
与电子技术，2002，Vol．24 No．6，103～105
[35] 刘妹琴，廖晓昕，RBF神经网络的一种鲁棒学习算法，华中理工大学学报，2000，Vol．28 No．2，8～10
[36] Marcelino Lazaro, Ignacio Santamara, Carlos Pantaleon , A new EM-based training algorithm for RBF networks, Neural Networks, 2002, No. 16, 69～77
[37] Iulian B. Ciocoiu, RBF networks training using a dual extended Kalman filter, Neurocomputing, 2002, No.48, 609～622
[38] 徐秉铮，张百灵，韦岗，神经网络理论与应用，第1版，广州：华南理工大学出版社，1994，250～350
[39] 阎平凡，多层前馈网络的泛化和学习问题。中国神经网络93年学术会议论文集，1993年，上册，68-77
[40] Shun-ichi Amari, Four Types of Learning Curves, Neural Computation, 4, 605～618(1992).
[41] D.B.Fogel, An Information Criterion for Optimal Neural Network Selection. IEEE Trans, on Neural Networks, 2, 490～497, 1991.
[42] Levin E. A Statistical Approach to Learning and Generalization in Layered Neural Network, Proc, IEEE, 78, 1568～1574(1990).
[43] Mukul A. Combining neural and conventional paradigms for modelling prediction and control, International Journal of Systems Secience, 1997, 28(1):65-81.
[44] 王向东，基于神经网络的集成电路芯片生产过程建模与优化，中国科学院半导体研究所博士论文，2001，30～35
[45] Vijaya G, Verma H K. Artificial Neural Network Based Wave Complex Detection in Electrcardiograms, International Journal of Systems Science, 1997, 28:125-132.
[46] 范建东，王唯一，荣亦诚，龚幼民，RBF神经网络应用于连铸漏钢预报，上海大学学报，2001，Vol．7 No．5，391～393
[47] 王中杰，柴天佑，邵诚，基于RBF神经网络的加热炉钢温预报模型，系统仿真学报，1999，Vol．11 No．3，181～185
[48] 柴天佑，谢书明，杜斌，任德祥，基于RBF神经网络的转炉炼钢终点预报，中国有色金属学报，1999，Vol．9 No．4，868～872
[49] Changqing Dong, Baosheng Jin, Daji Li, Predicting the heating value of MSW with a feedforward neural network, Waste Management, 2002, No.23, 103～106
[50] 李鑫滨，杨景明，丁喜峰，基于递推K-均值聚类算法的RBF神经网络及其在系统辨识中的应用，燕山大学学报，1999，Vol．23 No．4，363～366
[51] 刘妹琴，沈轶，廖晓昕，一类新的RBF神经网络在非线性系统建模中的应用，控制与决策，2001，Vol．16 No．3，277～281
[52] 李艳君，吴铁军，赵明旺，一种新的RBF神经网络非线性动态系统建模方法，系统工程理论与实践，2001，No．3，64～69
[53] Yinwei L, Sundarajan N, Saratchandran P. Ideification of time-varying nonlinear systems using minimal radial basis function neural network, IEEE Proc. Control Theory Appl. 1997, 144(2):202-208.
[54] 李泉华，热处理实用技术，第1版，北京；机械工业出版社，2000
[55] 孙大涌，热处理手册，第3版，北京：机械工业出版社，2001
[56] 张智星，孙春在等译，神经-模糊和软计算，第1版，西安：西安交通大学出版社，2000，159～178
[57] Nam Mai-Duy, Thanh Tran-Cong, Approximation of function and its derivatives using radial basis function networks, Applied Mathematical Modeling, 2002, No.27, 197～220
[58] M.B.Stinchcombe, Neural network approximation of continuous functionals and continuous functions on compactifications, Neural Networks, 1999, No.12, pp.467～477.
[59] T.Nguyen-Thien and T. Tran-Cong, Approximation of functions and their derivatives: A neural network implementation with applications, Applied Mathematical Modelling, 1999, No.23,687～704.
[60] Shin Suzuki, Constructive function-approximation by three-layer artificial neural networks, Neural Networks, 1998, No. 11, 1049～1058.
[61] Franco Scarselli and Ah Chung Tsoi, Universal Approximation Using Feedforward Neural Networks: A Survey of Some Existing Methods, and Some New Results. Neural Networks, 1998, Vol. 11, No. 1, pp. 15～37.
[62] Xin Li, On simultaneous approximations by radial basis function neural networks, Applied Mathematics and Computation, 1998, No.95, 75～89.
[63] C.Drioli, D.Rocchesso, Orthogonal least squares algorithm for the approximation of a map and its derivatives with a RBF network, Signal Processing, 2002, No.83,283～296
[64] Noboru. M. An integral representation of functions using three-layered networks and their approximation bounds. Neural Networks, 1996, 9(6): 947～956
[65] An-Chyau Huang, Yi-Fang Chiang, Function Approximation Using Serial Input Neural, Neurocomputing, 2002, No.47, pp.85～101.
[66] Nicolangelo Iannella, Andrew D.Back, A spiking neural network architecture
for nonlinear function approximation, Neural Networks, 2001, No. 14, 933～939
[67] 刘昆，颜钢锋，基于模糊RBF神经网络的函数逼近，计算机工程，2001，Vol．27 No．2，70～71
[68] 段晓君，神经网络的函数逼近能力分析，模糊系统与数学，1998，Vol．12 No．4，79～84
[69] 蔡宣三，最优化与最优控制，第一版，北京：清华大学出版社，1982
[70] 勒蕃，神经网络与神经计算机：原理．应用，第一版，成都：西南交通大学出版社，1991，125～137
[71] 苏兰，吴磊，张勇，粗集神经网络系统及其应用，电子科技大学学报，2002，Vol．31 No．4，426～428
[72] 李仁璞，王正欧，基于粗集理论和神经网络结合的数据挖掘新方法，情报学报，2002，Vol．21 No．6，674～679
[73] 张兆礼，孙圣和，粗神经网络及其在数据融合中的应用，控制与决策，2001，Vol．16 No．1，76～79
[74] 粟塔山，最优化计算原理与算法程序设计，第一版，北京：国防科技大学出版社，2001，83～98
[2] 王向东，基于神经网络的集成电路芯片生产过程建模与优化，中国科学院半导体研究所博士论文，2001
[3] 范建东，王唯一，荣亦诚，龚幼民，RBF神经网络应用于连铸漏钢预报，上海大学学报，2001，Vol．7 No．5，391～393
[4] 王中杰，柴天佑，邵诚，基于RBF神经网络的加热炉钢温预报模型，系统仿真学报，1999，Vol．11 No．3，181～185
[5] 柴天佑，谢书明，杜斌，任德祥，基于RBF神经网络的转炉炼钢终点预报，中国有色金属学报，1999，Vol．9 No．4，868～872
[6] W.S.McCulloch and W. Pitts, Bull. Math. Biophys, 1943, 5,115～133
[7] D.O.Hebb, The Organization of Behavior, Wiley, New York, 1949
[8] B.Widrow and M.Hoff, 1960 IRE WESCON Conr. Record, 96～104, Aug, 1960
[9] M.Minsky and S.Papert, Perceptrons, MIT Press, 1969
[10] Hopfield JJ. Neural networks and physical systems with emergent collective computing abilites, Press. NatL. Acad. Sci. USA, 1982, 79:2554～2558
[11] 黄德双，神经网络模式识别系统理论，第一版，北京：电子工业出版社，1996
[12] Peter L. Bartlett and Shai Ben-David, Hardness results for neural network approximation problems, Theoretical Computer Science, 2002, No.284, pp.53～66.
[13] J.Q.Gong, Bin Yao, Neural network adaptive robust control of nonlinear systems in semi-strict feedback form, Automatica, 2001, No.37, 1149～1160
[14] Jaime Arriagada, Pernilla Olausson, Azra Selimovic, Artificial neural network simulator for SOFC performance prediction, Journal of Power Sources , 2002, No. 112, 54～60
[15] 王守觉，多值和多阈值神经元及其网络组合与应用，电子学报，1996，Vol．24 No．5，1～6
[16] 王守觉，陈向东，曾玉娟，王向东，王戌靖，通用前馈网络及排序学习前向掩蔽模型在模式识别中的应用，电子学报，1998，Vol．26 No．8，1～6
[17] 张承福，对当前神经网络研究的几点看法，力学进展，1994，Vol．24 No．2，181～186
[18] 阎平凡，对多层前向神经网络研究的几点看法，自动化学报，1997，Vol．23 No．1，129～135
[19] Chen T, Chen H, Liu R W. Approximation capability in C(R) by multilayer feedforward networks and related problems, IEEE Trans. Neural Networks, 1995,6(1):25-30.
[20] Horinik K, Stinchcombe M, White H. Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks, Neural Networks, 1990, 3:551-560.
[21] Williamson R C, Helmke U. Existence and uniquences results for neural network approximation, IEEE Trans. Neural Networks, 1995, 6(1):2-13.
[22] Hornik K., Stinchcombe, M. and White H. Multi-layer feedforward networks are universal approximators. Neural Networks, vol. 2, pp. 359-366, 1989.
[23] Hornik K., Approximation capabilities of multilayer feedforward networks. Neural Networks, vol. 4 pp. 251-257, 1991.
[24] 阎平凡，对多层前向神经网络研究的进一步看法，电子学报，1999，Vol．27 No．5，82～85
[25] 张立明，人工神经网络的模型及其应用，第1版，上海：复旦大学出版社，1993，46～50
[26] 徐秉铮，张百灵，韦岗，神经网络理论与应用，第1版，广州：华南理工大学出版社，1994，250～350
[27] Mark Kon, Complexity of Neural Network Approximation with Limited Information: A Worst Case Approach, Journal of Complexity, 2001, No.17, 345～365
[28] 阎平凡，人工神经网络的容量、学习与计算复杂性，电子学报，1995，Vol．23 No．4，63～67
[29] 廖宁放，高稚允，BP神经网络用于函数逼近的最佳隐层结构，北京理工大学学报，1998，Vol．18 No．4，476～480
[30] 张立明，人工神经网络的模型及其应用，第1版，上海：复旦大学出版社，1993，46～50
[31] Jasmina Arifovic and Ramazan Gencay, Using genetic algorithms to select architecture of a feedforward artificial neural network, Physica, 2001, No.289, 574～594.
[32] 丛爽，宋瑞祥，钱镇，魏衡华，改进型B样条模糊神经网络，控制理论与应用，2001，Vol．18，No．2，277～281
[33] 朱明星，张德龙．RBF网络基函数中心选取算法的研究[J]．安徽大学学报(自然科学版)，2000，24(1)：72～78
[34] 王洪斌，杨香兰，王洪瑞，一种改进的RBF神经网络学习算法，系统工程
与电子技术，2002，Vol．24 No．6，103～105
[35] 刘妹琴，廖晓昕，RBF神经网络的一种鲁棒学习算法，华中理工大学学报，2000，Vol．28 No．2，8～10
[36] Marcelino Lazaro, Ignacio Santamara, Carlos Pantaleon , A new EM-based training algorithm for RBF networks, Neural Networks, 2002, No. 16, 69～77
[37] Iulian B. Ciocoiu, RBF networks training using a dual extended Kalman filter, Neurocomputing, 2002, No.48, 609～622
[38] 徐秉铮，张百灵，韦岗，神经网络理论与应用，第1版，广州：华南理工大学出版社，1994，250～350
[39] 阎平凡，多层前馈网络的泛化和学习问题。中国神经网络93年学术会议论文集，1993年，上册，68-77
[40] Shun-ichi Amari, Four Types of Learning Curves, Neural Computation, 4, 605～618(1992).
[41] D.B.Fogel, An Information Criterion for Optimal Neural Network Selection. IEEE Trans, on Neural Networks, 2, 490～497, 1991.
[42] Levin E. A Statistical Approach to Learning and Generalization in Layered Neural Network, Proc, IEEE, 78, 1568～1574(1990).
[43] Mukul A. Combining neural and conventional paradigms for modelling prediction and control, International Journal of Systems Secience, 1997, 28(1):65-81.
[44] 王向东，基于神经网络的集成电路芯片生产过程建模与优化，中国科学院半导体研究所博士论文，2001，30～35
[45] Vijaya G, Verma H K. Artificial Neural Network Based Wave Complex Detection in Electrcardiograms, International Journal of Systems Science, 1997, 28:125-132.
[46] 范建东，王唯一，荣亦诚，龚幼民，RBF神经网络应用于连铸漏钢预报，上海大学学报，2001，Vol．7 No．5，391～393
[47] 王中杰，柴天佑，邵诚，基于RBF神经网络的加热炉钢温预报模型，系统仿真学报，1999，Vol．11 No．3，181～185
[48] 柴天佑，谢书明，杜斌，任德祥，基于RBF神经网络的转炉炼钢终点预报，中国有色金属学报，1999，Vol．9 No．4，868～872
[49] Changqing Dong, Baosheng Jin, Daji Li, Predicting the heating value of MSW with a feedforward neural network, Waste Management, 2002, No.23, 103～106
[50] 李鑫滨，杨景明，丁喜峰，基于递推K-均值聚类算法的RBF神经网络及其在系统辨识中的应用，燕山大学学报，1999，Vol．23 No．4，363～366
[51] 刘妹琴，沈轶，廖晓昕，一类新的RBF神经网络在非线性系统建模中的应用，控制与决策，2001，Vol．16 No．3，277～281
[52] 李艳君，吴铁军，赵明旺，一种新的RBF神经网络非线性动态系统建模方法，系统工程理论与实践，2001，No．3，64～69
[53] Yinwei L, Sundarajan N, Saratchandran P. Ideification of time-varying nonlinear systems using minimal radial basis function neural network, IEEE Proc. Control Theory Appl. 1997, 144(2):202-208.
[54] 李泉华，热处理实用技术，第1版，北京；机械工业出版社，2000
[55] 孙大涌，热处理手册，第3版，北京：机械工业出版社，2001
[56] 张智星，孙春在等译，神经-模糊和软计算，第1版，西安：西安交通大学出版社，2000，159～178
[57] Nam Mai-Duy, Thanh Tran-Cong, Approximation of function and its derivatives using radial basis function networks, Applied Mathematical Modeling, 2002, No.27, 197～220
[58] M.B.Stinchcombe, Neural network approximation of continuous functionals and continuous functions on compactifications, Neural Networks, 1999, No.12, pp.467～477.
[59] T.Nguyen-Thien and T. Tran-Cong, Approximation of functions and their derivatives: A neural network implementation with applications, Applied Mathematical Modelling, 1999, No.23,687～704.
[60] Shin Suzuki, Constructive function-approximation by three-layer artificial neural networks, Neural Networks, 1998, No. 11, 1049～1058.
[61] Franco Scarselli and Ah Chung Tsoi, Universal Approximation Using Feedforward Neural Networks: A Survey of Some Existing Methods, and Some New Results. Neural Networks, 1998, Vol. 11, No. 1, pp. 15～37.
[62] Xin Li, On simultaneous approximations by radial basis function neural networks, Applied Mathematics and Computation, 1998, No.95, 75～89.
[63] C.Drioli, D.Rocchesso, Orthogonal least squares algorithm for the approximation of a map and its derivatives with a RBF network, Signal Processing, 2002, No.83,283～296
[64] Noboru. M. An integral representation of functions using three-layered networks and their approximation bounds. Neural Networks, 1996, 9(6): 947～956
[65] An-Chyau Huang, Yi-Fang Chiang, Function Approximation Using Serial Input Neural, Neurocomputing, 2002, No.47, pp.85～101.
[66] Nicolangelo Iannella, Andrew D.Back, A spiking neural network architecture
for nonlinear function approximation, Neural Networks, 2001, No. 14, 933～939
[67] 刘昆，颜钢锋，基于模糊RBF神经网络的函数逼近，计算机工程，2001，Vol．27 No．2，70～71
[68] 段晓君，神经网络的函数逼近能力分析，模糊系统与数学，1998，Vol．12 No．4，79～84
[69] 蔡宣三，最优化与最优控制，第一版，北京：清华大学出版社，1982
[70] 勒蕃，神经网络与神经计算机：原理．应用，第一版，成都：西南交通大学出版社，1991，125～137
[71] 苏兰，吴磊，张勇，粗集神经网络系统及其应用，电子科技大学学报，2002，Vol．31 No．4，426～428
[72] 李仁璞，王正欧，基于粗集理论和神经网络结合的数据挖掘新方法，情报学报，2002，Vol．21 No．6，674～679
[73] 张兆礼，孙圣和，粗神经网络及其在数据融合中的应用，控制与决策，2001，Vol．16 No．1，76～79
[74] 粟塔山，最优化计算原理与算法程序设计，第一版，北京：国防科技大学出版社，2001，83～98