多输出神经元模型的多层前向神经网络及其应用
|
摘要
神经网络是高度非线性动力学系统,又是自适应自组织系统,可用来描述认知、决策及控制等的智能行为,使得智能的认识和模拟成为神经网络理论研究的一个重要方面;同时,神经网络理论又成为信息并行处理的基础,PDP(并行分布处理)成为80年代中后期的一个研究新热点,它进一步拓展了计算概念的内涵,使神经计算、进化计算成为新的研究领域。引起了包括计算机科学、人工智能、认识科学、信息科学、微电子科学、自动控制与机器人、脑神经科学等学科与领域的科学家的巨大热情和广泛兴趣。
然而,传统神经元的M-P模型用连接权值和非线性激活函数分别模仿神经元的突触和细胞体的作用。在训练过程中,权值是可以调节的,但激活函数不能调节,是事先确定的。这种模型过于简单,其性能受到极大的限制。基于此,人们提出激活函数可调的神经元模型(TAF)以及由此模型构成的多层前向神经网络(TAF-MFNN),与传统网络相比,采用这种新型的神经元模型来解决某些问题时,所需要的网络会更为简单,网络性能会更好,推广泛化能力也会更强。但是,用来训练这种网络的BP算法收敛速度慢,容易陷入局部极小。
众所周知,RLS算法具有收敛速度快,收敛精度高等优点,本文将激活函数可调的神经网络进行变形,使得变形后的网络和原网络等价,给出了网络学习的快速算法。通过仿真实验,改进的算法大大提高网络的收敛速度和精度。在此基础上,本文将激活函数可调的神经网络进行改进,提出了多输出神经元模型(MO)以及由此模型构成的多层前向网络(MO-MFNN),并给出了网络的RLS算法、LM算法、LMAM算法和OLMAM算法。通过仿真实验,得到结论:在训练样本不太大时,选择利用LM算法、LMAM算法和OLMAM算法来训练网络;在训练样本很大时,选择利用RLS算法来训练网络。
在神经网络的训练中大多用如下均方误差函数(MS)作为网络的训练目标函数,直到现在,大多数快速算法都是基于MS基础上得到的。但是MS有两个主要缺点,一方面MS误差函数面存在很多的次最优解,在网络的训练过程中,易于陷
入这些次最优解;另一方面,MS误差函数是一个能满足各种不同方面应用的通用判决标准,利用MS误差函数训练网络容易出现过拟和现象。为了提高网络的性能,如降低测试误差和提高网络的泛化能力,在一些特殊的应用中,需考虑额外的假设和启发性的信息。其中一个利用先验知识的技巧是正则化方法,即构造一个正则化的目标函数。本文研究了MO-MFNN带正则化因子的学习算法,仿真实验表明,利用MO-MFNN可以减小计算复杂度和存储量。
神经网络具有并行性和自然容错性的能力,非常适合于非线性系统的实时自适应控制。到目前为止,还没有一种较为成熟的快速在线学习算法,而快速在线学习算法是实时控制的关键。多输出神经元模型的多层前向神经网络具有极大的网络容量,泛化能力强,网络结构简单,待定参数少,在线学习速度快,仿真实验表明,利用MO-MFNN进行非线性系统控制可以大大减小CPU时间,使其应用实时控制成为可能。
Neural networks is a high nonlinear dynamical and adaptive and self –organizational system. It can be used to describe the intelligent activation of cognition, decision and control e.t., which makes the intelligent cognition and simulation become the major facet of neural networks theoretic research. At the same time, neural networks becomes the bedrock of information parallel distributed processing (PDP). PDP becomes a new hot domain in the medium and upper term of 80’s last century. It ulteriorly extends the connotation of computation, which makes neural computation and evolution computation become a new research domain. It arouses enormous passion and broad interesting of scientist including computer science, artificial intelligent, cognition science, information science, micro-electronics, automatic control and robot, brain neurology e.t..
However, the traditional M-P neuron model which used connective weights and a nonlinear activation function simulate the operation of neural's synapse and soma respectively. During the training process, weights are tunable while the function is settled beforehand. Obviously this model is a simplified one comparing with that of the biology neural. Thus its capability is limited. Based on this, a tunable activation function neuron model (TAF) and a multilay feedforward neural networks with this neural model (TAF-MFNN) are presented. Compared to the traditional multilayer feedforward neural network (MFNN), the TAF-MFNN can deal with more difficult problems easily. Also it can simplify network's architecture and achieve excellent network performance and generalization capability. However, the speed of convergence of BP algorithm which is used to train the networks is slow. The other shortcoming of this algorithm is that it is prone to get into a local optimum.
It is well known, The RLS algorithm uses a modified BP algorithm to minimize the mean squared error between the desired output and the actual output with respect to the summation. Therefore, in our research, we propose to transform the architecture of the TAF-MFNN and enable it with a faster learning algorithm. The modified neural networks is equilvalent to the original networks. By simulations, the modified algorithm can improve the convergent speed and accuracy. Based on this, we have ameliorated the TAF model and presented a new neural networks with multi-output neural model (MO-
MFNN). The algorithms such as RLS algorithm, LM algorithm, LMAM algorithm and OLMAM algorithm are used to train MO-MFNN. It is obtained the conclusion: when the training samples is small, LM algorithm or LMAM algorithm or OLMAM algorithm is selected to train the networks; When the training sample is very large, RLS algorithm is used to train the networks.
The mean squared error function is used extensively in the training of backpropagation neural networks. Until now, most of the fast learning algorithms were derived based on the MS error function. Despite the popularity of the MS error function, there are two main shortcomings in applying those MS error based algorithms for general applications. On the one hand, there are many sub-optimal solutions on the MS error surface. The networks training may easily stall because of being stuck in one of the sub-optimal solutions. On the other hand, the MS error function, in general, is a universal objective function to cater for harsh criteria of different applications. However, there is a common view that different applications may emphasize on different aspects. To have an optimal performance such as a low training error and high generalization capability, additional assumptions and heuristic information on a particular application have to been included. One of the technique to absorb the a priori knowledge is regularization. Namely, a regularized index function is constructed. The learning algorithms for MO-MFNN with regularization are researched in this paper. By simulations, it is shows that it can decrease the computation complexity and storage by using of MO-MFNN.
Neural networ
然而,传统神经元的M-P模型用连接权值和非线性激活函数分别模仿神经元的突触和细胞体的作用。在训练过程中,权值是可以调节的,但激活函数不能调节,是事先确定的。这种模型过于简单,其性能受到极大的限制。基于此,人们提出激活函数可调的神经元模型(TAF)以及由此模型构成的多层前向神经网络(TAF-MFNN),与传统网络相比,采用这种新型的神经元模型来解决某些问题时,所需要的网络会更为简单,网络性能会更好,推广泛化能力也会更强。但是,用来训练这种网络的BP算法收敛速度慢,容易陷入局部极小。
众所周知,RLS算法具有收敛速度快,收敛精度高等优点,本文将激活函数可调的神经网络进行变形,使得变形后的网络和原网络等价,给出了网络学习的快速算法。通过仿真实验,改进的算法大大提高网络的收敛速度和精度。在此基础上,本文将激活函数可调的神经网络进行改进,提出了多输出神经元模型(MO)以及由此模型构成的多层前向网络(MO-MFNN),并给出了网络的RLS算法、LM算法、LMAM算法和OLMAM算法。通过仿真实验,得到结论:在训练样本不太大时,选择利用LM算法、LMAM算法和OLMAM算法来训练网络;在训练样本很大时,选择利用RLS算法来训练网络。
在神经网络的训练中大多用如下均方误差函数(MS)作为网络的训练目标函数,直到现在,大多数快速算法都是基于MS基础上得到的。但是MS有两个主要缺点,一方面MS误差函数面存在很多的次最优解,在网络的训练过程中,易于陷
入这些次最优解;另一方面,MS误差函数是一个能满足各种不同方面应用的通用判决标准,利用MS误差函数训练网络容易出现过拟和现象。为了提高网络的性能,如降低测试误差和提高网络的泛化能力,在一些特殊的应用中,需考虑额外的假设和启发性的信息。其中一个利用先验知识的技巧是正则化方法,即构造一个正则化的目标函数。本文研究了MO-MFNN带正则化因子的学习算法,仿真实验表明,利用MO-MFNN可以减小计算复杂度和存储量。
神经网络具有并行性和自然容错性的能力,非常适合于非线性系统的实时自适应控制。到目前为止,还没有一种较为成熟的快速在线学习算法,而快速在线学习算法是实时控制的关键。多输出神经元模型的多层前向神经网络具有极大的网络容量,泛化能力强,网络结构简单,待定参数少,在线学习速度快,仿真实验表明,利用MO-MFNN进行非线性系统控制可以大大减小CPU时间,使其应用实时控制成为可能。
Neural networks is a high nonlinear dynamical and adaptive and self –organizational system. It can be used to describe the intelligent activation of cognition, decision and control e.t., which makes the intelligent cognition and simulation become the major facet of neural networks theoretic research. At the same time, neural networks becomes the bedrock of information parallel distributed processing (PDP). PDP becomes a new hot domain in the medium and upper term of 80’s last century. It ulteriorly extends the connotation of computation, which makes neural computation and evolution computation become a new research domain. It arouses enormous passion and broad interesting of scientist including computer science, artificial intelligent, cognition science, information science, micro-electronics, automatic control and robot, brain neurology e.t..
However, the traditional M-P neuron model which used connective weights and a nonlinear activation function simulate the operation of neural's synapse and soma respectively. During the training process, weights are tunable while the function is settled beforehand. Obviously this model is a simplified one comparing with that of the biology neural. Thus its capability is limited. Based on this, a tunable activation function neuron model (TAF) and a multilay feedforward neural networks with this neural model (TAF-MFNN) are presented. Compared to the traditional multilayer feedforward neural network (MFNN), the TAF-MFNN can deal with more difficult problems easily. Also it can simplify network's architecture and achieve excellent network performance and generalization capability. However, the speed of convergence of BP algorithm which is used to train the networks is slow. The other shortcoming of this algorithm is that it is prone to get into a local optimum.
It is well known, The RLS algorithm uses a modified BP algorithm to minimize the mean squared error between the desired output and the actual output with respect to the summation. Therefore, in our research, we propose to transform the architecture of the TAF-MFNN and enable it with a faster learning algorithm. The modified neural networks is equilvalent to the original networks. By simulations, the modified algorithm can improve the convergent speed and accuracy. Based on this, we have ameliorated the TAF model and presented a new neural networks with multi-output neural model (MO-
MFNN). The algorithms such as RLS algorithm, LM algorithm, LMAM algorithm and OLMAM algorithm are used to train MO-MFNN. It is obtained the conclusion: when the training samples is small, LM algorithm or LMAM algorithm or OLMAM algorithm is selected to train the networks; When the training sample is very large, RLS algorithm is used to train the networks.
The mean squared error function is used extensively in the training of backpropagation neural networks. Until now, most of the fast learning algorithms were derived based on the MS error function. Despite the popularity of the MS error function, there are two main shortcomings in applying those MS error based algorithms for general applications. On the one hand, there are many sub-optimal solutions on the MS error surface. The networks training may easily stall because of being stuck in one of the sub-optimal solutions. On the other hand, the MS error function, in general, is a universal objective function to cater for harsh criteria of different applications. However, there is a common view that different applications may emphasize on different aspects. To have an optimal performance such as a low training error and high generalization capability, additional assumptions and heuristic information on a particular application have to been included. One of the technique to absorb the a priori knowledge is regularization. Namely, a regularized index function is constructed. The learning algorithms for MO-MFNN with regularization are researched in this paper. By simulations, it is shows that it can decrease the computation complexity and storage by using of MO-MFNN.
Neural networ
引文
McCulloch .W S,Pitts W. A Logical Calculus of the Ideas Immanet in Nervous Activity .Bulletin of Mathematical Biophysics, 1943,10(5):115~133.
Hebb D O.The Organization of Behavior. New York:Wiley,1949.
Eccles J C..Cholinergic and Inhibitory Synapses in A Pathway from Motor-axon Collaterals to Motorneurones . Journal Physiology,1954, 12(6): 524.
Rosenblatt F. The Perception: A Probabilistic Model for Information strorage and Organization in the Brain . Psychological Review, 1958, 23(68): 386~408.
Widrow B, Hoff M E, Adaptive Switching Circuits. 1960 IRE WESCON convention record: part 4. Computers: Man-machine Systems, Losangeles: 96~104.
Grossberg S , On the Serial Learning of Lists. Bio-science, 1969, 17(4): 201~253.
Grossberg S , Some Networks that Can Learn, Remember and Reproduce any Number of Compialted Space-time Patterns, II,stud Applied Mathematics 1970, 34(49):135~166.
Willshaw D J, Buneman O P, Longuest-higgins. HC Nature, 1969, 22(2):960.
Nilsson N J, Learning Machines: Foundations of Trainable Pattern Classifying Systems. McGraw-hill, New York, 1965.
Stein R B , Leung K V, Mangeron D, Oguztoreli M N , Improved Neuraonal models for Studying Neural Networks Kybernetik, 1974, 19(15): 1~9.
Heiden U. Ander, Existence of Periodic Solutions of a Nerve Equation. Biology Cybernitics, 1976, 23(21): 37~39.
Hopfield J J. Neural Networks and physical systems with emergent collective computational abilities. Proc Nat Acad Sci, 1982, 79: 2554~2558.
Marr D, Vision [M]. San Francisco: WHF Freeman, 1982.
Kirkpatrick S, Gellat K C D, Veechi M P , Optimization by Simulated Annealing , Science, 1983, 220(4598): 671~681.
Hinton G E , Sejuowshi T J, Ackley D H, Boltzmann Machines: Cotraint Satisfaction Networks that Learn. Carnegiemellon University, Tech, Report CMU-CS-84-119. 1984.
Piggio T, Analog model of Computation for III-posed Problems of Early Vision . Artificial Intelligence Lab Memo, 783, MIT, 1984.
Poggio T, Computational Vision and Regularization Theory. Nature (Lond), 1985, 27(3): 314~319.
Chua L O , Yand L. Celluar Neural networks: Theory. IEEE Transaction on Circuits and Systems, 1988, 35(10): 1257~1272.
Chua L O , Yand L, Celluar Neural Networks: Application, IEEE Transaction on Circuits and Systems, 1988, 35(10): 1273~1290.
Kosko B. Adaptive Bidirectional Associative Memories. Applied Optical, 1987, 26 (23): 4647~4680.
Kosko B, Bidirectional Associative Memories. IEEE Transaction On Man, System and Cybernitics . 1988, 18(1): 49~59.
廖晓昕.细胞神经网络的数学理论 (I) 、(II).中国科学 (A辑 ) , 1994, 24 (9): 902~910: 1037~1046.
赵杰煜,反馈随机二元神经网络,中国科学(E辑),2001, 31 (5): 471~480.
舒怀林,PID神经元和PID神经网络分析,见:1998中国控制会议论文集,长沙:国防科技大学出版社,1998,607~613.
舒怀林,基于PID神经网络的非线性时变系统辨识,自动化学报,2002,28(3),475~476.
刘普寅,一种新的模糊神经网络及其逼近能力,中国科学(E辑),2002, 32 (5): 76~86.
Qinghua Zhang and Albert Benveniste, Wavelet Networks, IEEE Transaction on Neural Networks, 1992, 3( 6). 889~898.
P.S. Sastry, G. Santharam and K. P. Unnikrishnan, Memory Neuron Networks for Identification and Control of Dynamical systems. IEEE Transaction on Neural Networks, 1994, 5( 2): 306~319
Jenkins B K, A R Tanguay Jr, Optical Architectures for Neural Network Implementation, Handbook of Neural Computing and Neural Networks. MIT Press, Boston, 1995:673~677.
McAulay A D, Wand J, Ma C, Optical Heteroassciative Memory Using Spatial Light Rebroadcasters. Applied Optical,1990, 29(14): 2067~2073.
阮昊 ,陈述春 ,戴凤妹 ,千福熹.利用电子俘获材料实现IPA神经网络模.光学学报 , 1997, 17(6):766~771.
Narendra K, Parthasarathy K, Identification and Control of Dynamical System Using Neural Networks. IEEE Transaction On Neural Networks, March 1990,1(1) :4~27.
戴先中 ,刘军 ,冯纯伯.连续非线性系统的神经网络α阶逆系统控制方法.自动化学报 ,1998, 24(4): 463~468.
Miller W T, Real time Application of Neural Networks for Sensor based Control of Robots. With Vision. IEEE Transaction On Man, System and Cybernitics 1989, 23(19): 825~831.
Bulsari A, Some Analytical Solutions to the General Approximation Problem for Feedforword Neural Networks. Neural Networks, 1993, (6) :991-996.
Aibara K, Chaotic Neural Networks. Physical Letter A, 1990, 144 (6,7):334~3340.
Inoue M, Nagayoshi A., A Chaos Neuro-computer. Physical Letter A, 1991, 158(8): 373~376.
Inoue M, Nakamoto K., Epilepsyina Chaos Neuro-computer Model. SPIEVOL, 236, Chaos in Biology and Medicine, 1993, 77~84.
Satoru Iaske, On neural approximation of fuzzy system. In : Proceedings of INCNN’92, Vol. 1, New York: IEEE, 1992. 1263~1268.
Jokinen Petri A. On the relations between radial basis function networks and fuzzy system. In: Proceeding of INCNN’29, Vol. 1, New York: IEEE, 1992, 1220~1225.
Okada Hiroyuki, Initializing multilayer neural networks with fuzzy logic. In: Proceeding of INCNN’92, Vol. 1, 1239~1244.
Cai Yaling, Hon Keung Kwan, A fuzzy neural networks with fuzzy classification. In: Proceedings of ASMESCI’94, Wuhan: Process of Huazhong University of Science and Technology, 1994: 894~899.
B E. Using spectral techniques for improved performance in ANN, Proc IEEE, 1993. 500~505.
Lee S, Kil R M. A Gaussian potential function network with hierarchnically self-organizing learning, Neural Networks, 1991, 4(7):207~224.
Stork D. G, Allen J D ,et al, How to solve the N-bit parity problem with two hidden units, Neural Networks, 1992,5(9):923~926.
Wu Y S, “A new approach to design a simplest ANN for performing certain specific problems.” In: Proceedings of the International Conference on Neural Information Processing, Beijing, 1995,477~480.
吴佑寿,赵明生,丁小青,一种激励函数可调的新人工神经网络及应用,中国科学,E辑,1997,27(1):55~60。
吴佑寿,赵明生. 激活函数可调的神经元模型及其有监督学习与应用。中国科学,E辑,2001,31(3):263~272.
沈艳军,汪秉文,激活函数可调的神经元网络的一种快速算法,中国科学(E辑),2003,33(8):733~740。
汪秉文,沈艳军,多输出神经元模型的多层前向神经网络及其应用,控制理论与应用,待发表。
胡守仁 ,余少波 ,戴葵 ,神经网络导论. 长沙:国防科技大学出版社, 1992.
Chen S., and Billings, S.A., Nonlinear System Identification using Neural Networks, Int. J. Control, 1990, 51(6):1191~1214.
Chen S. and Billings S.A., Neural Networks for Nonlinear Dynamics System Modeling and Identification, Int. J. Control. 1992,56(2):319~346.
古勇 ,胡协和 .基于改进PRPE算法的神经网络建模及其在工业电加热炉中的应用 .控制理论与应用 , 1996, 13( 6) :785~790
王旭东 ,邵惠鹤 .RBF神经网络在非线性系统建模中的应用 .控制理论与应用 , 1997, 14( 1) :59~ 66
金宏 ,张宏钺 .基本样条神经网络及其非线性建模 .控制与决策 , 1999, 14( 5) :469 ~ 472
钱峻 ,邵惠鹤 .一种小波神经网络的在线建模和校正算法 .模式识别与人工智能 , 2000, 13( 1) :16~ 20
张星昌 .具有动态补偿能力的神经网络模型及其在动态系统建模中的应用 .控制理论与应用 , 1996, 13( 6) :823~826
刘川来等 .基于人工神经网络的轮胎硫化过程建模 .第二届全球华人智能控制与智能自动化大会论文集 .西安 :西安交通大学出版社 , 1997, 595~ 598
曹劲等 .基于自校正回归神经网络的预报建模 .信息与控制 , 1998, 27( 2) :156~160
张兰玲等 .基于多层局部回归神经网络的复杂生产过程,预测控制 .模式识别与人工智能 , 1998, 11( 1) :75~81
Atiya, A.F., Alexander G. Parlos, New Results on Recurrent network training: unifying the algorithms and accelerating convergence. IEEE Transaction on Neural Networks, 2000, 11 (3): 97~709.
Rumelhart D E , McClelland J L and the Group. Parallel Distributed Processing, MIT Press, vol.Ⅰ & Ⅱ, Cambridge, Massachusettes, 1986.
S.Singhal and L.Wu, Training feedforward networks with the extended Kalman algorithm, in ICASSP-89:1989, int. Conf. Acoust., Speed, Signal Processing, vol.2, 1989, pp. 1187-1190.
S.Singhal and L.Wu, "Training feedforward networks with the extended Kalman filter," in Proc. IEEE Int. Conf. Acoust., Speed, Signal Processing, Glasgow, U.K.,1989, pp.1187-1190.
S.Shah, F.Palmieri, and M.Datum, "Optimal filtering algorithm for fast learning in feedforward neural networks," Neural Networks, 1992, 5(2): 779-787
Octavian Stan and Edward Kamen, "A local Linearized Least Squares Algorithm for Training Feedforward Neural networks ", IEEE Trans. On Neural Networks. 2000, 11(2), 487~495.
Scalero R S and Tepedelenioglu N,A fast new algorithm for training feedforword Neutral Networks, IEEE Trans. On Signal Processing 1992, 40( 2) : 202~210.
Mahmood R. Azimi-Sadjadi and Ren-Jean Liou. Fast Learning Processing of Multilayer Neural Network Using Recursive Learning Squares Method , IEEE Trans on Signal Processing, 1992, 40(2):447~450.
M.T. Hagan and M.Menhaj, Training feedforward networks with the Marquardt algorithm, IEEE trans. Neural Networks, 1994, 5(6):989-993.
Nikolaos Ampazis and Stavros J. Perantonis, Two Highly Efficient Second-order Algorithms for Training Feedforward Networks, IEEE Trans. Neural Networks, 2002, 13(5): 1064-1074.
Chi-Tat Leung, Tommy W.S. Chow. Adaptive regularization parameter selection method for enhancing generalization capability of neural networks. Artifical Intelligence, 1999, 107(9): 347-356.
R.Setiono, A penalty-function approach for pruning feedforward neural networks. Neural Computation, 1997,102(8): 185-204.
Chi-Sing Leung, Gilbert H. Young, John Sum, and Wing-kay Kan, On the regularization of Forgetting Recursive Least Square. IEEE Trans. on neural networks, 1999, 10(6): 1482~1486.
Chi-Sing Leung, Ah-Chung Tsoi, Lai Wan Chan, Two Regularization For Recursive Least Squared Algorithms in Feedforward Multilayered Neural networks. IEEE. Trans. On Neural Netwroks. 2001,12(6): 1314~1332.
李艳君 ,吴铁军 ,赵明旺 .一种GA正交优化法结合的RBF神经网络建模方法 .第三届全球智能控制与自动化大会论文集 ,2000,882~884
王旭东 ,邵惠鹤 .RBF神经网络在非线性系统建模中的应用 .控制理论与应用 ,1997,14(1):59~66
巩敦卫 ,许世范 ,王雪松 .递归神经网络在选煤厂跳汰系统建模中的应用研究 .第三届全球智能控制与自动化大会论文集 , 2000, 1147 ~115 0
Pramod Gupta, Naresh K.Sinha. An improved approach for nonlinear system identification using neural networks. Journal of Franklin Institute 1999,336(7):721~734.
鲍鸿 ,黄心汉 .用模糊RBF神经网络简化模型设计多变量自适应模糊控制器 .控制理论与应用 , 2000, 17 ( 2) :169~ 17 4
陈小红 ,高峰 ,钱积新 ,孙优贤 .基于径基函数网络的精馏塔自适应控制 .控制理论与应用 , 1998,15 ( 2) :227 ~231
解学军,张大雷,基于径向基网络的非线性离散时间系统的自适应控制,自动化学报,2000,26(3):414~418。
李鸿儒 ,边春元 ,顾树生 .基于神经网络的一类非线性系统的自适应控制 .控制与决策 , 19 9 9 , 14(增刊 ) :5 11~5 15
Berenji, H.R. and Khadkr, P.S., Adaptive Fuzzy control with Reinforcement Learning. ACC, San Francsico, 1993, 1840~1844.
彭小奇 ,梅炽 ,周孑民 ,唐英 .多变量系统的模糊神经网络控制模型与其应用 .控制理论与应用 , 1995 , 12( 3) :351~ 359
王殿辉 ,柴天佑 .自适应模糊神经网络控制器设计的线性化方法 .控制与决策 , 1995 , 10( 1) :21~27
濮卫兴 ,陈来九 .用B样条神经网络设计自适应模糊控制器 .控制理论与应用 , 1996 , 13( 4) :448~45 4
王笑颜 ,符雪桐 .一种自学习模糊神经网络多变量自适应控制器 .控制理论与应用 , 1999, 16 ( 2) :309~312
李歧强 ,钱积新 ,李现明 ,金萍 .自适应模糊神经网络控制器在电阻加热炉中的应用 .控制与决策 , 1999 , 14( 2) :189~192
Narenda. K. S. and Mukhopadhydy, S., Adaptive control of nonlinear mulrivariable systems using neutral networks , Neural Networks, 1994, 7(8): 737~752.
金耀初 ,蒋静坪 ,诸静 .结合模糊推理的多变量自适应控制 .信息与控制 , 1994, 23( 4) :223~228
刘延年 ,忻欣 ,冯纯伯 .基于神经网络的一类非线性连续系统的稳定自适应控制 .控制理论与应用 , 1996, 13(1) :70~75
诸勇 ,钱积新 .基于神经网络的一类离散非线性系统的稳定自适应控制 .模式识别与人工智能 , 1998, 11( 3) :335~340
程启明 ,万德钧 ,吴峻 ,马壮 .一种新型神经网络控制器及在自动舵上的应用 .控制与决策 , 1999 ,14( 2) :125~129
Chen, F. C., Liu , C.C., Adaptively controlling nonlinear continues time systems using multilayer neural networks , IEEE trans. on Automatic Control, 1994,39(6): 1306-1310.
Chen, F. C., Khalil, H.K. , Adaptive control of a class of nonlinear discrete time systems using neural networks . IEEE Trans. on Automatic Control , 1995,40(5), 791-801.
Ioannou , P. A. and Sun J. , Robust adaptive control. Upper Saddle River, NJ: Prentice Hall.
Polycarpou, M. M. and Ioannou, P. A., Stable adaptive neural control scheme for nonlinear systems . IEEE Trans. on Automatic Control,1996, 41(3), 447-451.
Rovithakis, G.A. and Christodoulou, M.A., Direct adaptive regulation of unknown nonlinear dynamical systems via dynamic neural networks. Systems, Man and Cybernetics, IEEE Transactions on , 1995,25(12):1578 -1594.
Rovithakis, G.A., and Christodoulou, M.A., Adaptive control of unknown plants using dynamical neural networks. IEEE Trans. on Systems, Man, Cybernetics, 1994, 24(3), 400-412.
Rovithakis, G.A., and Christodoulou, M.A., Direct adaptive regulation of unknown nonlinear dynamical neural networks via dynamical neural networks. IEEE Trans. on Systems , Man, Cybernetics, 1995, 25(12), 1578-1595.
Rovithakis, G.A., and Christodoulou, M.A., Neural adaptive regulation of unknown nonlinear dynamical systems. Systems, Man and Cybernetics, Part B, IEEE Transactions on Systems , Man, Cybernetics, 1997, 27( 5): 810 -822.
Rovithakis, G.A., Tracking control of multi-input affine nonlinear dynamical systems with unknown nonlinearities using dynamical neural networks.
Systems, Man and Cybernetics, Part B, IEEE Transactions on , 1999, 29 (2 ): 179 –189
Rovithakis, G. A., Robust neural adaptive stabilization of unknown systems with measurement noise , Systems, Man and Cybernetics, Part B, IEEE Transactions on , 1999,29( 3 ): 453 -459.
Rovithakis, G. A., Stable adaptive neurocontrol design via Lyapunov function derivative estimation, Automatica , 2001, 37 (8): 1213-1221.
Psaltis D, Sideris A, Yamamura A, A multilayered neural networks controller, IEEE Control System Magazine, 1988, 8(8): 17-21.
Goh C J. Model reference control of nonlinear systems via implicit function emulation. Int. J. Control, 1994, 60(23):91-115.
Levin A U , Narendra K S . Control of nonlinear dynamical systems using neural networks Prak II: observability, identification, and control , IEEE Trans. Neural Networks, 1996, 7(1):30-42.
Goh C J, Lee T H , Direct adaptive control of nonlinear systems via implicit function emulation, Control Theory and Advanced Technology, 1994, 10(3): 539-552.
Ge S S, Hang C C, Zhang T, Nonlinear adaptive control using neural networks and its application to CSTR system. J. Process Control, 1998, 8(3):313-323.
Jin L, Nikiforuk P N, Gupta M M, Fast neural networks learning and control of discrete-time nonlinear systems. IEEE Trans. Syst, Man and Cybernetics, 1999, 25(3): 478-488.
周东华,非线性系统的自适应控制导论,清华大学出版社,2002。
Hebb D O.The Organization of Behavior. New York:Wiley,1949.
Eccles J C..Cholinergic and Inhibitory Synapses in A Pathway from Motor-axon Collaterals to Motorneurones . Journal Physiology,1954, 12(6): 524.
Rosenblatt F. The Perception: A Probabilistic Model for Information strorage and Organization in the Brain . Psychological Review, 1958, 23(68): 386~408.
Widrow B, Hoff M E, Adaptive Switching Circuits. 1960 IRE WESCON convention record: part 4. Computers: Man-machine Systems, Losangeles: 96~104.
Grossberg S , On the Serial Learning of Lists. Bio-science, 1969, 17(4): 201~253.
Grossberg S , Some Networks that Can Learn, Remember and Reproduce any Number of Compialted Space-time Patterns, II,stud Applied Mathematics 1970, 34(49):135~166.
Willshaw D J, Buneman O P, Longuest-higgins. HC Nature, 1969, 22(2):960.
Nilsson N J, Learning Machines: Foundations of Trainable Pattern Classifying Systems. McGraw-hill, New York, 1965.
Stein R B , Leung K V, Mangeron D, Oguztoreli M N , Improved Neuraonal models for Studying Neural Networks Kybernetik, 1974, 19(15): 1~9.
Heiden U. Ander, Existence of Periodic Solutions of a Nerve Equation. Biology Cybernitics, 1976, 23(21): 37~39.
Hopfield J J. Neural Networks and physical systems with emergent collective computational abilities. Proc Nat Acad Sci, 1982, 79: 2554~2558.
Marr D, Vision [M]. San Francisco: WHF Freeman, 1982.
Kirkpatrick S, Gellat K C D, Veechi M P , Optimization by Simulated Annealing , Science, 1983, 220(4598): 671~681.
Hinton G E , Sejuowshi T J, Ackley D H, Boltzmann Machines: Cotraint Satisfaction Networks that Learn. Carnegiemellon University, Tech, Report CMU-CS-84-119. 1984.
Piggio T, Analog model of Computation for III-posed Problems of Early Vision . Artificial Intelligence Lab Memo, 783, MIT, 1984.
Poggio T, Computational Vision and Regularization Theory. Nature (Lond), 1985, 27(3): 314~319.
Chua L O , Yand L. Celluar Neural networks: Theory. IEEE Transaction on Circuits and Systems, 1988, 35(10): 1257~1272.
Chua L O , Yand L, Celluar Neural Networks: Application, IEEE Transaction on Circuits and Systems, 1988, 35(10): 1273~1290.
Kosko B. Adaptive Bidirectional Associative Memories. Applied Optical, 1987, 26 (23): 4647~4680.
Kosko B, Bidirectional Associative Memories. IEEE Transaction On Man, System and Cybernitics . 1988, 18(1): 49~59.
廖晓昕.细胞神经网络的数学理论 (I) 、(II).中国科学 (A辑 ) , 1994, 24 (9): 902~910: 1037~1046.
赵杰煜,反馈随机二元神经网络,中国科学(E辑),2001, 31 (5): 471~480.
舒怀林,PID神经元和PID神经网络分析,见:1998中国控制会议论文集,长沙:国防科技大学出版社,1998,607~613.
舒怀林,基于PID神经网络的非线性时变系统辨识,自动化学报,2002,28(3),475~476.
刘普寅,一种新的模糊神经网络及其逼近能力,中国科学(E辑),2002, 32 (5): 76~86.
Qinghua Zhang and Albert Benveniste, Wavelet Networks, IEEE Transaction on Neural Networks, 1992, 3( 6). 889~898.
P.S. Sastry, G. Santharam and K. P. Unnikrishnan, Memory Neuron Networks for Identification and Control of Dynamical systems. IEEE Transaction on Neural Networks, 1994, 5( 2): 306~319
Jenkins B K, A R Tanguay Jr, Optical Architectures for Neural Network Implementation, Handbook of Neural Computing and Neural Networks. MIT Press, Boston, 1995:673~677.
McAulay A D, Wand J, Ma C, Optical Heteroassciative Memory Using Spatial Light Rebroadcasters. Applied Optical,1990, 29(14): 2067~2073.
阮昊 ,陈述春 ,戴凤妹 ,千福熹.利用电子俘获材料实现IPA神经网络模.光学学报 , 1997, 17(6):766~771.
Narendra K, Parthasarathy K, Identification and Control of Dynamical System Using Neural Networks. IEEE Transaction On Neural Networks, March 1990,1(1) :4~27.
戴先中 ,刘军 ,冯纯伯.连续非线性系统的神经网络α阶逆系统控制方法.自动化学报 ,1998, 24(4): 463~468.
Miller W T, Real time Application of Neural Networks for Sensor based Control of Robots. With Vision. IEEE Transaction On Man, System and Cybernitics 1989, 23(19): 825~831.
Bulsari A, Some Analytical Solutions to the General Approximation Problem for Feedforword Neural Networks. Neural Networks, 1993, (6) :991-996.
Aibara K, Chaotic Neural Networks. Physical Letter A, 1990, 144 (6,7):334~3340.
Inoue M, Nagayoshi A., A Chaos Neuro-computer. Physical Letter A, 1991, 158(8): 373~376.
Inoue M, Nakamoto K., Epilepsyina Chaos Neuro-computer Model. SPIEVOL, 236, Chaos in Biology and Medicine, 1993, 77~84.
Satoru Iaske, On neural approximation of fuzzy system. In : Proceedings of INCNN’92, Vol. 1, New York: IEEE, 1992. 1263~1268.
Jokinen Petri A. On the relations between radial basis function networks and fuzzy system. In: Proceeding of INCNN’29, Vol. 1, New York: IEEE, 1992, 1220~1225.
Okada Hiroyuki, Initializing multilayer neural networks with fuzzy logic. In: Proceeding of INCNN’92, Vol. 1, 1239~1244.
Cai Yaling, Hon Keung Kwan, A fuzzy neural networks with fuzzy classification. In: Proceedings of ASMESCI’94, Wuhan: Process of Huazhong University of Science and Technology, 1994: 894~899.
B E. Using spectral techniques for improved performance in ANN, Proc IEEE, 1993. 500~505.
Lee S, Kil R M. A Gaussian potential function network with hierarchnically self-organizing learning, Neural Networks, 1991, 4(7):207~224.
Stork D. G, Allen J D ,et al, How to solve the N-bit parity problem with two hidden units, Neural Networks, 1992,5(9):923~926.
Wu Y S, “A new approach to design a simplest ANN for performing certain specific problems.” In: Proceedings of the International Conference on Neural Information Processing, Beijing, 1995,477~480.
吴佑寿,赵明生,丁小青,一种激励函数可调的新人工神经网络及应用,中国科学,E辑,1997,27(1):55~60。
吴佑寿,赵明生. 激活函数可调的神经元模型及其有监督学习与应用。中国科学,E辑,2001,31(3):263~272.
沈艳军,汪秉文,激活函数可调的神经元网络的一种快速算法,中国科学(E辑),2003,33(8):733~740。
汪秉文,沈艳军,多输出神经元模型的多层前向神经网络及其应用,控制理论与应用,待发表。
胡守仁 ,余少波 ,戴葵 ,神经网络导论. 长沙:国防科技大学出版社, 1992.
Chen S., and Billings, S.A., Nonlinear System Identification using Neural Networks, Int. J. Control, 1990, 51(6):1191~1214.
Chen S. and Billings S.A., Neural Networks for Nonlinear Dynamics System Modeling and Identification, Int. J. Control. 1992,56(2):319~346.
古勇 ,胡协和 .基于改进PRPE算法的神经网络建模及其在工业电加热炉中的应用 .控制理论与应用 , 1996, 13( 6) :785~790
王旭东 ,邵惠鹤 .RBF神经网络在非线性系统建模中的应用 .控制理论与应用 , 1997, 14( 1) :59~ 66
金宏 ,张宏钺 .基本样条神经网络及其非线性建模 .控制与决策 , 1999, 14( 5) :469 ~ 472
钱峻 ,邵惠鹤 .一种小波神经网络的在线建模和校正算法 .模式识别与人工智能 , 2000, 13( 1) :16~ 20
张星昌 .具有动态补偿能力的神经网络模型及其在动态系统建模中的应用 .控制理论与应用 , 1996, 13( 6) :823~826
刘川来等 .基于人工神经网络的轮胎硫化过程建模 .第二届全球华人智能控制与智能自动化大会论文集 .西安 :西安交通大学出版社 , 1997, 595~ 598
曹劲等 .基于自校正回归神经网络的预报建模 .信息与控制 , 1998, 27( 2) :156~160
张兰玲等 .基于多层局部回归神经网络的复杂生产过程,预测控制 .模式识别与人工智能 , 1998, 11( 1) :75~81
Atiya, A.F., Alexander G. Parlos, New Results on Recurrent network training: unifying the algorithms and accelerating convergence. IEEE Transaction on Neural Networks, 2000, 11 (3): 97~709.
Rumelhart D E , McClelland J L and the Group. Parallel Distributed Processing, MIT Press, vol.Ⅰ & Ⅱ, Cambridge, Massachusettes, 1986.
S.Singhal and L.Wu, Training feedforward networks with the extended Kalman algorithm, in ICASSP-89:1989, int. Conf. Acoust., Speed, Signal Processing, vol.2, 1989, pp. 1187-1190.
S.Singhal and L.Wu, "Training feedforward networks with the extended Kalman filter," in Proc. IEEE Int. Conf. Acoust., Speed, Signal Processing, Glasgow, U.K.,1989, pp.1187-1190.
S.Shah, F.Palmieri, and M.Datum, "Optimal filtering algorithm for fast learning in feedforward neural networks," Neural Networks, 1992, 5(2): 779-787
Octavian Stan and Edward Kamen, "A local Linearized Least Squares Algorithm for Training Feedforward Neural networks ", IEEE Trans. On Neural Networks. 2000, 11(2), 487~495.
Scalero R S and Tepedelenioglu N,A fast new algorithm for training feedforword Neutral Networks, IEEE Trans. On Signal Processing 1992, 40( 2) : 202~210.
Mahmood R. Azimi-Sadjadi and Ren-Jean Liou. Fast Learning Processing of Multilayer Neural Network Using Recursive Learning Squares Method , IEEE Trans on Signal Processing, 1992, 40(2):447~450.
M.T. Hagan and M.Menhaj, Training feedforward networks with the Marquardt algorithm, IEEE trans. Neural Networks, 1994, 5(6):989-993.
Nikolaos Ampazis and Stavros J. Perantonis, Two Highly Efficient Second-order Algorithms for Training Feedforward Networks, IEEE Trans. Neural Networks, 2002, 13(5): 1064-1074.
Chi-Tat Leung, Tommy W.S. Chow. Adaptive regularization parameter selection method for enhancing generalization capability of neural networks. Artifical Intelligence, 1999, 107(9): 347-356.
R.Setiono, A penalty-function approach for pruning feedforward neural networks. Neural Computation, 1997,102(8): 185-204.
Chi-Sing Leung, Gilbert H. Young, John Sum, and Wing-kay Kan, On the regularization of Forgetting Recursive Least Square. IEEE Trans. on neural networks, 1999, 10(6): 1482~1486.
Chi-Sing Leung, Ah-Chung Tsoi, Lai Wan Chan, Two Regularization For Recursive Least Squared Algorithms in Feedforward Multilayered Neural networks. IEEE. Trans. On Neural Netwroks. 2001,12(6): 1314~1332.
李艳君 ,吴铁军 ,赵明旺 .一种GA正交优化法结合的RBF神经网络建模方法 .第三届全球智能控制与自动化大会论文集 ,2000,882~884
王旭东 ,邵惠鹤 .RBF神经网络在非线性系统建模中的应用 .控制理论与应用 ,1997,14(1):59~66
巩敦卫 ,许世范 ,王雪松 .递归神经网络在选煤厂跳汰系统建模中的应用研究 .第三届全球智能控制与自动化大会论文集 , 2000, 1147 ~115 0
Pramod Gupta, Naresh K.Sinha. An improved approach for nonlinear system identification using neural networks. Journal of Franklin Institute 1999,336(7):721~734.
鲍鸿 ,黄心汉 .用模糊RBF神经网络简化模型设计多变量自适应模糊控制器 .控制理论与应用 , 2000, 17 ( 2) :169~ 17 4
陈小红 ,高峰 ,钱积新 ,孙优贤 .基于径基函数网络的精馏塔自适应控制 .控制理论与应用 , 1998,15 ( 2) :227 ~231
解学军,张大雷,基于径向基网络的非线性离散时间系统的自适应控制,自动化学报,2000,26(3):414~418。
李鸿儒 ,边春元 ,顾树生 .基于神经网络的一类非线性系统的自适应控制 .控制与决策 , 19 9 9 , 14(增刊 ) :5 11~5 15
Berenji, H.R. and Khadkr, P.S., Adaptive Fuzzy control with Reinforcement Learning. ACC, San Francsico, 1993, 1840~1844.
彭小奇 ,梅炽 ,周孑民 ,唐英 .多变量系统的模糊神经网络控制模型与其应用 .控制理论与应用 , 1995 , 12( 3) :351~ 359
王殿辉 ,柴天佑 .自适应模糊神经网络控制器设计的线性化方法 .控制与决策 , 1995 , 10( 1) :21~27
濮卫兴 ,陈来九 .用B样条神经网络设计自适应模糊控制器 .控制理论与应用 , 1996 , 13( 4) :448~45 4
王笑颜 ,符雪桐 .一种自学习模糊神经网络多变量自适应控制器 .控制理论与应用 , 1999, 16 ( 2) :309~312
李歧强 ,钱积新 ,李现明 ,金萍 .自适应模糊神经网络控制器在电阻加热炉中的应用 .控制与决策 , 1999 , 14( 2) :189~192
Narenda. K. S. and Mukhopadhydy, S., Adaptive control of nonlinear mulrivariable systems using neutral networks , Neural Networks, 1994, 7(8): 737~752.
金耀初 ,蒋静坪 ,诸静 .结合模糊推理的多变量自适应控制 .信息与控制 , 1994, 23( 4) :223~228
刘延年 ,忻欣 ,冯纯伯 .基于神经网络的一类非线性连续系统的稳定自适应控制 .控制理论与应用 , 1996, 13(1) :70~75
诸勇 ,钱积新 .基于神经网络的一类离散非线性系统的稳定自适应控制 .模式识别与人工智能 , 1998, 11( 3) :335~340
程启明 ,万德钧 ,吴峻 ,马壮 .一种新型神经网络控制器及在自动舵上的应用 .控制与决策 , 1999 ,14( 2) :125~129
Chen, F. C., Liu , C.C., Adaptively controlling nonlinear continues time systems using multilayer neural networks , IEEE trans. on Automatic Control, 1994,39(6): 1306-1310.
Chen, F. C., Khalil, H.K. , Adaptive control of a class of nonlinear discrete time systems using neural networks . IEEE Trans. on Automatic Control , 1995,40(5), 791-801.
Ioannou , P. A. and Sun J. , Robust adaptive control. Upper Saddle River, NJ: Prentice Hall.
Polycarpou, M. M. and Ioannou, P. A., Stable adaptive neural control scheme for nonlinear systems . IEEE Trans. on Automatic Control,1996, 41(3), 447-451.
Rovithakis, G.A. and Christodoulou, M.A., Direct adaptive regulation of unknown nonlinear dynamical systems via dynamic neural networks. Systems, Man and Cybernetics, IEEE Transactions on , 1995,25(12):1578 -1594.
Rovithakis, G.A., and Christodoulou, M.A., Adaptive control of unknown plants using dynamical neural networks. IEEE Trans. on Systems, Man, Cybernetics, 1994, 24(3), 400-412.
Rovithakis, G.A., and Christodoulou, M.A., Direct adaptive regulation of unknown nonlinear dynamical neural networks via dynamical neural networks. IEEE Trans. on Systems , Man, Cybernetics, 1995, 25(12), 1578-1595.
Rovithakis, G.A., and Christodoulou, M.A., Neural adaptive regulation of unknown nonlinear dynamical systems. Systems, Man and Cybernetics, Part B, IEEE Transactions on Systems , Man, Cybernetics, 1997, 27( 5): 810 -822.
Rovithakis, G.A., Tracking control of multi-input affine nonlinear dynamical systems with unknown nonlinearities using dynamical neural networks.
Systems, Man and Cybernetics, Part B, IEEE Transactions on , 1999, 29 (2 ): 179 –189
Rovithakis, G. A., Robust neural adaptive stabilization of unknown systems with measurement noise , Systems, Man and Cybernetics, Part B, IEEE Transactions on , 1999,29( 3 ): 453 -459.
Rovithakis, G. A., Stable adaptive neurocontrol design via Lyapunov function derivative estimation, Automatica , 2001, 37 (8): 1213-1221.
Psaltis D, Sideris A, Yamamura A, A multilayered neural networks controller, IEEE Control System Magazine, 1988, 8(8): 17-21.
Goh C J. Model reference control of nonlinear systems via implicit function emulation. Int. J. Control, 1994, 60(23):91-115.
Levin A U , Narendra K S . Control of nonlinear dynamical systems using neural networks Prak II: observability, identification, and control , IEEE Trans. Neural Networks, 1996, 7(1):30-42.
Goh C J, Lee T H , Direct adaptive control of nonlinear systems via implicit function emulation, Control Theory and Advanced Technology, 1994, 10(3): 539-552.
Ge S S, Hang C C, Zhang T, Nonlinear adaptive control using neural networks and its application to CSTR system. J. Process Control, 1998, 8(3):313-323.
Jin L, Nikiforuk P N, Gupta M M, Fast neural networks learning and control of discrete-time nonlinear systems. IEEE Trans. Syst, Man and Cybernetics, 1999, 25(3): 478-488.
周东华,非线性系统的自适应控制导论,清华大学出版社,2002。