MCA算法的改进及收敛性分析
|
摘要
人工神经网络是对人脑的反应机制进行简化、抽象和模拟建立起来的数学模型,通过大量基本组成单位——人工神经元的相互连接而对外界环境输入的信息进行并行分布式的处理,具有较强的自适应性和容错性.作为人工神经网络的一个应用——MCA神经网络学习算法,就是寻找一个方向,使得数据空间在这个方向上的投影有最小的方差.由于其应用的广泛性,MCA算法的收敛性变得非常重要.因为确定离散时间(Deterministic Discrete Time,DDT)系统不要求算法的学习率收敛到零,而且还可以保持算法的离散特征,所以基于DDT系统的MCA收敛性分析是近年来人们研究的热点.
本文对Oja-Xu MCA和Ojan MCA学习算法进行了研究.对于前者,我们在归一化Oja-Xu MCA算法的基础上又做了进一步的改进,提出了固定步长的跳步归一化及自适应变步长的跳步归一化方法,提高算法的收敛速度和学习精度,并且还对固定步长的跳步归一化方法做了权值有界性的证明。对于后者,我们在理论上对算法的收敛性进行了分析,将原有学习率的取值范围扩大了一倍,并通过数值试验验证了我们的理论结果。
本文的结构安排如下:第一章介绍了人工神经网络及MCA神经网络学习算法的相关背景知识,第二章对归一化Oja-Xu MCA算法进行了改进,第三章对Ojan MCA算法的收敛性做了进一步的研究,最后是结论。
Artificial neural network(ANN) is a mathematical model based on the simplification, abstraction and simulation of the reaction system of human brain.ANN deals with information from outside environment in a parallel manner by collection of many basic units called neuron,which ensures the ANN a good quality of self-adaptation and error tolerance. As an application of ANN,MCA neural network learning algorithm,is to search a direction to let the data space have the least variance on the direction.Because of its wide application,the convergence of the MCA algorithm is very important.As the Deterministic Discrete Time (DDT) system doesn't require the learning rate convert to zero and conserve the discrete of the algorithm,the convergence of MCA algorithm based on DDT is the hotspot of people's work.
This thesis studies the Oja-Xu MCA learning algorithm and the Ojan MCA learning algorithm.To the former,we make some improvements based on the normalizing improvement,put forward the fixed interval normalizing method and adaptive interval normalizing method,which improve the convergence speed and the accuracy.In addition,we prove the boundedness of the fixed interval normalizing method.To the latter,we analysis the convergence of the learning algorithm,and enlarge the scope of the learning rate to twice, which is proved by the numerical experimentation.
The structure of this thesis is organized as follows.Chapter 1 gives a brief introduction of ANN and the knowledge of MCA learning algorithm.Chapter 2 makes some improvements based on the normalizing improvement of the Oja-Xu MCA learning algorithm. Chapter 3 is concerned with the further study of the convergence of the Ojan MCA learning algorithm.Finally,a brief conclusion is given.
本文对Oja-Xu MCA和Ojan MCA学习算法进行了研究.对于前者,我们在归一化Oja-Xu MCA算法的基础上又做了进一步的改进,提出了固定步长的跳步归一化及自适应变步长的跳步归一化方法,提高算法的收敛速度和学习精度,并且还对固定步长的跳步归一化方法做了权值有界性的证明。对于后者,我们在理论上对算法的收敛性进行了分析,将原有学习率的取值范围扩大了一倍,并通过数值试验验证了我们的理论结果。
本文的结构安排如下:第一章介绍了人工神经网络及MCA神经网络学习算法的相关背景知识,第二章对归一化Oja-Xu MCA算法进行了改进,第三章对Ojan MCA算法的收敛性做了进一步的研究,最后是结论。
Artificial neural network(ANN) is a mathematical model based on the simplification, abstraction and simulation of the reaction system of human brain.ANN deals with information from outside environment in a parallel manner by collection of many basic units called neuron,which ensures the ANN a good quality of self-adaptation and error tolerance. As an application of ANN,MCA neural network learning algorithm,is to search a direction to let the data space have the least variance on the direction.Because of its wide application,the convergence of the MCA algorithm is very important.As the Deterministic Discrete Time (DDT) system doesn't require the learning rate convert to zero and conserve the discrete of the algorithm,the convergence of MCA algorithm based on DDT is the hotspot of people's work.
This thesis studies the Oja-Xu MCA learning algorithm and the Ojan MCA learning algorithm.To the former,we make some improvements based on the normalizing improvement,put forward the fixed interval normalizing method and adaptive interval normalizing method,which improve the convergence speed and the accuracy.In addition,we prove the boundedness of the fixed interval normalizing method.To the latter,we analysis the convergence of the learning algorithm,and enlarge the scope of the learning rate to twice, which is proved by the numerical experimentation.
The structure of this thesis is organized as follows.Chapter 1 gives a brief introduction of ANN and the knowledge of MCA learning algorithm.Chapter 2 makes some improvements based on the normalizing improvement of the Oja-Xu MCA learning algorithm. Chapter 3 is concerned with the further study of the convergence of the Ojan MCA learning algorithm.Finally,a brief conclusion is given.
引文
[1]S.Haykin.Neural networks:a comprehensive foundation.New York:Macmillan,1994,1-30.
[2]M.T.Hagan,H.B.Demuth,M.H.Beale著,戴葵等译.神经网络设计.北京:机械工业出版社,2002,34-55.
[3]吴一全,朱兆达.图像处理中阈值选取方法30年(1962-1992)的进展(一).数据采集与处理,1993,8(3):37-45.
[4]吴一全,朱兆达.图像处理中阈值选取方法30年(1962-1992)的进展(二).数据采集与处理,1993,8(4):29-43.
[5]C.H.Teh,Chin.R.T.On image analysis by the methods of moments.IEEE Trans.on Pattern Analysis and Machine Intelligence,1988,10(4):496-513.
[6]郭力宾,吴微.二维图像中交叉点的神经网络识别.大连理工大学学报,2003,43:548-550.
[7]孔俊,吴微,赵卫海.识别数学符号的神经网络方法.吉林大学自然科学学报,2001,3:11-16.
[8]Hou L,Wu W,Zhu B,Li F.A segmentation method for merged characters using self-organizing map neural networks.Journal of Information and Computational Science,2006,3(2):219-226.
[9]吴微,陈维强,刘波.用bp神经网络预测股票市场涨跌.大连理工大学学报,2001,41(1):9-15.
[10]张玉林,吴微.用bp神经网络捕捉股市黑马初探.运筹与管理,2004,13(2):123-130.
[11]张立明著.人工神经网络的模型及其应用.上海:复旦大学出版社,1994:10-12
[12]彭德中.MCA神经网络理论与应用[D].成都:电子科技大学,2006:20-28
[13]Q.Zhang.On the discrete -time dynamics of a PCA learning algorithm.Neurocomputing,2003,55(3):761-769.
[14]Z.Yi,M.Ye,J.C.LvandK.K.Tan.Convergence analysis of a deterministic discrete dime system of Oja's PCA learning algorithm.IEEE Transactions on Neural Networks,2005,16(6):1318-1328.
[15]J.H.Manton,U.Helmke and I.M.Y.Marells.Dynamical Systems for Principal and Minor Component Analysis.Proceedings of the 42nd IEEE Transactions on Decision and Control,2003,1863-1868.
[16]L.Ljung.Analysis of recursive stochastic algorithms.IEEETrans.Autom.Contr.,vo l.AC-22,no.3,pp.551-575,Mar.1977.
[17]P.J.Zufiria.On the discrete-time dynamics of the basic Hebbian neural-network nods.IEEE Trans.NeuralNetw.,vol.13,no.6,pp.1342-1352,Nov.2002.
[18]E.Oja.A simplified neuron mode as a principal component analyzer.J.Math.Biol.,vol.15,pp.167-273,1982.
[19]E.Oja.Principal components,minor components and linear neural networks.Neural Netw.,vol.5,pp.927-935,1992.
[20]L.Xu,E.Oja and C.Suen.Modified Hebbian learning for curve and surface fitting.Neural Netw.,volS,pp.441-457,1992.
[21]Dezhong Peng and Zhang Yi.A modified Oja-xu MCA learning algorithm and its convergence analysis.IEEE Transaction on circuits and systems,vol.54,no.4,2007.
[22]Dezhong Peng and Zhang Yi.Convergence analysis of the OJAn MCA learning algorithm by the deterministic discrete time method.Theoretical Computer Science 378(2007)87-100
[23]马建仓,牛奕龙,陈海洋.盲信号处理.北京:国防工业出版社,2006.
[24]Wang L Y,Karhunen J.A unified neural bigradient algorithm for robust PCA and MCA.Int.J.Neural Syst.1996,7:53-67.
[25]Chen T P,Amari S,Lin Q.A unified algorithm for principal and minor components extraction.Neural Networks,1998,11:385-390.
[26]Xu L.Least mean squre error reconstruction principal for self-organizing neuralnets.Neural Networks,1993,6:627-648.
[27]吴微著.神经网络计算.北京:高等教育出版社,2003,5;9-16:41-48.
[28]蒋宗礼著.人工神经网络导论.北京:高等教育出版社,2001,27-29.
[2]M.T.Hagan,H.B.Demuth,M.H.Beale著,戴葵等译.神经网络设计.北京:机械工业出版社,2002,34-55.
[3]吴一全,朱兆达.图像处理中阈值选取方法30年(1962-1992)的进展(一).数据采集与处理,1993,8(3):37-45.
[4]吴一全,朱兆达.图像处理中阈值选取方法30年(1962-1992)的进展(二).数据采集与处理,1993,8(4):29-43.
[5]C.H.Teh,Chin.R.T.On image analysis by the methods of moments.IEEE Trans.on Pattern Analysis and Machine Intelligence,1988,10(4):496-513.
[6]郭力宾,吴微.二维图像中交叉点的神经网络识别.大连理工大学学报,2003,43:548-550.
[7]孔俊,吴微,赵卫海.识别数学符号的神经网络方法.吉林大学自然科学学报,2001,3:11-16.
[8]Hou L,Wu W,Zhu B,Li F.A segmentation method for merged characters using self-organizing map neural networks.Journal of Information and Computational Science,2006,3(2):219-226.
[9]吴微,陈维强,刘波.用bp神经网络预测股票市场涨跌.大连理工大学学报,2001,41(1):9-15.
[10]张玉林,吴微.用bp神经网络捕捉股市黑马初探.运筹与管理,2004,13(2):123-130.
[11]张立明著.人工神经网络的模型及其应用.上海:复旦大学出版社,1994:10-12
[12]彭德中.MCA神经网络理论与应用[D].成都:电子科技大学,2006:20-28
[13]Q.Zhang.On the discrete -time dynamics of a PCA learning algorithm.Neurocomputing,2003,55(3):761-769.
[14]Z.Yi,M.Ye,J.C.LvandK.K.Tan.Convergence analysis of a deterministic discrete dime system of Oja's PCA learning algorithm.IEEE Transactions on Neural Networks,2005,16(6):1318-1328.
[15]J.H.Manton,U.Helmke and I.M.Y.Marells.Dynamical Systems for Principal and Minor Component Analysis.Proceedings of the 42nd IEEE Transactions on Decision and Control,2003,1863-1868.
[16]L.Ljung.Analysis of recursive stochastic algorithms.IEEETrans.Autom.Contr.,vo l.AC-22,no.3,pp.551-575,Mar.1977.
[17]P.J.Zufiria.On the discrete-time dynamics of the basic Hebbian neural-network nods.IEEE Trans.NeuralNetw.,vol.13,no.6,pp.1342-1352,Nov.2002.
[18]E.Oja.A simplified neuron mode as a principal component analyzer.J.Math.Biol.,vol.15,pp.167-273,1982.
[19]E.Oja.Principal components,minor components and linear neural networks.Neural Netw.,vol.5,pp.927-935,1992.
[20]L.Xu,E.Oja and C.Suen.Modified Hebbian learning for curve and surface fitting.Neural Netw.,volS,pp.441-457,1992.
[21]Dezhong Peng and Zhang Yi.A modified Oja-xu MCA learning algorithm and its convergence analysis.IEEE Transaction on circuits and systems,vol.54,no.4,2007.
[22]Dezhong Peng and Zhang Yi.Convergence analysis of the OJAn MCA learning algorithm by the deterministic discrete time method.Theoretical Computer Science 378(2007)87-100
[23]马建仓,牛奕龙,陈海洋.盲信号处理.北京:国防工业出版社,2006.
[24]Wang L Y,Karhunen J.A unified neural bigradient algorithm for robust PCA and MCA.Int.J.Neural Syst.1996,7:53-67.
[25]Chen T P,Amari S,Lin Q.A unified algorithm for principal and minor components extraction.Neural Networks,1998,11:385-390.
[26]Xu L.Least mean squre error reconstruction principal for self-organizing neuralnets.Neural Networks,1993,6:627-648.
[27]吴微著.神经网络计算.北京:高等教育出版社,2003,5;9-16:41-48.
[28]蒋宗礼著.人工神经网络导论.北京:高等教育出版社,2001,27-29.