非对称神经网在模式识别中的应用
摘要
本文重点讨论了由蒙特卡罗—选择变异规则设计的非对称反馈神经网络的动力学特性,以及这类神经网络在模式识别和联想记忆等应用领域中所表现出来的优越性。本文的第二部分研究非线性晶格点阵中孤波碰撞时的空间跳跃行为。
蒙特卡罗—选择变异规则(简称MCA)是由赵鸿教授提出的用于设计非对称神经网络的一种整体学习规则。其主要思想是通过有选择地随机改变连接矩阵使系统在给定的训练目标下达到整体最优。对于反馈神经网络这类高维非线性动力系统而言,其训练目标是控制存储模式所激发的神经元局域场分布从而使系统获得最佳性能。由此发现的参数空间的三个动力学相区(即:“混沌相”、“记忆相”和“混合相”)不但对实际应用中表现出极大的优越性,而且对于研究这类时空离散动力系统的整体行为也具有重要的理论价值。
因此,在本文中我们首先介绍了MCA学习规则及其在整个神经网络理论体系中所处的地位和作用,并通过选择投影的方法对不同相区系统的状态空间做了进一步刻画和描述。接着,我们详细讨论了“混沌相”在模式识别中的应用。处于这一参数区的神经网络具有两个基本性质:其一是各个存储模式的吸引域都很小而且被一个充满状态空间的混沌轨道彼此分开;其二是伪吸引子原则上被完全消除。正是这两个特性使得这类神经网络克服了传统反馈神经网络的固有缺陷而成为模式识别的有力工具。为了使它们更具有实用性和可控性,我们
This thesis is composed of two parts. In the first one, we have studied the dynamical behaviors of the asymmetric neural networks designed by t he Monte Carlo Adaptation rule, and their great potential for pattern reco gnition is mainly emphasized. The second one shows a primary investigat ion for the spatial shift of the solitons in nonlinear lattices.The Monte Carlo Adaptation (MC-adaptation) rule is proposed by Pro f. Hong Zhao to design asymmetric neural networks with associative me mory. The basic idea of this global study rule is to obtain a certain optimi zation by adaptively changing the values of the connection matrix. By ap plying this rule, one may find three different dynamical phases, i.e. the ch aos phase, the memory phase and the mixture phase, which are essentially important not only for practical applications but also for understanding th e global dynamical behaviors of the general discrete systems.Therefore, this thesis is start with a brief introduction of the Monte Ca rlo Adaptation rule. And then, we give the basic ideas of using the neural networks with chaos phase for pattern recognition. Their intrinsic advanta ges for this purpose are emphasized. To make them more utility and contr ollability, we have modified the original MC-adaptation rule. As an exam ple, this modified rule has been applied for the printed Chinese Character recognition. Based on the analysis of the distribution of the local fields ob
tained by the memory patterns, we have further extended the MC-adaptati on rule to be suitable for designing the multi-states neural networks.Finally, we quantitatively measure the spatial shifts of the solitons usi ng the method of molecular dynamics. Although it is commonly believed that the solitons will make a random spatial shift when it interacts with an other quasiparticle, the corresponding "signature" has never been observe d neither experimentally or in a numerical simulation. For the first time, we give a direct evidence of this phenomenon and make a qualitative anal ysis for its mechanism.
蒙特卡罗—选择变异规则(简称MCA)是由赵鸿教授提出的用于设计非对称神经网络的一种整体学习规则。其主要思想是通过有选择地随机改变连接矩阵使系统在给定的训练目标下达到整体最优。对于反馈神经网络这类高维非线性动力系统而言,其训练目标是控制存储模式所激发的神经元局域场分布从而使系统获得最佳性能。由此发现的参数空间的三个动力学相区(即:“混沌相”、“记忆相”和“混合相”)不但对实际应用中表现出极大的优越性,而且对于研究这类时空离散动力系统的整体行为也具有重要的理论价值。
因此,在本文中我们首先介绍了MCA学习规则及其在整个神经网络理论体系中所处的地位和作用,并通过选择投影的方法对不同相区系统的状态空间做了进一步刻画和描述。接着,我们详细讨论了“混沌相”在模式识别中的应用。处于这一参数区的神经网络具有两个基本性质:其一是各个存储模式的吸引域都很小而且被一个充满状态空间的混沌轨道彼此分开;其二是伪吸引子原则上被完全消除。正是这两个特性使得这类神经网络克服了传统反馈神经网络的固有缺陷而成为模式识别的有力工具。为了使它们更具有实用性和可控性,我们
This thesis is composed of two parts. In the first one, we have studied the dynamical behaviors of the asymmetric neural networks designed by t he Monte Carlo Adaptation rule, and their great potential for pattern reco gnition is mainly emphasized. The second one shows a primary investigat ion for the spatial shift of the solitons in nonlinear lattices.The Monte Carlo Adaptation (MC-adaptation) rule is proposed by Pro f. Hong Zhao to design asymmetric neural networks with associative me mory. The basic idea of this global study rule is to obtain a certain optimi zation by adaptively changing the values of the connection matrix. By ap plying this rule, one may find three different dynamical phases, i.e. the ch aos phase, the memory phase and the mixture phase, which are essentially important not only for practical applications but also for understanding th e global dynamical behaviors of the general discrete systems.Therefore, this thesis is start with a brief introduction of the Monte Ca rlo Adaptation rule. And then, we give the basic ideas of using the neural networks with chaos phase for pattern recognition. Their intrinsic advanta ges for this purpose are emphasized. To make them more utility and contr ollability, we have modified the original MC-adaptation rule. As an exam ple, this modified rule has been applied for the printed Chinese Character recognition. Based on the analysis of the distribution of the local fields ob
tained by the memory patterns, we have further extended the MC-adaptati on rule to be suitable for designing the multi-states neural networks.Finally, we quantitatively measure the spatial shifts of the solitons usi ng the method of molecular dynamics. Although it is commonly believed that the solitons will make a random spatial shift when it interacts with an other quasiparticle, the corresponding "signature" has never been observe d neither experimentally or in a numerical simulation. For the first time, we give a direct evidence of this phenomenon and make a qualitative anal ysis for its mechanism.
引文
[1] Maxwell, J. C., On governors, Proc. R. Soc. Lond., 1868, vol.16, 270-283.
[2] Hodgkin, A. L., and Huxley, A. F., A quantitative description of mem-brane current and its application to conduction and excitation in nerve, J. Physiol. Lond. 117, 1952:500-544.
[3] Xiao-Jing Wang and Gyorgy Buzsaki, Gamma Oscillation by Synapti c Inhibition in a Hippocampal Interneuronal Network Model., J. Neurosc i., October 15, 1996, 16(20):6402-6413 6403.
[4] DJ Surmeier, J Bargas, and HC Hemmings., Modulation of calcium c urrents by a D1 dopaminergic protein kinase/phosphatase cascade in rat n eostriatal neurons, Neuron, 1995 Feb;14(2):385-97.
[5] Idan Segev and Michael London, Untangling Dendrites with Quantita tive Models, Scinenee, 27 October 2000 VOL 290,744-750.
[6] Alain Destexhe and Denis Par., Impact of Network Activity on the Integrative Properties of Neocortical Pyramidal Neurons In Vivo, Journal of Neurophysiology, 1999.
[7] McCulloch W S, Pitts W., A Logical Calculus of the Ideas Immanent in Nervous Activity, Bulletin of Mathematical iophysics, 1943, (5): 115～133.
[8] Hans Scharstein, Input-output relationship of the Leaky-Integrator Ne uron Model, Journal of Mathematical Biology, 1979, Vo18, No.4, 403-420.
[9] W Gerstnerand WM Kistler, Spiking Neuron Models: Single Neurons Populations, Plasticity, Cambridge University Press. 2002.ISBN: 05 218 90799.
[10] Wolfgang Maass, Pulsed neural networks, MIT Press Cambridge, M A, USA ISBN: 0-626-13350-4.
[2] Hodgkin, A. L., and Huxley, A. F., A quantitative description of mem-brane current and its application to conduction and excitation in nerve, J. Physiol. Lond. 117, 1952:500-544.
[3] Xiao-Jing Wang and Gyorgy Buzsaki, Gamma Oscillation by Synapti c Inhibition in a Hippocampal Interneuronal Network Model., J. Neurosc i., October 15, 1996, 16(20):6402-6413 6403.
[4] DJ Surmeier, J Bargas, and HC Hemmings., Modulation of calcium c urrents by a D1 dopaminergic protein kinase/phosphatase cascade in rat n eostriatal neurons, Neuron, 1995 Feb;14(2):385-97.
[5] Idan Segev and Michael London, Untangling Dendrites with Quantita tive Models, Scinenee, 27 October 2000 VOL 290,744-750.
[6] Alain Destexhe and Denis Par., Impact of Network Activity on the Integrative Properties of Neocortical Pyramidal Neurons In Vivo, Journal of Neurophysiology, 1999.
[7] McCulloch W S, Pitts W., A Logical Calculus of the Ideas Immanent in Nervous Activity, Bulletin of Mathematical iophysics, 1943, (5): 115～133.
[8] Hans Scharstein, Input-output relationship of the Leaky-Integrator Ne uron Model, Journal of Mathematical Biology, 1979, Vo18, No.4, 403-420.
[9] W Gerstnerand WM Kistler, Spiking Neuron Models: Single Neurons Populations, Plasticity, Cambridge University Press. 2002.ISBN: 05 218 90799.
[10] Wolfgang Maass, Pulsed neural networks, MIT Press Cambridge, M A, USA ISBN: 0-626-13350-4.