神经元电路设计实现的研究
|
摘要
人工神经网络是现代信息处理领域的一个重要方法。相对于软件实现,硬件实现方法能充分发挥神经网络并行处理的特点。用模拟电路实现神经网络电路形式简单、功耗低、速度快等优点,占用芯片面积小,可以提高在神经网络芯片上神经元的集成度。而数字电路则易于存储信息。因此数模混合实现神经元电路是一种比较理想的方式。
文中分析了神经元的模型,提出了电路实现的方法。主要从减少神经网络中神经元个数及提高单个神经元信息处理能力两个角度出发,以数字逻辑的多阈值神经网络实现作为研究内容,提出用较少的神经元实现数字逻辑,并且使实现的数字逻辑具有较高信息密度的可能性。
首先分析了多阈值神经元的工作原理,并提出设计多阈值神经元的方法。用两个MOS晶体管组成突触电路,然后基于开关电路,结合限幅电压开关理论提出多阈值神经元阈值判别函数的开关级设计方法。从开关级设计实现了三值逻辑运算中与、非、文字运算的多阈值神经元电路。对设计出的电路进行PSPICE模拟并测量相关参数,结果表明,该设计方法简便、规范,结构简单,并且在实现相同逻辑功能时,采用多阈值神经元电路相对于单阈值神经元电路,大幅降低硬件成本。
最后文中还给出了一种基于双MOS管的数字权值存储式突触电路的方法,建立了多阈值神经元的多值逻辑运算统一模型。
Artificial neural network is an important method in the field of modern information processing.Comparing to software realization method,hardware realization method can fully exert the characteristics of parallel work of neural network.Neural networks realized by analog circuits have simpler structure,lower power consumption and higher speed and occupy small chip area than by digital circuits,but digital circuit is more easy storage information.So neuron circuit Realized by analog/digital mixed circuit is an ideal method.
By analysing the principle of nerton,a method was developed for designing neuron circuits in this paper.Mainly proceeding from the mission of reducing neuron numbers in neural networks(NNS)and enhancing the information proceeing abilities of single neuron,as well focusing on the digital logic implementation by multi-thresholded neurons(MTNS),this dissertation makes it possible that digital logic can be implemented by a small mumber of neurons and the NNS realize digital logic may have high information density.
First analyzing the principle of multi-thresholded neurons(MTNs),give the designing multi-thresholded neuron circuits(MTNCs).First,the voltage-mode synapse circuits wan designed using by two MOS transistors.Then based on a verdict-converting switch(VCS),an approach was propose for designing the circuits of multi-thresholded verdict function(MTVF) at switch level.Several MTNCs were designed for implementing the literal、AND、NOT operation as three operations in ternary logic at switch level.The result of simulation with PSPICE showed that the designed circuits had the correct logic function with simple structure,what's more,when achieving the same logic function,The MTNCs decreased large amount of hardware expense compared with the circuits.
Finally the paper give a method was developed for designing the synapse circuits of digital weight-storage base on two MOS transistors.Building the unified model for multi-thresholded neurons in Multi-valued operations.
文中分析了神经元的模型,提出了电路实现的方法。主要从减少神经网络中神经元个数及提高单个神经元信息处理能力两个角度出发,以数字逻辑的多阈值神经网络实现作为研究内容,提出用较少的神经元实现数字逻辑,并且使实现的数字逻辑具有较高信息密度的可能性。
首先分析了多阈值神经元的工作原理,并提出设计多阈值神经元的方法。用两个MOS晶体管组成突触电路,然后基于开关电路,结合限幅电压开关理论提出多阈值神经元阈值判别函数的开关级设计方法。从开关级设计实现了三值逻辑运算中与、非、文字运算的多阈值神经元电路。对设计出的电路进行PSPICE模拟并测量相关参数,结果表明,该设计方法简便、规范,结构简单,并且在实现相同逻辑功能时,采用多阈值神经元电路相对于单阈值神经元电路,大幅降低硬件成本。
最后文中还给出了一种基于双MOS管的数字权值存储式突触电路的方法,建立了多阈值神经元的多值逻辑运算统一模型。
Artificial neural network is an important method in the field of modern information processing.Comparing to software realization method,hardware realization method can fully exert the characteristics of parallel work of neural network.Neural networks realized by analog circuits have simpler structure,lower power consumption and higher speed and occupy small chip area than by digital circuits,but digital circuit is more easy storage information.So neuron circuit Realized by analog/digital mixed circuit is an ideal method.
By analysing the principle of nerton,a method was developed for designing neuron circuits in this paper.Mainly proceeding from the mission of reducing neuron numbers in neural networks(NNS)and enhancing the information proceeing abilities of single neuron,as well focusing on the digital logic implementation by multi-thresholded neurons(MTNS),this dissertation makes it possible that digital logic can be implemented by a small mumber of neurons and the NNS realize digital logic may have high information density.
First analyzing the principle of multi-thresholded neurons(MTNs),give the designing multi-thresholded neuron circuits(MTNCs).First,the voltage-mode synapse circuits wan designed using by two MOS transistors.Then based on a verdict-converting switch(VCS),an approach was propose for designing the circuits of multi-thresholded verdict function(MTVF) at switch level.Several MTNCs were designed for implementing the literal、AND、NOT operation as three operations in ternary logic at switch level.The result of simulation with PSPICE showed that the designed circuits had the correct logic function with simple structure,what's more,when achieving the same logic function,The MTNCs decreased large amount of hardware expense compared with the circuits.
Finally the paper give a method was developed for designing the synapse circuits of digital weight-storage base on two MOS transistors.Building the unified model for multi-thresholded neurons in Multi-valued operations.
引文
[1][日]山口亨,等,(马炫泽),人工神经网络与模糊信号处理[M]。科学出版社,2003
[2]Lau Clifford ed Neural networks:Theoretical foundation and analysis.A Selected Preprint Volume,Neural Networks Count- cil,Sponsor,IEEE Press,19922
[3]Gisutham B.,Srikanthan T.,Asari K.V.Ahigh speed flat CORCIC based neuron with multi-level activation function for robust pattern recognition.In:Proceedings of the 5th IEEE International Workshop on Computer Architectures for MachinePerception,Padova.2000,87-94
[4].周伟雄,靳东明,李志坚.模拟神经元电路实现研究现状与进展.固体电子学研究与进展.2002,Vol.22,No.3,2 69-279.
[5]Draghici Sorin.Neural Networks in Analog Hardware-Design andImpiemerttation Issues}R 」.48202.Detroit.MI.USA.Wayne State University.2000
[6]陈 卢.基于PWM的VLSI神经网络和CPU电源控制器的实现研究[D].博士学位论文,北京:清华大学微电子研究所,2001
[7]Moraga C.,Heider R.New lamps for old igeneralized mul-tiple-valued neurons.In:Proceedings of the 29th IEEE Inter-national Symposium on Multiple-Valued Logic,Freiburg.1999,36-41
[8]Wang Shou-Jue.Multi-valued neuron(MVN)and multi-thresholded neuron(MTN),their combination and application,Acta Electronics Silica,1996,3(3):1-6lin Chinese)
[9]Torsten L.Hardware learning in analogue VLSI neural net-works.Lyngby:University of Denmark.1994
[10]Chible H..Analog circuit for synapse neural networks VLSI implementation.In Proceedings of the 7th IEEE International Conference on Circuits and Systems,Iounieh,2000,2:1004-1007
[11]Bermak A.,Bouzerdoum A..VLSI implementation of a neural network classifier based on the saturating linear activation Function.In:Proceedings of the 9th International Con ference on Neural Information Processing,Singapore,2002,981-985
[12]Cesare A..Meyer E..Hardware requirements for digital VLSI implementation of neural networks,In:Proceedings of International Ioint Conference on Neural Networks,Singapore,1991,1873-1878
[13]Gao Li-Na,Qiu Guan-Yuan,A programmable synapse analog CMOS neural network,Acta Electronics Sinica,1998,26(2):120-122(in Chinese)(高丽娜,邱关源.一种连续可调权值CMOS神经网络.电子学报,1998,26(2):120-122)
[14]Wu Xun-Wei.Design Principles of Multivalued Logic Circuits.Hangzhou;Hangzhou University Press,1994(in Chinese)(吴训威.多值逻辑电路设计原理.杭州:杭州大学出版社 1994.18-25)
[15]Moritoshi Y.Ippei H.SOM(Self-Organizing Map) im- plemented by wafer scale integration-its self-organizing behavior under defects}A}.Proceedings of the Annual IEEE International Conference on Innovative Systems in Silicon[C],Univ of Tsukuba 1996:323-329
[16]Moritoshi Y,Ippei H,KeijiM,et al.Fault-tolerant self-organizing map implemented by wafer-scale in-tegration[J].IEEE Transcations on Very Large Scale Integration(VLSI)Systems,1998:257-264
[17]Faknraie,Sied Mehdi.VLSI Compatible Implemerua-lions for Artifical Neural Networks[M],Boston[M].Boston:Kluwer Academic Publishers,1997
[18]沈杰.靳东明.李志坚.双MOS 晶体 管等效电阻[J].清华大学学报(自然科学版).1999,39(5)104-107
[19]王守觉,李兆洲,陈向东等.通用神经网络硬件中神经元基本数学模型的讨论[J].电子学报,2001;29(5):577-580
[20]Joogho Choi.Bing J Sheu,Josephine,A gaussian synapse circuit for analog VLSI neural networks[J]. IEEE Transcations on Very Large Scale Integration(V LSI )Systems, 1994:2(1) : 129-133
[21] Vittoz E, Oguey H. Maher M A. et al. Analog storage of adjustable aynaptic weights [A]. Neural Networks [C]. Davalo. Eric. London:Macmillan, 1991
[22] Hafliger H. Rasche C. Floating gate analog memory for parameter and variable ltorage in a learning silicon neuron [J].Proceedings of the 1999 IEEE International Syposium on Circuits and syterms. 1999, 21416～41928
[23]Rodriguez E. Quintana J M. Avedillo M J. et al. Sorting networks implemented as vMOS circuits. Electronics Letters , 1998:34(23): 2237- 2238
[24] Sage Jay P . Withers Richard S. MVOS/CCD circuits for neural network implementation [J]. Proceedings IEEE International Symposium on Circuits and sytems, 1989:2:1 207—1209
[25] Holmes A J, Churcher S, Hajto J. et al. Design of analogue synapse circuits using non-volatile a-Si :H memory devices [J]. Proceedings of IEEE International Sym_ posium on Circuits and Systems, 1994,61351-35433
[26] Lau K T. Lee S T, Ong V K S. Four-quadrant analogue CMOS multiplier cell for VLSI signal and information processing[J]. IEEE Proceedings:Circuits, Devices and Systems, 1998:145(2) : 132-13434
[27] Coue D, Wilson G. A four-quadrant subthreshold mode multiplier for analog neural network applications[J]. IEEE Transactions on Neural Network, 1996: 7(5):1212-121940
[28]Montalvo A J. Gyurcsik R S, Paulo J J. Toward a general-purpose analog VLSI neural network with on-chip peturbation [J]. IEEE Transactious on Neural Net works, 1997,8(2) :413— 423
[29]McCulloch W S.Pitts W. A logical calculus of the ideas in nervous activity. Bulletin of Mathematical. 1943, (5) : 115—133
[30] 戴葵．神经网络实现技术。国防科技大学出版社,长沙,1998
[31]Hebb D O. The organization of behavior, New York:wiley.1949
[32]Rosenblatt F. The Perception:A probabilistic model for information storage and organization in the brain. Psychological review, 1958, (65). 386—408
[33]Miroslav S. Fast neural network implementation. Neural Network World , 1999(5) ,375—391
[34]Thiran P, Peiris P, Hochet B. Quantization effects in digitally behaving circuit implementation of kohonen networks. IEEETrans. on Neural Network , 1994 , Vlo5. No3,450-458
[35]Yasunga M, Msuda N, Yagyu M, etal. Design fabrication and evaluation of a 5 inch wafer scale neural network LSI composed of 576 digital neural. Proc. Int. Joint Conf. on Neural Networks IJCNN'90, San Diego: 1990, 527-523
[36] John A L, Torsten Lehmann. An analog CMOS chip set for neural networks with arbitary topologies. IEEETrans . Neural Networks, 1992, Vol. 3, No. 3, 441—444.
[37]Soelberg Y, Sigvartsen R, Lande TS. An analog continuous-time neural network. Analog Integrated Circuits and Signal Processing ,1994, Vol.5 235-246
[38]Richard C,marwan J, et al. A hybrid analog and digtal VLSI neural network for intracardiac morphology classification. IEEE Journal of Solid State Circuits ,1995 , Vol. 30, No. 5, 108—111
[39]Alhalabj B A, Bayoumi M A, On-chip learning for a scalable hybird neural architecture. IEEE International Symposium on Circuits and Systems. 1997. 677—680.
[40]Diahanshahi H ,et al.A Unified Synapde-Neuron building block for hybrid VLSI Neural Netwoks.IEEE International Symposium on Circuits and Systems.1996.493-486
[41]Ezz I E,Hong K Y,Mohamed A Y.Implementations of artificial neural networks using current-mode pulse width modulation technique.IEEE Trams.on Neural Networks.1997,Vol.8,No.3,532-547
[42]Lehmann T.Implementation issues of self-learning pulsed integrated neural system.Proccedings of 13rd NORCHIP Conference,Copenhagen.1995
[43]Hamilton A,Churcher S,et al.Pulse stream VLSI circuits and systems:the Epsilon neural network chipset.Intenational Journal of Neural systems.1993,4(4),395-405
[44]Au R,Yamashita T,Shibata T,Ohmi T.Neuron-MOS multiple-valued memroy technology for intelligent data processing.The 41~(st) IEEE International Solid-State Circuits Conference.San Francisco:IEEE,1994.270-271.
[45]Ngom A,Stojmenovic I,Obradovic Z.Minimization of multivalued multithreshold perceptrous using genetic algorithms.Proccedings of the 28~(th) IEEE International Symposium on Multiple-Valued Logic.Fukuoka:IEEE,1998.209-214.
[46]Murray A F,Tarassenko].Analog neural VLSI,Apulse stream approach London:Chapman and Hall,1994.
[47]Xiaolei Zhu,Jizhong Shen.Circuit Implementation of Multi-Thresholded Neuron(MTN) Using BiCMOS Technology.Proceedings of International Joint ConFerence on Neural Networks,Montreal,Canada,July 31-August 4,2005.
[48]吴杰,邱关源.CMOS电流镜及其应用湖南大学学报.1996.23(0:99-100)
[49]马晓敏,胡子萍.数字逻辑的神经网络设计.电路与系统学报,1998,3(3)51-58.
[50]Shah P.A short term analogue memory[A].Proceedings of 18th European Solid State Circuits Conference[C],1992:127-130
[51]Mead C.Analog VLSI and Neural System[M],Mass:Addison-Wesley,1989
[52]Lee S T, Lau K T. Low power building block for artificial neural networks. Electronics Letters, 1995, 31 (19): 1618—1619
[2]Lau Clifford ed Neural networks:Theoretical foundation and analysis.A Selected Preprint Volume,Neural Networks Count- cil,Sponsor,IEEE Press,19922
[3]Gisutham B.,Srikanthan T.,Asari K.V.Ahigh speed flat CORCIC based neuron with multi-level activation function for robust pattern recognition.In:Proceedings of the 5th IEEE International Workshop on Computer Architectures for MachinePerception,Padova.2000,87-94
[4].周伟雄,靳东明,李志坚.模拟神经元电路实现研究现状与进展.固体电子学研究与进展.2002,Vol.22,No.3,2 69-279.
[5]Draghici Sorin.Neural Networks in Analog Hardware-Design andImpiemerttation Issues}R 」.48202.Detroit.MI.USA.Wayne State University.2000
[6]陈 卢.基于PWM的VLSI神经网络和CPU电源控制器的实现研究[D].博士学位论文,北京:清华大学微电子研究所,2001
[7]Moraga C.,Heider R.New lamps for old igeneralized mul-tiple-valued neurons.In:Proceedings of the 29th IEEE Inter-national Symposium on Multiple-Valued Logic,Freiburg.1999,36-41
[8]Wang Shou-Jue.Multi-valued neuron(MVN)and multi-thresholded neuron(MTN),their combination and application,Acta Electronics Silica,1996,3(3):1-6lin Chinese)
[9]Torsten L.Hardware learning in analogue VLSI neural net-works.Lyngby:University of Denmark.1994
[10]Chible H..Analog circuit for synapse neural networks VLSI implementation.In Proceedings of the 7th IEEE International Conference on Circuits and Systems,Iounieh,2000,2:1004-1007
[11]Bermak A.,Bouzerdoum A..VLSI implementation of a neural network classifier based on the saturating linear activation Function.In:Proceedings of the 9th International Con ference on Neural Information Processing,Singapore,2002,981-985
[12]Cesare A..Meyer E..Hardware requirements for digital VLSI implementation of neural networks,In:Proceedings of International Ioint Conference on Neural Networks,Singapore,1991,1873-1878
[13]Gao Li-Na,Qiu Guan-Yuan,A programmable synapse analog CMOS neural network,Acta Electronics Sinica,1998,26(2):120-122(in Chinese)(高丽娜,邱关源.一种连续可调权值CMOS神经网络.电子学报,1998,26(2):120-122)
[14]Wu Xun-Wei.Design Principles of Multivalued Logic Circuits.Hangzhou;Hangzhou University Press,1994(in Chinese)(吴训威.多值逻辑电路设计原理.杭州:杭州大学出版社 1994.18-25)
[15]Moritoshi Y.Ippei H.SOM(Self-Organizing Map) im- plemented by wafer scale integration-its self-organizing behavior under defects}A}.Proceedings of the Annual IEEE International Conference on Innovative Systems in Silicon[C],Univ of Tsukuba 1996:323-329
[16]Moritoshi Y,Ippei H,KeijiM,et al.Fault-tolerant self-organizing map implemented by wafer-scale in-tegration[J].IEEE Transcations on Very Large Scale Integration(VLSI)Systems,1998:257-264
[17]Faknraie,Sied Mehdi.VLSI Compatible Implemerua-lions for Artifical Neural Networks[M],Boston[M].Boston:Kluwer Academic Publishers,1997
[18]沈杰.靳东明.李志坚.双MOS 晶体 管等效电阻[J].清华大学学报(自然科学版).1999,39(5)104-107
[19]王守觉,李兆洲,陈向东等.通用神经网络硬件中神经元基本数学模型的讨论[J].电子学报,2001;29(5):577-580
[20]Joogho Choi.Bing J Sheu,Josephine,A gaussian synapse circuit for analog VLSI neural networks[J]. IEEE Transcations on Very Large Scale Integration(V LSI )Systems, 1994:2(1) : 129-133
[21] Vittoz E, Oguey H. Maher M A. et al. Analog storage of adjustable aynaptic weights [A]. Neural Networks [C]. Davalo. Eric. London:Macmillan, 1991
[22] Hafliger H. Rasche C. Floating gate analog memory for parameter and variable ltorage in a learning silicon neuron [J].Proceedings of the 1999 IEEE International Syposium on Circuits and syterms. 1999, 21416～41928
[23]Rodriguez E. Quintana J M. Avedillo M J. et al. Sorting networks implemented as vMOS circuits. Electronics Letters , 1998:34(23): 2237- 2238
[24] Sage Jay P . Withers Richard S. MVOS/CCD circuits for neural network implementation [J]. Proceedings IEEE International Symposium on Circuits and sytems, 1989:2:1 207—1209
[25] Holmes A J, Churcher S, Hajto J. et al. Design of analogue synapse circuits using non-volatile a-Si :H memory devices [J]. Proceedings of IEEE International Sym_ posium on Circuits and Systems, 1994,61351-35433
[26] Lau K T. Lee S T, Ong V K S. Four-quadrant analogue CMOS multiplier cell for VLSI signal and information processing[J]. IEEE Proceedings:Circuits, Devices and Systems, 1998:145(2) : 132-13434
[27] Coue D, Wilson G. A four-quadrant subthreshold mode multiplier for analog neural network applications[J]. IEEE Transactions on Neural Network, 1996: 7(5):1212-121940
[28]Montalvo A J. Gyurcsik R S, Paulo J J. Toward a general-purpose analog VLSI neural network with on-chip peturbation [J]. IEEE Transactious on Neural Net works, 1997,8(2) :413— 423
[29]McCulloch W S.Pitts W. A logical calculus of the ideas in nervous activity. Bulletin of Mathematical. 1943, (5) : 115—133
[30] 戴葵．神经网络实现技术。国防科技大学出版社,长沙,1998
[31]Hebb D O. The organization of behavior, New York:wiley.1949
[32]Rosenblatt F. The Perception:A probabilistic model for information storage and organization in the brain. Psychological review, 1958, (65). 386—408
[33]Miroslav S. Fast neural network implementation. Neural Network World , 1999(5) ,375—391
[34]Thiran P, Peiris P, Hochet B. Quantization effects in digitally behaving circuit implementation of kohonen networks. IEEETrans. on Neural Network , 1994 , Vlo5. No3,450-458
[35]Yasunga M, Msuda N, Yagyu M, etal. Design fabrication and evaluation of a 5 inch wafer scale neural network LSI composed of 576 digital neural. Proc. Int. Joint Conf. on Neural Networks IJCNN'90, San Diego: 1990, 527-523
[36] John A L, Torsten Lehmann. An analog CMOS chip set for neural networks with arbitary topologies. IEEETrans . Neural Networks, 1992, Vol. 3, No. 3, 441—444.
[37]Soelberg Y, Sigvartsen R, Lande TS. An analog continuous-time neural network. Analog Integrated Circuits and Signal Processing ,1994, Vol.5 235-246
[38]Richard C,marwan J, et al. A hybrid analog and digtal VLSI neural network for intracardiac morphology classification. IEEE Journal of Solid State Circuits ,1995 , Vol. 30, No. 5, 108—111
[39]Alhalabj B A, Bayoumi M A, On-chip learning for a scalable hybird neural architecture. IEEE International Symposium on Circuits and Systems. 1997. 677—680.
[40]Diahanshahi H ,et al.A Unified Synapde-Neuron building block for hybrid VLSI Neural Netwoks.IEEE International Symposium on Circuits and Systems.1996.493-486
[41]Ezz I E,Hong K Y,Mohamed A Y.Implementations of artificial neural networks using current-mode pulse width modulation technique.IEEE Trams.on Neural Networks.1997,Vol.8,No.3,532-547
[42]Lehmann T.Implementation issues of self-learning pulsed integrated neural system.Proccedings of 13rd NORCHIP Conference,Copenhagen.1995
[43]Hamilton A,Churcher S,et al.Pulse stream VLSI circuits and systems:the Epsilon neural network chipset.Intenational Journal of Neural systems.1993,4(4),395-405
[44]Au R,Yamashita T,Shibata T,Ohmi T.Neuron-MOS multiple-valued memroy technology for intelligent data processing.The 41~(st) IEEE International Solid-State Circuits Conference.San Francisco:IEEE,1994.270-271.
[45]Ngom A,Stojmenovic I,Obradovic Z.Minimization of multivalued multithreshold perceptrous using genetic algorithms.Proccedings of the 28~(th) IEEE International Symposium on Multiple-Valued Logic.Fukuoka:IEEE,1998.209-214.
[46]Murray A F,Tarassenko].Analog neural VLSI,Apulse stream approach London:Chapman and Hall,1994.
[47]Xiaolei Zhu,Jizhong Shen.Circuit Implementation of Multi-Thresholded Neuron(MTN) Using BiCMOS Technology.Proceedings of International Joint ConFerence on Neural Networks,Montreal,Canada,July 31-August 4,2005.
[48]吴杰,邱关源.CMOS电流镜及其应用湖南大学学报.1996.23(0:99-100)
[49]马晓敏,胡子萍.数字逻辑的神经网络设计.电路与系统学报,1998,3(3)51-58.
[50]Shah P.A short term analogue memory[A].Proceedings of 18th European Solid State Circuits Conference[C],1992:127-130
[51]Mead C.Analog VLSI and Neural System[M],Mass:Addison-Wesley,1989
[52]Lee S T, Lau K T. Low power building block for artificial neural networks. Electronics Letters, 1995, 31 (19): 1618—1619