摘要
基于存储器的神经网络模型(如N-tuple神经网络(NTNN)、稀疏分布存储器模型(SDM)等)，因其结构简单，硬件易实现；查表式的学习算法，运行速度快，因此受到众多研究者的关注。在许多领域获得了成功应用，也是神经网络走向商品化的基础模型。然而，也正是它们“无权值(weightless)计算”的结构等诸原因使它们表达非线性映射的能力不强，且其行为缺乏理论分析，因此影响了它们的进一步应用。本论文的目的是从修改它们的结构和学习算法着手进行推广性研究。通过学习能力分析、与相关模型的比较到人脸识别应用的研究都证实了推广后的新模型可行和有效。并且获得了更多原始模型不具有的性质，应用范围从原来只能进行二值模式的识别(非二值模式必须二值化)拓展到能直接处理实数向量输入模式来实现函数逼近、灰度人脸的识别。本论文的研究主要包括以下几方面的内容：
1、针对一类回归RAM式神经网络进行推广。提出了一个新型自适应模式识别系统一基于稀疏RAM的N-tuple神经网络模型(SR-NTNN)，它既可以用于模式识别，也能用于函数逼近，具有比较一般性的特点。增加的可调参数使新模型表现更灵活，同时减少了存储开销，抑制了RAM式神经网络容易饱和的问题。NTNN、SDM都可以作为其特例。最后，通过实验证实了它所具有的函数逼近能力。
2、在保留SDM的稀疏分布存储特征的基础上修改了SDM的结构和学习算法，提出逼近型的SDM模型。拓展了原SDM只能应用于联想记忆的应用范围，并且对新模型进行了学习能力的理论分析，分析表明它具有与小脑模型关节控制器(CMAC)相当的学习能力，尽管它们的量化方式完全不一样。逼近型SDM有优于CMAC的许多特点：不需要哈稀(Hashing)技术、不存在分块效应以及更易于理解和实现。理论分析和示例表明了该改进模型的合理性和有效性，函数逼近的效果优于CMAC。
3、经典的N-tuple分类器及单层查找感知器模型(SLLUP)由于结构简单、运算速度快、便于硬件实现而得到广泛应用。但由于它们均是基于RAM的结构而必须二值化输入样本，因此在进行大维数样本的识别问题时受到许多限制。为此将稀疏分布存储的概念引入SLLUP,提出了基于稀疏RAM的逼近型N-tuple模型。该模型从结构上看，囊括了逼近型SDM和SLLUP，但这决非只是结构上的简单推广，当稀疏地址编码直接采用实数向量时，新模型获得了SLLUP根本不可能具有的特点，即直接对输入模式进行N-tuple采样，使得处理大维数的样本数据真正成为可能，从而表现出较SLLUP明显的优势和更大的灵活性。函数逼近的实验表明该新模型通过选择适当的参数，逼近效果远优于SLLUP和逼近型SDM。
4、将CMAC、逼近型SDM、SLLUP以及基于稀疏RAM的逼近型N-tuple模
基于稀疏RAM的神经网络及其人脸识别应用研究
型等纳入到一个称为一般存储器神经网络（GMNN）的模型框架中，其共同的特点
是由三部分构成：输入空间量化、存储器地址产生器、查表式某种组合输出。其本
质是通过地址选择隐含地实现从低维到高维空间的映射从而能更好地解决非线性的
分类和回归问题，显示出核（kernel）方法特点。特别当产生的地址个数是有限常数
且网络输出是加权线性和时，证明了此类网络可以收敛到最小平方误差解，从而为
更好地应用和拓展此类网络提供了理论基础。
5、提出了一系列基于Ntuple特征子模式划分的人脸识别方法。在人脸识别应
用上的研究不仅证实了本文所提模型的直接处理大维数样本的能力，而且展示了一
类区别于基于显式地进行特征提取的人脸识别方法的新颖性。单个NtUple的分类或
逼近能力是弱的，但通过不同的组合各个Ndple的方式可改善系统的性能。由此提
出了结构式组合和输出组合的两种组合方式。稀疏RAM逼近型Ntuple网络即属结
构式；通过结合Naive七ayes规则和直接投票的方法等进行的组合属输出组合方式；
另外采用Boosting方法集成若干N－tuple，获得了结构组合输出再组合的高级集成方
法。实验是基于Benchmark人脸数据库ORL的，其特点：l人 既不需要图象的完整
特征，也无需进行特征提取预处理，只需少量N－tUple特征，再按所提方法进行组合；
n、输出组合方法对 N－tuple特征划分方式以及大小不敏感；3人 Boosting NtUples
与常规Boosting方法的区别在于不仅仅每个基本分类器使用不同的训练集，而且同
一个训练样本提交给不同基本分类器不同的特征。实验结果证实所有提出的方法均
获得了较好的识别效果（误识率在6．10％左右），所使用的特征数占整幅图象的10－
100％不等，但实验表明更多的特征并没有更好的效果。事实上，所提的各种组合方
法，尤其是 Boosting NtuPles方法，开辟了一条进行特征选择的新途径。
The memory-based neural networks, such as N-tuple neural network (NTNN) and Sparse distributed memory model (SDM) etc., have attracted great attention by virtue of their simple architectures for easy hardware implementation, of lookup tables algorithm for fast operation. Their successful applications in many areas make them act as the basis of a commercial product. It is just their "weightless" or "RAMnet" architectures that results in poor nonlinear map, and at the same time few theoretical analyses about their behaviors have been gotten, so that further applications are limited. This dissertation starts by modifying their structures and algorithms for improving their performances. The extensive studies of analyses for learning ability, comparisons with related models and face recognition applications confirm that our generalized models are feasible and effective. Many good properties are obtained that original models can't have, the scope of applications has been expanded from binary pattern recognition (i
f not, need to be converted to binary string) to function approximation and gray face recognition, all these benefit from the novel models' abilities to dealing with real vector inputs directly. Several contributions have been made as follows:
1. We generalize a class of regress NTNN with RAM and present a novel adaptive pattern recognition system桝 N-tuple neural network model with sparse RAM that can be applied to pattern recognition as well as function approximation task. The increased adjustable parameters make new model flexible, cut down memory requirement and restrain NTNN's defect of easy saturation. It is a general model to some extent in which both NTNN and SDM can be regarded as a special case. Finally, experiments have shown its ability of approximating functions.
2. The approximate SDM has been presented by modifying original SDM's structure and algorithm, retaining original SDM's characteristic of sparse distribution. It exceeds original SDM in application for original SDM is only applied to the associate memory. The theoretical analysis about the novel SDM shows that its learning ability is in common with CMAC while both quantification manners aren't the same. Furthermore, no block effect appears and Hashing technology is not used in new model but the reverse in CMAC. Theoretical analysis and example have shown this improved model effective and reasonable, better performance in function approximation than CMAC does.
3. SLLUP(single-layer lookup perceptrons) as well as classical N-tuple classifier is extensively used in many regions because of their simple architecture, fast
operation and easy realization to hardware. At the same time, this kind of architecture based on RAM results in that input sample need to be converted to binary vector. Consequently, their applications in large dimension sample recognition are limited. Therefore, the approximate N-tuple model based on sparse RAM has been presented by integrating SLLUP with sparse distributed memory, SLLUP as well as approximate SDM becomes a special case there. This is not simple generalizing architecture, since it is really possible that the novel model can deal with large dimensional samples directly when sparse address code is immediately real vector one, that is, N-tuple sampling can be operated on input examples directly that can't in SLLUP. Function approximation experiment demonstrates that this new model can get better performance than CMAC and approximate SDM by selecting parameters appropriately.
4. There is a kind of networks, such as CMAC, approximate type SDM, SLLUP and approximate type N-tuple network based on sparse RAM, titled a general memory neural network (GMNN). It consists of: input space quantification, memory address generator, combined output by memory lookup operations. The essence of its operation, analogous to kernel method, is that a nonlinear map can be achieved by address selection operation leading to a higher dimension space, so that better classification or regression performance can be obtaine
引文
[1] [Adini1997] Y. Adini et al: Face recognition: the problem of compensating for illumination changes. IEEE Trans. PAMI, 1997, 19(7), 721-732.
[2] [Albusl971] J. S. Albus: A theory of cerebellar function. Mathematical Biosciences, 1971, 10, 25-61.
[3] [Albus1975] J. S. Albus: A new approach to manipulator control: The cerebellar model articulation controller(CMAC), J. Dynamic Syst., mens., Contr., Trans. ASME, 1975, 97(3), 220-227.
[4] [Albus1979] J.S. Albus: A model of the brain for robot control, Part2: A neurological model, BYTE, 1979, 4(7), 54-95.
[5] [Albus1981] J.S. Albus: Brains, Behaviour, and Robotics. Byte Books, 1981.
[6] [Aleksander1984] Aleksander, I. Thomas, W. V., and Bowden, P.A.: WISARD—a radical step forward in image recognition, Sensor Review, July, 1984, 120-124.
[7] [Battiti1992] R. Battiti: First-and second-order methods for learning: Between steepest descent and newton's method, Neural Comput., vol. 4, 1992,141-166.
[8] [Bauer1999] Eric Bauer and Ron Kohavi: An empirical comparison of voting classification algorithms: bagging, boosting, and variants, Machine Learning, 1999, 36, 105-142.
[9] [Belhumeur1997] Peter N. Belhumeur, et al.: Eigenfaces vs. fisherfaces: Recognition using class specific linear projection, IEEE Trans. On PAMI, 1997, 19(7), 711-720.
[10] [Bishop1995] C. M. Bishop: Neural networks for pattern recognition. Clarendon press, Oxford. 1995.
[11] [Bledsoe1959] Bledsoe, W.,Browning, Ⅰ.: Pattern recognition and reading by machine. IRE Joint Computer Conference. 1959, 225-232.
[12] [Breiman1998] Leo Breiman: Arcing classifiers. Annals of Statistics, 1998, 26(3), 801-849.
[13] [Brunelli1993] Roberto Brunelli and Tomaso Poggio: Face recognition: Features versus templates, IEEE Trans. On PAMI, 1993, 15(10), 1042-1052.
[14] [Cappelli2001] Raffaele Cappelli, Dario Maio, and Davide Maltoni: Multispace KL for pattern representation and classification. IEEE Trans. On PAMI, 2001, 23(9), 977-996.
[15] [Chang2000] C. C. Chang, C. W. Hsu, and C.J. Lin: The analysis of decomposition methods for support vector machines, IEEE Trans. On Neural Networks, 2000,11(4), 1003-1008.
[16] [Chen1991] S. Chen, C. F. Cowan, and P.M. Grant: Orthogonal least squares learning algorithms for radial basis function networks, IEEE Trans. Neural Networks, 1991, 2, 302-309.
[17] [Chen1992] S. Chen, S. A. Billings, and P.M. Grant: Recursive hybrid algorithm for nonlinear system identification using radial basis function networks, Int. J. Contr.,1992, 55(5), 1051-1070.
[18] [Chen1993] C. L. Chen, W. C. Chen, and F. Y. Chang,: Hybrid learning algorithm for Gaussian potential function networks, Proc. Inst. Elect., 1993, 140(6), 442-448.
[19] [Chou1989] Philip A. Chou: The capacity of the Kanerva associative memory. IEEE Trans. on information theory, 1989, 35(2), 281-298.
[20] [Demers1993] D. Demers and G.W. Cottrell: Nonlinear dimensionality reduction, in Advances in Neural Information Processing Systems 5, S. J. Hanson, J. D. Cowan, and C. Lee Giles, Eds. San Mateo, CA: Morgan Kaufmann, 1993, 580-587.
[21] [Dollfus1999] D. Dollfus, L. Beaufort: Fat neural network for recognition of position-normalised objects. Neural Networks, 1999, 12, 553-560.
[22] [Domingos1997] Domingos. P.,Pazzani. M.: Beyond independence: conditions for the optimality of the simple Bayesian classifier. Machine Learning, 1997, 29(2/3),103-130.
[23] [Eduardo1997] Eduardo do Valle Simes et al.: The adaptive weight using RAM. IEEE intel, conf. of Systems, Man, Cyber.,1997, 4053-4056
[24] [Eldracher1997] Martin Eldracher, et al.: Adaptive encoding strongly improves function approximation with CMAC, Neural Computation, 1997, 9, 403-417.
[25] [Fahlman1988] S. E. Fahlman: Faster learning variations on Back Propagation: An empirical study, Proceedings of the 1988 connectionist model summer school, 1988,38-51.
[26] [Fairhurst2000] M.C. Fairhurst and M.S. Hoque: Moving window classifier: approach to off-line image recognition. Electronics Letters, 2000, 36(7), 628-630.
[27] [Fan1997] Kuo-chin Fan and Yuan-kai Wang: A genetic sparse distributed memory approach to the application of handwritten character recognition.
Pattern recognition, 1997, 30(12), 2015-2022.
[28] [Francisco1998a] Francisco J. Gonzlez-Serrano, et. al.: Generalizing CMAC architecture and training, IEEE Trans. On Neural Networks, 1998, 9(6), 1509-1514.
[29] [Francisco1998b] Francisco J. Gonzlez-Serrano, et. al.: Fourier analysis of the generalized CMAC neural network, Neural Networks, 1998, 11, 391-396.
[30] [Freund1995] Yoav Freund: Boosting a weak learning algorithm by majority. Information and Computation, 1995, 121(2), 256-285.
[31] [Freund1997] Yoav Freund and Robert E. Schapire: A decision-theoretic generalization of on-line learning and application to boosting. Journal of Computer and System Sciences, 1997, 55(1), 119-139.
[32] [Freund1999] Yoav Freund and Robert E. Schapire: A short introduction to boosting, Journal of Japanese Society for Artificial Intelligence, 1999, 14 (5), 771-780.
[33] [Funahashi1989] K. Funahashi: On the approximate realization of continuous mappings by neural networks. Neural Networks, 1989,2,183-192.
[34] [Funahashi1998] Ken-ichi Funahashi: Multilayer neural networks and Bayes decision theory. Neural Networks, 1998, 11, 209-213.
[35] [Geman1992] Geman. S.,Bienenstock. E.,and Doursat. R.: Neural networks and the bias/variance dilemma. Neural Computation, 1992, 4, 1-48.
[36] [Gutta2000] Srinivas Gutta et al.: Mixture of experts for classification of gender, ethnic origin, and pose of human faces, IEEE Trans. On Neural Networks, 2000, 11(4), 948-960.
[37] [Hastie1996] Trevor Hastie and Robert Tibshirani: Discriminant adaptive nearest neighbor classification, IEEE Trans. On PAMI, 1996, 18(6), 607-615.
[38] [Haykin2001] Simon Haykin: Neural networks-A comprehensive foundation, Second Edition, Beijing: Tsinghua University Press, 2001.
[39] [Hely1997] T.A. Hely et al.: A new approach to Kanerva's sparse distributed memory. IEEE Trans. on Neural Networks, 1997,8(3),791-794.
[40] [Hornik1991] K. Hornik: Approximation capabilities of multiplayer feedforward networks, Neural Networks, 1991, 4, 251-257.
[41] [Ichimura1996] T. Ichimura, T. Takano, and E. Tazaki: Learning of neural networks using hybrid genetic algorithm, in 4th European Congr. Intell. Techniques and Soft Computing (EUFIT' 96), 1996.
[42] [Jain2000] Anil K. Jain, Robert P. W. Duin, and Jianchang Mao: Statistical
pattern recogition: A review. IEEE Trans. on pattern analysis and machine intelligence, 2000, 22(1), 4-37.
[43] [Jung1996] Dz-Mou Jung, et al.: N-tuple features for OCR revisited, IEEE Trans. On PAMI, 1996, 18(7), 734-745.
[44] [Jrgensen1999] Thomas Martini Jrgensen and Christian Linneberg: Theoretical analysis and imoroved decision criteria for the n-tuple classifier. IEEE Trans. on PAMI, 1999, 21(4), 336-347.
[45] [Kanerva1988] P. Kanerva: Sparse distributed memory. MIT press, Cambridge, Massachusetts, 1988.
[46] [Kaur1998] Devinder Kaur and David Brownell: Associative RAM-net memory neural target classifier, Opt. Eng., 1998, 37(7), 2043-2054.
[47] [Keeler1988] James D. Keeler: Comparison between Kanerva's SDM and Hopfield-type neural networks. Cognitive Science 1988,12, 299-329.
[48] [Kim1992] Hyongsuk kim and Chun-Shin Lin: Use of adaptive resolution for better CMAC learning, IJCNN, 1992, 517-522.
[49] [Kohavi1996] Kohavi. R. and Wolpert. D.H.: Bias plus variance decomposition for zero-one loss functions, http://robotics.stanford.edu/users/ronnyk.
[50] [Kohonen1995] T. Kohonen: Self-Organizing Maps, Springer series in Information Sciences, vol. 30, Berlin, 1995.
[51] [Kolcz1994] A. Kolcz and N. M. Allinson: Application of the CMAC input encoding scheme in the N-tuple approximation network. IEE Proc.-comput. digit, tech., 1994, 141(3), 177-183.
[52] [Kolcz1995] Aleksander Kolcz and Nigel M. Allinson: General memory neural network-extending the properties of basis networks to RAM-based architectures. In Proc. 1995 IEEE Int. Conf. on Neural Networks(ICNN' 95), 1995, 1638-1643.
[53] [Kolcz1996] Aleksander Kolcz and Nigel M. Allinson: N-tuple Regression Network. Neural Networks, 1996, 9(5), 855-869.
[54] [Kolcz1999a] Aleksander Kolcz and Nigel M. Allinson: Basis function models of the CMAC network. Neural Networks, 1999, 12, 107-126.
[55] [Kolcz1999b] Aleksander Kolcz and Nigel M. Allinson: The general memory neural network and its relationship with basis function architectures. Neurocomputing. 1999, 29, 57-84.
[56] [Kolcz2000] Aleksander Kolcz: N-tuple network, CART, and Bagging, Neural Computation, 2000, 12, 293-304.
[57] [Lades1993] M. Lades et al: Distortion invariant object recognition in
the dynamic link architecture. IEEE Trans. Comput.,1993, 42, 300-311
[58] [Lane1992] Stephen H. Lane, David A. Handelman, and Jack J. Celfand: Theory and development of higher-order CMAC neural networks. IEEE Control System, Apri1,1992.
[59] [Lawrence1997] Steve Lawrence, et. al.: Face recognition: a convolutional neural-network approach, IEEE Trans. On Neural Networks, 1997, 8(1), 98-112.
[60] [Levy2000] Avraham Levy and Michael Lindenbaum: Sequential Karhunen-Loeve basis extraction and its application to images. IEEE Trans. On Image Processing, 2000, 9(8), 1371-1374.
[61] [Li1999] Stan Z. Li and Juwei Lu: Face recognition using the nearest feature line method, IEEE Trans. On Neural Networks, 1999, 10(2),439-443.
[62] [Lin1997] Chun-shin Lin and Ching-Tsan Chiang: Learning convergence of CMAC technique, IEEE Trans. on Neural networks, 1997, 8 (6), 1281-1292.
[63] [Lin1998] Chun-Shin Lin and Ching-Tsan Chiang: Integration of CMAC technique and weighted regression for efficient learning and output differentiability, IEEE Trans. on systems, man, and cybernetics—part B: cybernetics, 1998,28(2), 231-237.
[64] [Liu1998] Chengjun Liu and Harry Wechsler: Probabilistic reasoning models for face recognition. in Proc. IEEE Computer Soc. Conf. Computer Vision and Pattern Recognition, 1998, Santa Barbara, CA, 23-25. 827-832.
[65] [Liu2OOOa] Chengjun Liu and Harry Wechsler: Evolutionary pursuit and its application to face recognition. IEEE Trans. On PAMI, 2000, 22(6), 570-582.
[66] [Liu2OOOb] Chengjun Liu and Harry Wechsler: Robust coding schemes for indexing and retrieval from large face databases. IEEE Trans. On Image Processing, 2000, 9(1), 132-137.
[67] [Loog2001] Marco Loog et al. : Multiclass linear dimension reduction by weighted pairwise fisher criteria, IEEE Trans. PAMI, 2001, 23(7), 762-766.
[68] [Lucas1997] S.M. Lucas: Continuous n-tuple classifier and its application to face recognition, Electronics Letters, 1997, 33(20), 1676-1678.
[69] [Mao2000] K.Z. Mao, K.C. Tan, and W. Ser: Probabilistic Neural-Network structure determination for pattern classification, IEEE Trans. On Neural Networks, 2000, 11(4), 1009-1016.
[70] [Michie1994]Michie, D.,Spiegelhalter, D.J.,andTaylor, C.C.:Machine learning, neural and statistical classification. Englewood Cliffs, NJ:
Prentice-Hall, 1994.
[71] [Miller1990] W.T. Miller, F.H. Glanz, and L.G. Kraft: CMAC: An associative neural network alternative to backpropagation, PROC. IEEE, 1990, 78(10), 1561-1567.
[72] [Miller1998] David J. Miller and Hasan S. Uyar: Combined learning and use for amixturemodel equivalent to the RBFclassifier, Neural Computation, 1998,10,281-293.
[73] [Moghaddam1997] Baback Moghaddam, and Alex Pentland: Probabilistic Visual learning for object representation. IEEE Trans. On PAMI, 1997, 19(7), 696-710.
[74] [Moghaddam2000] Baback Moghaddam, Tony Jebara, Alex Pentland: Bayesian face recognition. Pattern Recognition. 2000, 33,1771-1782.
[75] [Moody1989] J.E. Moody and C.J. Darken: Fast learning in networks of locally tuned processing units, Neural Comput., 1989, 1, 281-294.
[76] [Murase1995] Hiroshi Murase and Michael Lindenbaum: Partial eigenvalue decomposition of large images using spatial temporal adaptive method, IEEE Trans. On Image Processing, 1995,4(5), 620-629.
[77] [Nagao1998] Kenji Nagao and Masaki Sohma: Weak orthogonalization of face and perturbation for recognition, In Proc. IEEE Compt. Soc. Conf. On CVPR, 1998, 23-25, 845-852.
[78] [Neal1996] R. Neal: Bayesian learning for neural networks, Spring Verlag, New York 1996.
[79] [Nie1994] Junhong Nie and D.A. Linkens: FCMAC: a fuzzified cerebellar model articulation controller with self-organizing capacity. Aucommatica, 1994, 30(4), 655-664.
[80] [Oja1992] E. Oja: Principal components, Minor components, and Linear neural networks, Neural Networks, 1992, 5(6), 927-936.
[81] [0ja1997] E. Oja: The nonlinear PCA learning rule in independent component analysis, Neurocomputing, 1997, 17(1),25-45.
[82] [ORL] http://www. camorl. co. uk/facedatabase, html.
[83] [Osuna1997] Edgar Osuna, Robert Freund, Federico Girosi: Training support vector machines: an application to face detection. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, New York:IEEE, 1997, 130-136.
[84] [Parlos1994] Alexander G. Parlos, Benito Fernandez, Amir F. Atiya, Jayakumar Muthusami, and Wei K. Tsai: An Accelerated Learning Algorithm for
Multilayer Perceptron Networks. IEEE Trans. on Neural Networks, 1994,5(3),493-497.
[85] [Phillips1997] P. Jonathon Phillips et al.: The FERET evaluation methodology for face recognition algorithms. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, New York:IEEE, 1997, 137-143.
[86] [Prager1989] R.W. Prager and F. Fallside: The modified Kanerva model for automatic speech recognition, Comput. Speech Language, 1989, 3, 61-81.
[87] [Psichogios1994] D.C. Psichogios, and L.H. Ungar: SVD-NET:An algorithm that automatically selects network structure, IEEE Trans. on Neural Networks, 1994,5(3),513-515.
[88] [Quinlan1996] J.R. Quinlan: Bagging, boosting and C4.5. In Proceedings of the Thirteenth National Conference On Artificial Intelligence, 1996, 725-730.
[89] [Rohwer1998] Richard Rohwer and Michal Morciniec: The Theoretical and Experimental Status of the n-tuple Classifier. Neural Networks, 1998, 11(1), 1-14.
[90] [Rumelhart1986] D.E. Rumelhart, G.E. Hinton, and R.J. Williams:Learning internal representations by error propagation, in Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol. 1. MIT Press, Cambridge, MA, 1986.
[91] [Ryan1995] Shaun W. Ryan and John H. Andreae: Improving the performance of Kanerva's associate memory. IEEE Trans. on Neural Networks, 1995,6(i),125-130.
[92] [Schapire1990] Robert E. Schapire: The strength of weak learnability. Machine Learning, 1990, 5(2),197-227.
[93] [Schapire1998] Robert E. Schapire, Yoav Freund, Peter Bartlett, Wee Sun Lee: Boosting the margin: A new explanation for the effectiveness of voting methods. The Annals of Statistics, 1998, 26(5), 1651-1686.
[94] [Sch(?)lkopf 1997] Bernhard Sch(?)lkopf et al.: Comparing support vector machines with Gaussian kernels to radial function classifiers. IEEE Trans. on signal processing, 1997, 45(11), 2758-2765.
[95] [Schwenk2000] Holger Schwenk, Yoshua Bengio: Boosting Neural Networks. Neural Computation, 2000, 12, 1869-1887.
[96] [Shah1999] Jagesh V. Shah and Chi-Sang Poon: Linear independence of internal representations in multilayer perceptrons. IEEE Trans. on Neural
Networks, 1999,10(1),10-18.
[97] [Smagt1994] P. Van Der Smagt: Minimzation methods for training feedforward neural networks, Neural Networks, 1994,7(1),1-11.
[98] [Sutanto1997] E.L. Sutanto, J.D. Mason, and K. Warwick: Mean-tracking clustering algorithm for radial basis function center selection, Int. J. Contr., 1997,67(6),961-977.
[99] [Swets1996] Daniel L. Swets and J.J. Weng: Using discriminant eigenfeatures for image retrieval. IEEE Trans. On PAMI, 1996, 18(8), 831-836.
[100] [Tarling1993] R. Tarling and R. Rohwer: Efficient use of training data in the n-tuple recogition method. Electronics Letters, 1993, 29(24), 2093-2094.
[101] [Tattersall1991] G.D. Tattersall, S. Foster and R.D. Johnston: Singlelayer lookup perceptrons. IEE PROCEEDINGS-F, 1991, 138(1), 46-54.
[102] [Tax2000] David M.J. Tax, et al.: Combining multiple classi fiers by averaging or by multiplying? Pattern Recognition, 2000, 33, 1475-1485.
[103] [Thompson1995] David E. Thompson and Sunggyu Kwon: Neighborhood sequential and random training techniques for CMAC, IEEE Trans. On Neural Networks, 1995,6(1),196-202.
[104] [Tomandl2001] Dirk Tomandl and Andres Schober:A modified regression neural network (MGRNN) with new, efficient training algorithms as a robust 'black box'-tool for data analysis. Neural Networks, 2001,14,1023-1034.
[105] [Turk1991] MatthewA. Turk and Alex P. Pentland: Face recognition using eigenfaces. In Proc. Of IEEE Conf. On CVPR, 1991,586-591.
[106] [Uykan1996] Z. Uykan ,and C. G(?)zelis: Input-output clustering for determining the centers of radial basis function network, Signal processing and its applications, 1996, Antalya, Turkey, 677-682.
[107] [Uykan2000] ZekeriyaUykan, C. G(?)zelis,et. al.:Analysis of input-output clustering for determining centers of RBFN. IEEE Trans. on Neural networks, 2000,11(4),851-858.
[108] [Wang2000] Wang shi-tong, Baldwin, J.F.,Martin, T.P.: Theoretical results on learning convergence of generalized fuzzy CMAC, Journal of Software, 2000, 11(11), 1440-1450.
[109] [Webb1998] Andrew R. Webb and Simon Shannon:Shape-adaptive radial basis functions. IEEE Trans. on Neural Networks, 1998, 9(6), 1155-1166.
[110] [White1989] H. White: Learning in artificial neural networks. Neural
Comput. 1989, 1(4),425-464.
[111] [Wong1992] Yiu-fai Wong and Athanasios Sideris:hearning convergence in the cerebellar model articulation controller, IEEE Trans. on Neural networks, 1992, 3 (1), 115-121.
[l12] [Woodward1997]J.D. Woodward: Biometrics: privacy's foe or privacy's friend? Proceedings of IEEE, 1997, 85(9), 1480-1492.
[113] [Yang1992] Yang Guoqing, Chen Songcan and Lu Jun: Multilayer Parallel Distributed Pattern Recognition System Model Using Sparse RAM Nets. IEE PROCEEDINGS-E, 1992, 139(2), 144-146.
[114] [陈松灿1992]陈松灿等：面向问题的稀疏分布式记忆模型，航空学报，1992，13(12)，665-669。
[115] [丁丽娟1997]丁丽娟：数值计算方法，北京：北京理工大学出版社，1997。
[116] [何超2001]何超，徐立新，张宇河：CMAC算法收敛性分析及泛化能力研究，控制与决策，2001，16(5)，523-529。
[117] [金忠1999]金忠，杨静宇，陆建峰：一种具有统计不相关性的最佳鉴别矢量集，计算机学报，1999，22(10)，1-4。
1118] [罗忠1997]罗忠，谢永斌，朱重光：CMAC学习过程收敛性的研究，自动化学报，1997，23(4)，455-461。
[119] [史荣昌1996]史荣昌：矩阵分析，北京：北京理工大学出版社，1996。
[120] [张翠平2000)张翠平，苏光大：人脸识别技术综述，中国图象图形学报，2000，5(A)(11)，885-894。
[121] [张学工2000]张学工(译)：统计学习理论的本质，北京：清华大学出版社，2000。
[122] [周杰2000]周杰，卢春雨，张长水，李衍达：人脸自动识别方法综述，电子学报，2000，28(4)，102-106。
[123] [周志华2002]周志华，陈世福：神经网络集成，计算机学报，2002，25(1)，1-8。