摘要
近十年来，基于统计学习理论的支持向量机方法逐渐成为机器学习的重要研究方向。与传统的基于经验风险最小化原则的学习方法不同，支持向量机基于结构风险最小化，能在训练误差和分类器容量之间达到一个较好的平衡，它具有全局最优、适应性强、推广能力强等优点。但是直到目前为止，支持向量机方法还存在一些问题，例如训练时间过长、核参数的选择等，成为限制支持向量机应用的瓶颈。本文的研究主要围绕以上两个问题展开，研究结果在多个国际通用的基准数据集上进行验证。
论文的主要成果如下：
1)系统地研究了支持向量机的训练方法。目前支持向量机的训练算法是以序贯最小最优化(SMO)为代表的，其中工作集的选择是实现SMO算法的关键。本文对基于Zoutendijk最大下降方向法和函数逼近的工作集选择方式进行了总结和整理，并对这种选择策略重新进行了严格的数学推导。研究指出，当二次规划问题的Gram矩阵在非正定的情况下，目前存在的工作集选择算法存在某些不足。
2)对于大规模训练集的缩减研究。支持向量机在小样本情况下具有优于别的机器学习算法的性能，但并不意味着支持向量机只限于应用在小样本情况。现实中的问题大多具有大规模的样本，虽然目前有了以SMO为代表的快速训练算法，但对于大规模训练集仍然存在训练时间过长的缺点，不能满足实时性的要求。本文根据支持向量的几何分布，提出了在原输入空间和高维映射空间中预选支持向量的两种方法。原输入空间预选支持向量方法是受启发于最近邻规则，通过与支持向量的几何分布结合，使用Delaunay三角网络寻求包含支持向量的边界集的原理。受聚类方法的启发，基于样本类别质心的方法实现了高维特征空间支持向量的预选。实验证明这两种支持向量预选策略是有效的，在大幅缩减训练时间的同时基本不损失SVM的推广能力和预测性能。
3)对支持向量机模型选择的研究。支持向量机通过核函数将样本从输入空间映射到高维特征空间(Hilbert空间)，从而实现在特征空间中寻求线性判别超平面。但是，不同的核对应着不同的特征空间，而支持向量机的训练结果在不同的核映射下往往有不同的效果。本文通过对像集线性可分程度和模型复杂程度的估计，寻找可以使学习机器具有良好推广能力的特征空间，并以此为标准实现核的选择。特征空间确定之后，分析惩罚因子与间隔宽度之间的关系，通过间隔宽度实现对惩罚因子的选择。本文的模型选择方法并不寻求核函数、惩罚因子与学习机器推广能力之间的解析表达式，而是以间接的方法估计参数对学习机器推广能力的影响，指导模型的选择。
4)对机器学习的实际应用的研究。本文对机器学习的重要问题——人脸识别进行了研究，提出了一种基于关键部件的人脸识别方法。由于一对余多类分类算法缺乏理论上的依据，本文以后验概率作为支持向量机的输出，实现了以相似度为判别标准的多类分类算法。对ORL和YALE人脸图像数据库进行仿真实验，结果表明，该方法具有对表情、姿态以及角度的变化具有较好的鲁棒性。本文研究了SVM在金融领域的一个典型应用——个人信用评估，主要探讨了基于SVM的特征选择和提取方法(遗传算法和主分量分析法)的实际应用效果。实证分析表明，小样本信用数据下SVM的准确度和推广性能显著好于BP神经网络；基于遗传算法的SVM能使银行检测出信用评级的关键决定因素。这对于我国银行进行个人信用评价具有重要的现实意义。
During the last decades, the method of Support Vector Machine (SVM) based on the Statistical Learning Theory became an important research field in machine learning. Different from those traditional algorithms based on empirical risk minimization rule, SVM is based on structural risk minimization rule. So SVM can achieve a good balance between empirical risk and classifier capacity. In addition, SVM has other advantages such as global optimization, excellent adaptability and generalization ability. However, there are still some problems with SVM, such as too time-consuming and difficulty of selecting kernel parameters, which restricts the application of SVM. Thus, our study focuse on the above- mentioned issues. The research results have been tested on several benchmark data sets of the world.
The contributions of this dissertation include:
1) Study on the training algorithm of SVM. Currently, sequential minimal optimization (SMO) algorithm has become the best training algorithm for SVM, Working set selection is the key of implementing SMO. We studied the working set selection strategy based on Zoutendijk's maximal descent direction method and function approximation deeply, and deduced it strictly. We found that the existing selection method has some defaults when the Hessian matrix of the quadric programming problem is not positive definite.
2) Study on the reduction for large-scale training set. SVM has better performance than other learning algorithms in case of small sample. But that does not mean that SVM is only used for small sample. As a matter of fact, a majority of problems encountered in real world belong to large-scale data set. In case of large sample, even the SMO algorithm consumes too much training time and can not satisfy the real-time requirements. Based on the geometry distribution of the support vectors, two reduction strategies for pre-selection of support vectors in primal input and high-dimentional feature space are proposed. Enlightened by clustering method, the strategies for for pre-selection of support vectors in high-dimentional space is based on category centroid of sample. Enlightened by a combination of the nearest neighbor rule and geometric distribution, the strategies for for pre-selection of support vectors in primal input space is based on the approach of searching boundary set including support vector using Delaunay Triangulations network. Experiments results show that these two reduction strategies are effective in that they can reduce training time sharply without downgrading the generation ability and prediction accuray.
3) Study on model selection of SVM. After mapping the samples from primal input space to high-dimentional feature space (Hilbert space) using kernel function, we can obtain linear discriminant hyperplane in the feature space. Different kernels correspond to different feature spaces, and different results of SVMs are obtained by mapping based on different kernels. By measuring the linear discrimination degree and the complexity of models, a feature space with good generation ability for learning machine is found, and kernel selection is performed. Once feature space is determined, the relationship between the penalty factor and the margin is analyzed, and penalty factor is selected by means of the margin. Instead of building an analytic formula reflecting the relationship between kernel function, penalty factor and generalization ability, the proposed model selection method seeks to estimate the effect of parameter selection on generalization performance indirectly and provide a guide for model selection.
4) Study on application of SVM. In this paper, two typical problems are studied using SVM in detail. Firstly, for the typical problem of pattern classification - face recognition, a new identification method based on face component is proposed. Because one-against-rest SVM classifation algorithm is still lack of theoretic foundation, another multi-classification algorithm with similarity as discrimination standard, which use posterior probability as output of SVM is proposed. The experiment is conducted on the ORL and YALE face image database. The result indicates that the proposed method is robust in that it is insensitive to expression and pose variations. Secondly, a classical application in financial domain, personal credit evaluation, is studied. In particular, the application of two feature selection and extraction methods based on SVM (genetic algorithm and principal component analysis) is discussed. Empirical experiment gives useful suggestions. In case of small credit data sample, SVM outperforms BP neural network in terms of prediction accuracy and generaliazation ability. In addition, the hybrid method of combining SVM and genetic algorithm can help bank to identify the critical factors affecting credit evaluation. These conclusions can be of great significance for domestic banks to evaluate personel credit.
引文
[1] 史忠植．知识发现．北京：清华大学出版社，2002
[2] Alexander Gammerman. Machine Learning: Progress and Prospects. Royal Holloway. University of London. Egham, UK, 1996
[3] Nilsson J. Introduction to Machine Learning. Stanford University, 1996
[4] Fisher R A. The use of multiple measurements in taxonomic problems. Annals of Eugenics, 1936, 7(2): 179-188
[5] Rosenblatt F. The perceptron: a probabilistic model for information storage and organization in the brain. Psychological Review, 1958, 65:386-408
[6] Novikoff A B. On Convergence Proofs on Perceptrons. Proceedings of Symposium on the Mathematical Theory of Automata, Poltechnic Institute of Brooklyn, 1962(12): 615-622
[7] Minsky M, Parpert S. Perceptron. Cambridge, MA: MIT Press, 1969
[8] 王珏，石纯一．机器学习研究．广西师范大学学报(自然科学版)，2003，21(2)：1-15
[9] Ivanov V V. On linear problems which are not well-posed. Soviet Math Docl, 1962, 3(4):981-983
[10] Tikhonov A N. On solving ill-posed problem and method of regularization. Doklady Akademii Nauk USSR, 1963,153:50-504
[11] Vapnik V N, Stefanyuk A R. Nonparametric Methods for Estimation Probability Densities. Automation and Remote Control, 1978, 8:27-35
[12] Solomonoff R J. A Preliminary Report on General Theory of Inductive Inference. Technical Report ZTB- 138, Zator Company, Cambridge, 1960
[13] Kolmogorov A N. Three Approaches to the Quantitative Definition of Information. Problem of Information Transmission, 1965, 1 (1): 1-7
[14] Rissanen J. Modeling by Shortest Data Description. Automatica, 1978, 14: 465-471
[15] Rumellhart D E, Hinton G E, Williams R J. Learning Internal Representations by Error Propagation. Nature, 1986, 323(6188):533-536
[16] Pawlak Z. Rough-set theoretical aspects of reasoning about data. Boston, MA: Kluwer Academic Publishers, 1991
[17] Valiant L G. A theory of Leamability. Commun. ACM Press, 1984, 27(11):1134-1142
[18] Vapnik V N. Estimation of Dependences Based on Empirical Data. New York: Springer-Verlag, 1982
[19] Vapnik V N. The Nature of Statistical Learning Theory. Springer, New York,1995
[20] Vapnik V N. The Nature of Statistical Learning Theory. Springer, New York, 2000
[21] 张学工译．统计学习理论的本质．北京：清华大学出版社，2000
[22] Keams M, Vazirani U. An introduction to computational learning theory. Cambridge, MA: MIT Press, 1994
[23] Vapnik V N. Statistical Leaming Theory. Wiley, New York, 1998
[24] 许建华，张学工译．统计学习理论．北京：电子工业出版社，2004
[25] 边肇祺，张学工．模式识别．北京：清华大学出版社，2000
[26] Burges C J C. A Tutorial on Support Vector Machines for Pattern Recognition. Data Mining and Knowledge Discovery, 1998, 2(2): 121-167
[27] 邓乃扬，田英杰．数据挖掘中的新方法——支持向量机．北京：科学出版社，2004
[28] 卢增祥，李衍达．交互SVM学习算法及其在文本信息过滤中的应用．清华大学学报，1999
[29] 张铃，张钹．神经网络的规划学习算法．计算机学报，1994，17(9)：669-675
[30] 张铃．支持向量机理论与基于规划的神经网络学习算法．计算机学报，2001，24(2)：113-118
[31] Y. Lin. On the support vector machine. Technical Report 1029, Department of Statistics, University of Wisconsin, Madison WI, 2000
[32] 阎辉，张学工，马云潜，李衍达．基于变异函数的径向基核函数参数估计．自动化学报，2002，28(3)：450-455
[33] Mika S, Ratsch G, Weston J, et al. Fisher Discriminant Analysis With Kernels.Neural Networks for Signal Processing 9. New York:IEEE Press, 1999:41-48
[34] Scholkopf B, Smola A, Muller K R. Kernel principal component analysis. In Scholkopf B, Burges C J C, & A. J. Smola, editors, Advanced in Kernel Methods-Support Vector Machine. MIT Press, Cambridge, MA, 1999:327-352
[35] Platt J C. Fast training of support vector machines using sequential minimal optimization. Advances in Kernel Method-Support Vector Learning. Cambridge, MA: MIT Press, 1999. 185-208
[36] Platt J C. Using Sparseness and Analytic QP to Speed Train of SVMs. In: Advances in Neural Infomation Processing System, Cambridge, MA:MIT Press, 1999,11
[37] Keerthi S S, Shevade S, Bhattacharyya C, and Murthy K. Improvements to platt's SMO algorithm for SVM classifier design. Neural Computation, 2001, 13(3):637-649
[38] Keerthi S S, Gilbert E G Convergence of a generalized SMO algorithm for SVM classifier design. Machine Learning, 2002(46):351-360
[39] Zhang Ling, Zhang Bo. Neural Networks based on classifiers for a vast amount of data. In: Proceeding of 3rd Pacific-Asia Conference PAKDD-99, Methodologies for Knowledge Discovery and Data Mining. Berlin: Springer- Verlag, 1999:238-246
[40] Zhang Ling, Zhang Bo. A geometrical represention of McCulloch-Pitts neural model and its applications. IEEE Trans. Neural Networks, 1999, 10(4): 925-929
[41] Burges C J C, Scholkopf B. Improving the Accuracy and Speed of Support Vector Machines. In Advances in Neural Information Processing Systems 9,M.Mozer, M. Jordan and T. Petsche, Eds, MIT Press, Cambridge, MA, 1997:375-381
[42] Burges C J C. Simplified support vector decision rules. In L. Saitta, EDS., Proc. 13th International Conf, on Machine Learning, San Mateo, CA, 1996:71-77
[43] Scholkopf B, Knirsch P, Smola A. Fast approximation of support vector kernel expansions, and an interpretation of clustering as approximation in feature spaces. DAGM-Symposium, Berlin:Springer Verlag, 1998:124-132.
[44] Osuna E, Girosi F. Reducing the run-time complexity of support vector machines. In Alexander J. Smola; B. Scholkopf and Christopher J.C. Burges, Eds. Advance in kernel learning, MIT Press, 1999:271-284
[45] Scholkopf B, Smola A J, Bartleftt P L. New support vector algorithms. Neural Computation, 2000,12(5): 1207-1245
[46] Chang C C, Lin C J. Training nu-Support Vector Classifiers: Theory and Algorithms. Neural Computation, 2001,13(9):2119-2147
[47] Friess T T, et al. The kernel-adatron algorithm: A fast and simple learning procedure for support vector machines, ICML98,1998:188-196
[48] Mangasarian O L,Musicant D R. Successive overrelaxation for support vector machines.IEEE Trans.on Neural Networks, 1999,10(5): 1032-1037
[49] Scholkopf B, Platt J C, Shawe Taylor J, et al... Estimating the support of a high-dimensional distribution. Neural Computation, 2001,13(7): 1443-1471
[50] Francois Poulet. Multi-way distributed SVM algorithms. Proceeding of ECML/PKDD'2003 Int. Workshop on Parallel and Distributed Algorithms for Data Mining, 2003
[51] Lee Y J, Mangasarian O L. RSVM: Reduced support vector machines. Wisconsin: University of Wisconsin, 2000
[52] Suykens J A K, J Vandewalle. Least Squares support vector machine classifiers. Neural Processing Letters, 1999,9(3):293-300
[53] Chew H G, Bogner Robert E, Lim C C. Dual No-Support Vector Machine with Error Rate and Training Size Biasing. Proceedings of 26th IEEE ICASSP2001, Salt Lake City, USA, 2001:1269-1272
[54] Olvi L M, David R M. Successive overrelaxiation for support vector machines. IEEE Trans. Neural Networks. 1999,10(5): 1032-1037
[55] S. Tong, D. Koller. Support vector machine active learning with applications to text classification. In: Proc, of the 17th Int'l Conf, on Machine Learning,2000, 7
[56] S. Tong, E. Chang. Support vector machine active learning for image retrieval. In: Proc. ACM Multimedia 2001, Ottawa, Canada, 2001,9
[57] Welston J, Watkins C. Multiclass support vector machines. TR CSD-TR-98- 04, Dept. of Computer Science Egham, England, 1998
[58] Platt J, et al. Large Margin DAGs for multiclass classification. In: Advances in Neural Information Processing System 12, MIT Press, 2000:547-553_________
[59] B J Bredensteiner, K P Bennett. Multicategory classification by support vector machines. Computational Optimizations and Application, 1999:53-79
[60] K Crammer, Y Singer. On the leamability and design of output codes for multiclass problems. Computational Learning Theory, 2000:35-46
[61] Tax D, Duin R. Combining one-class classifiers. In: Proceedings of the Second International Workshop Multiple Classifier Systems, Lecture Notes in Computer Science. vol. 2096, Springer Verlag, Berlin, 2001:299-308
[62] Qi Miao, Shi-Fu Wang. Nonlinear Model Predictive Control Based on Support Vector Regression. In Proceedings of International Conference on Machine Learning and Cybernetics, 2002, 3:1657-1661
[63] Kruif B J, Vries T J A. On Using a Support Vector Machine in Learning Feed-Forward Control. In Proceedings of Int. Conf. on Advanced Intelligent Mechatronics, Como, Italy, July 2001:272-277
[64] Suykens J A K. Nonlinear Modeling and Support Vector Machines. In Proceedings of the 18th IEEE Instrumentation and Measurement Technology Conference, 2001, 1:287-294
[65] Rychetsky M, Ortmann S, Glesner M. Support Vector Approaches for Engine Knock Detection. International Joint Conference on Neural Networks, 1999, 2:969-974
[66] 李凌均，张周锁，何正嘉．基于支持向量机的机械故障智能分类研究．小型微型计算机系统，2004，25(4)：667-670
[67] 王成栋，朱永生，张优云等．时频分析与支持向量机在柴油机气阀故障诊断中的应用．内燃机学报，2004，22(3)：245-251
[68] Osuna E, Freund R, Girosi E Training support vector machines: an application to face detection. In: Proc. Computer Vision and Pattern Recognition, 1997:130-136
[69] Schmidt M. Identifying speaker with support vector network. In Interface'96 Proceedings. Sydney, 1996
[70] Blanz V, Scholkopf B, Burges J C, et al. Comparison of view-based object recognition algorithms using realistic 3d models. In C. Vonder Malsburg, W. von Seelen, J. C. Vorbuggen, et al. Artificial Neural Networks, ICANN'96:251-256, Berlin, 1996. Springer Lecture Notes in Computer Science, Vol. 1112
[71] Joachims T. Text categorization with support vector machines. Technical Report, LS_Ⅷ No.23, University of Dortmund, 1997
[72] Huang Z, Cheng H, Hsu C J, et al. Credit rating analysis with support vector machines and neural networks:a market comparative study. Decision Support Systems, 2004(37):543-558
[73] Kim K J. Financial time series forecasting using support vector machines. Neurocompution, 2003, 55(2):307-319
[74] Cristianini N, Shawe-Taylor J. An Introduction to Support Vector Machines and Other Kernel-based Learning Methods. Cambridge University Press, Cambridge, UK, 2000
[75] 薛毅．最优化原理与方法．北京：北京工业大学出版社，2003
[76] 褚蕾蕾，陈绥旭，周梦．计算智能的数学基础．北京：科学出版社，2002
[77] 袁亚湘，孙文瑜．最优化理论与方法．科学出版社，北京，2001
[78] Osuna E, Freund R, Girosi E Improved training algorithm for support vector machines. In Proc. IEEE Workshop on Neural Networks & Signal Processing, 1997:276-285
[79] Osuna E, Freund R, Girosi F. Support vector machines: Training and applicaitions. Technical Report A.I. Memo No. 1602, Artificial Intelligence Lab,MIT, 1997
[80] Joachims T. Making Large-Scale SVM Learning Practical. In: Scholkopf B, Burges C, Smola A, eds. Advances in Kernel Methods-Support Vector Learning.Cambridge: MIT Press, 1999:169-184
[81] Lin C J. Asymptotic convergence of an SMO algorithm without any assumptions. IEEE Transactions on Neural Networks, 2002, 13(1):248-250
[82] Lin C J. On the convergence of the decomposition method for support vector machines. IEEE Transactions on Neural Networks, 2001, 12(6): 1288-1298
[83] Fan R E, Chen P H, Lin C J. Working set selection using second order information for training SVM. Journal of Machine Leaming Research, 2005(6): 1889-1918
[84] Rychetsky M, Ortmann S, Ullmann M, et al. Accelerated training of support vector machines. In International Joint Conference on Neural Networks, Washington, USA, IEEE., 1999:998-1003
[85] YANG Minghsuan, Ahuja N. A geometric approach to train support vector machines. In Proceeding of CVPR 2000, Hilton Head Island, IEEE, 2000:430-437
[86] ZHANG L, ZHOU W D, JIAO L C. Pre-extracting Support vectors for support vector machine. In Proceeding of ICSP2000, Beijing, China, IEEE, 2000.1432-1435
[87] 李青，焦李成，周伟达．基于向量投影的支撑向量预选取．计算机学报，2005，28(2)：145-152
[88] 曹淑娟，刘小茂，张钧等．基于类中心思想的去边缘模糊支持向量机．计算机工程与应用，2006(22)：146-149
[89] Duda R O, Hart P E, Stork D G. Pattern classification. John Wiley and Sons, Inc, 2001
[90] Preparata F P，Shamos M J．计算几何导论．庄心谷译．北京：科学出版社，1992
[91] 周培德．计算几何——算法分析与设计．北京：清华大学出版社，2000
[92] Chang Chih Chung, Lin Chihjen. LIBSVM: a library for support vector machines, http://www.csie.ntu.edu.tw/～cjlin/libsvm
[93] Wu Youshou, Zhao Mingsheng, Ding Xiaoqing. A new kind of ANN based on active function and its application. Science in China (Series E), 1997,27(1):55-60(in Chinese)
[94] Murphy P. M., Aha D. W.. UCI machine leaming repository, http://www.ics.uci.edu/～mleam, 2004-01-15
[95] Joachims T. SVM~(light): Support Vector Machine. http://svmlight.joachims.org
[96] Collobert R, Bengio S. SVMTorch: support vector machines for large-scale regression problems. Journal of Machine Learning Research, 2001, 1:143-160 (http://www.idiap.ch/leaming/SVMTorch.html)
[97] Jaakkola T, Haussler D. Probabilistic Keme IRegression Models. Proceedings of the Seventh Workshop on AI and Statistics. San Francsico, 1999
[98] 张铃．基于核函数的SVM与三层前向神经网络的关系．计算机学报，2002，25(7)：696-700
[99] Hsu C W, Chang C C, Lin C J. A practical Guide to Support Vector Classification. http://www.csie.ntu.edu.tw/～cjlin
[100] Smola A J. Learning with kernels. Technical University of Berlin, 1998
[101] Bengio Y. Gradient-Based Optimization of Hyper-Parameters. Neural Computation, 2002, 12(8):1889-1900
[102] Lin H T, Lin C J. A study on sigmoid kernels for SVM and the training of non-PSD kernels by SMO-type methods, http://www.csie.ntu.edu.tw/～cjlin
[103] Scholkopf B, et al. Comparing Support Vector Machines with Gaussian Kernels to Radial Basis Function Classifiers. IEEE Trans. On Signal Processing, 1997, 45:2758-2765
[104] Joachims T. Estimating the Generalization Performance of a SVM Efficiently. LS Ⅷ-Report 25, University Dortmund, Germany, 1999
[105] 董春曦，饶鲜，杨绍全，徐松涛．支持向量机参数选择方法研究．系统工程与电子技术，2004，26(8)：1117-1120
[106] Chung K M, Kao W C, Sun C L, et al. Radius margin bounds for support vector machines with the rbfkemel. Neural Computation, 2003, 7(1):70-78
[107] Keerthi S S. Efficient tuning of SVM hyperparameters using radius/margin bound and iterative algorithm.IEEE Transactions on Neural Networks, 2002, 13(5):1225-1229
[108] Chapelle O, Vapnik V N, Bousquet O, Mukherjee S. Choosing multiple parameters for support vector machines. Machine Learning, 2002, 46(1): 131-159
[109] 吴涛．核函数的性质、方法及其在障碍检测中的应用．国防科学技术大学博士学位论文，2003
[110] Scholkopf B, Mika S, Burges C, et al. Input space vs feature space in kernel-based methods. IEEE Transactions on Neural Networks. 1999, 10(5):1000-1017
[111] 田盛丰，黄厚宽．回归型支持向量机的简化算法．软件学报，2002，13(6)：1169-1172
[112] Vapnik V N，Chapelle O．Bounds 0n error expectation for support vector machines．Neural Computation，2000，12(9)：2013-2036
[113] 田捷，杨鑫．生物特征识别技术理论与应用．北京：电子工业出版社，2005
[114] 张翠平，苏光大．人脸识别技术综述．中国图象图形学报，2000，5(11)：885-894
[115] Hong Z Q. Algebraic feature extraction of image for recognition. Pattern Recognition, 1991,24(3):211-219
[116] Samaria F, Young S. HMM based architecture for face identification. Image and Computer Vision, 1994.12(4):537-543
[117] Eremad K, Chellappa R. Discriminant analysis for recognition of human face images. J. of Optical Society of American, 1997, 8(14): 1724-1733
[118] Lades M, Vorbruggen J C, Buhmann J, et al. Distortion invariant object recognition in the dynamic link architecture. IEEE Trans.on Computers, 1993, 42:300-311
[119] Penev P S, Atick J J. Local feature analysis: A general statistical theory for object representation. Network: Computation in Neural Systems, 1996,7(3):477-500
[120] Brunelli R, Poggio T. Face recognition: features versus templates. IEEE Transactions on PAMI, 1993,15(10): 1042-1052
[121] 游素亚，张永越，李武军，徐光佑．一种基于多视点图像的可变姿态人脸识别系统．中国人民警官大学学报(自然科学版)，1997(1)
[122] Song G, Ai H, Xu G, Zhuang L. Automatic Video Based Face Verification and. Recognition by Support Vector Machines, Proceedings of SPIE-The Intemational. Society for Optical Engineering, v5286, n1, 2003:127-132
[123] 梁路宏，艾海舟．人脸检测研究综述．计算机学报，2002，25(2)：449-458
[124] 周志华，皇甫杰，张宏江，陈祖翰．基于神经网络集成的多视角人脸识别．计算机研究与发展，2001，38(10)：1204-12lO
[125] 张军平．流形学习及应用．中科院自动化所博士学位论文，2003
[126] 王蕴红，谭铁牛，朱勇．基于奇异值分解和数据融合的脸像鉴别．计算机学报，2000，20(3)：649-653
[127] Y.Tian, T.Tan and Y.Wang, Do Singular Values Contains Adequate Information for Face Recognition? Pattern Recognition, 2003, 36(3):649-655
[128] S.Shan, B.Cao, W.Gao, D.Zhao. Extended Fisherface For Face Recognition From A Single Example Image Per Person. IEEE Intemational Symposium on Circuits and Systems, 2002(IEEE ISCAS2002), 2:81-84
[129] S.Shan, Y.Chang, W.Gao, B.Cao. Curse Of Mis-Alignment In Face Recognition:Problem And A Novel Mis-Alignment Learning Solution, To appear in Proceeding of The 6th International Conference on Face and Gesture Recognition, Korea, May, 2004
[130] 刘青山．人脸跟踪与识别的研究．中科院自动化所博士学位论文，2003
[131] 刑昕，汪孔桥，沈兰荪．基于器官跟踪的人脸实时跟踪方法．电子学报，2000，28(6)：29-31
[132] 山世光．人脸识别中若干关键问题的研究．中国科学院研究生院博士学位论文，2004
[133] Osuna E, Freund R, Girosi E Training Support Vector Machines, An Application to Face Detection. In: Proc. of CVPR'97, Puerto Rico, 1997
[134] 张敏贵，潘泉，张洪才等．基于支持向量机的人脸分类．计算机工程，2004，30(11)：110-112
[135] 王宏漫，欧宗瑛．基于支持向量机的人脸识别方法研究．小型微型计算机系统，2004，25(1)：139-142
[136] Ming-Hsuan Yang, David J. Kriegrnan, Narendra Ahuja. Detecting Faces in Images: A Survey. IEEE PAMI, 2002, 24(1):34-58
[137] Rosenfeld R, Kak A C. Digital Picture Processing. New York: Academic Press,1976
[138] Anil K. Jain. Fundamentals of Digital Image Processing. Prentice-Hall Inc., 1989
[139] 靳蕃．神经计算智能基础：原理·方法．成都：西南交通大学出版社，2000
[140] Peter N Belhumeur, David J Kriengman, David J Kriengman. Eigenfaces vs. Fisherfaces: Recognition using class specificlinear projection. IEEE Trans. Pattern Anal Machine Intell, 1997, 19(7):711-720
[141] Sarle W S.Periodic Posting to the Usenet Newsgroup Comp.Ai Neural-nets. Neural Network FAQ. 1997
[142] Bottou L, Cortes C, Denker J, et al. Comparison of Classifier Methods: A Case Study in Handwriting Digit Recognition. Proc. of Int. Conf. Pattern Recognition, 1994:77-87
[143] Krebel U. Pairwise Classification and Support Vector Machines. Scholkopf B, Burges C J C, Smola A J. Advances in Kemel Methods: Support Vector Learning. MA: MIT Press, 1999:255-268
[144] Weston J, Watkins C. Multi-class Support Vector Machines. Brussels: Proceedings of ESANN'99, 1999:233-265
[145] Platt J C, Cristianini N, Shawe-Taylor J. Large margin DAGs for multiclass classification. Advances in Neural Information Processing Systems, 2000, 12(12):547-553
[146] Fumitake Takahashi, Shigeo Abe. Decision-Tree-based Multiclass Support Vector Machines. http://frenchblue.scitec.kobeu.ac.jp/～abe/pdf/iconip02takahashi. pdf, 2002
[147] Rifkin R, Klautau A. In defense of One-Vs-All classification. Journal of Machine Learning Rearch, 2004, 5:101-141
[148] Platt J C. Probabilities for Support Vector Machines. Advances in Large Margin Classifiers. Massachusetts: MIT Press, 2000:61-74
[149] ORL Face Database. http://www.uk.research.att.com/facedatabase.html,2004
[150] YALE. The Yale face database, http://cvc.yale.edu/projects/yalefaces/yale faces, html, 2005
[151] Thomas M. Cover, Peter E. Hart. Nearest neighbor pattern classification. IEEE Transactions on Information Theory, 1967, IT-13(1):21-27
[152] Specht D F. PNN and polynomial adlineas complementary techniques for classification. Transction on Neural Network, 1990, 1(1): 111-121
[153] 中国建设银行网站.http://www.ccb.corn/portal/cn/home/index.html
[154] Thomas L C, Edelman D B, Crook J N. Credit Scoring and Its Application.Society for Industrial and Applied Mathematics, Philadelphia, 2002
[155] 朱兴德，冯铁军．基于GA神经网络的个人信用评估．系统工程理论与实践，2003，23(12)：70-75，115
[156] 吴德胜，梁樑．遗传算法优化神经网络及信用评价研究．中国管理科学，2004，12(1)：68-74
[157] 杨力，童艳梅，阮守武等．V-fold交叉验证和BP神经网络在信用评价 中的应用．运筹与管理，2005，14(4)：140-143
[158] 王宪全，陈李刚．基于遗传算法和BP神经网络的信用风险测量模型．哈尔滨工业大学学报(社会科学版)，2006，8(5)：87-92
[159] 杨毓，蒙肖莲．用支持向量机(SVM)构建企业破产预测模型．金融研究，2006．10：65-75
[160] 肖智，王明恺，谢林林．基于支持向量机的大学生助学贷款个人信用评价．清华大学学报(自然科学版)，2006，46(S1)：1120-1124
[161] Barzilay O, Brailovsky D L. On domain knowledge and feature selection using a support vector machine. Pattem Recognition Letters, 1999, 20:475-484
[162] Brown M P, Grundy W N, Lin D, et al. Knowledge-based analysis of microarray gene expression data by using support vector machines. Proceedings of the National Academy of Sciences, 2000, 97:262-267
[163] Furey T S, Cristianini N, Duffy N, et al. Support vector machine classification and validation of cancer tissue samples using microarray expression data. Bioinformatics, 2000, 16(10):906-914
[164] Chiang Leo H, Kotanchek Mark E, Kordon Arthur K. Fault Diagnosis Based on Fisher Discriminant Analysis and Support Vector Machines. Computers & Chemical Engineering, 2004, 28:1389-1401
[165] Fortuna J, Capson D. Improved support vector classification using PCA and ICA feature space modification.Pattem Recognition, 2004, 37(6):1117-1129
[166] Shin K S, Lee T S, Kim H J. An application of support vector machines in bankruptcy prediction model, Expert Systems with Applications, 2005, 28:127-135
[167] Tay F E, Cao H. Application of support vector machines in financial time series forecasting. Omega, 2001, 29(1):309-317
[168] Engelbrecht A P. A New Pruning Heuristic Based on Variance Analysis of Sensitivity Infomation. IEEE Trans. on Neural Networks, 2001, 12(6): 1386-1399
[169] Kwak N, Choi C. Input Feature Selection for Classification Problem. IEEE Trans. on Neural Networks, 2002, 13(1):143-159
[170] Choubey S K, Deogun J S. A Comparison of Feature Selection Algorithm in the Context of Rough classifiers. 5th IEEE Int Conf Fuzzy Systems. New Orleans: Prentice Hall, 1995:1122-1128
[171] 韩祯祥，张琦．粗糙集理论及其应用综述．控制理论与应用，1999，16(2)：153-157
[172] Holland J H. Adaptation in Nature and Artificial Systems. The University of Michigan Press, 1975
[173] Goldberg D E. Genetic Algorithms in Search, Optimization and Machine Learning. Addison-wisley, Reading, MA, 1989