支持向量机算法及其应用研究
|
摘要
基于统计学习理论的支持向量机算法具有坚实的数学理论基础和严格的理论分析,具有理论完备、全局优化、适应性强、推广能力好等优点,是机器学习中的一种新方法和研究新热点。它使用结构风险最小化原则,综合了统计学习、机器学习和神经网络等方面技术,在最小化经验风险的同时,有效地提高了算法泛化能力。它与传统的机器学习方法相比,具有良好的潜在应用价值和发展前景。
本文分析和总结了现有的几种典型支持向量机算法,提出了基于组合式多类别分类器思想的PCA支持向量机算法、加权PCA支持向量机算法、借鉴核函数方法的小波支持向量机算法、RS-SVM动态预测方法、模糊二叉树支持向量机等算法,对其算法性能和应用作了深入研究。主要工作包括:
1.系统地研究了支持向量机的求解方法。主要有支持向量机的二次规划求解法、选块法、分解法、序列最小优化方法、基于Lagrange函数的迭代求解方法即Lagrange支持向量机、基于Smoothing处理的牛顿求解方法。这些方法是通过求解凸二次规划问题或将大规模问题转化成若干子问题再求解凸二次规划问题,或者是转化为无约束最优化问题再利用比较成熟的最优化方法求解。通过对它们的分析,为提出新的支持向量机算法提供了理论基础。
2.研究了基于L_p范数分类间隔的三种支持向量机。重点研究了L_1范数支持向量机在线性和非线性两种情形下的算法理论和实现,分析了采用L_p范数度量分类间隔的L_p范数支持向量机最优化问题表示方法。在高维特征空间中,L_1范数支持向量机表现出了较好的特征压缩效果,而且可以节省测试计算时间。
3.研究了PCA支持向量机及扩展算法。提出了PCA支持向量机组合式分类方法,解决了传统支持向量机不能进行特征变换的预处理问题。提出了加权PCA支持向量机算法,较好地解决了样本数目的不平衡对分类性能所带来的影响。其次,借鉴核函数思想,构造了一种核PCA(Kernel PCA)方法并用于特征变换,通过与支持向量机组合成为一种新的具有特征变换功能的Kernel PCA支持向量机分类算法。三种新的组合式支持向量机算法均可用于需消除噪声情形的模式识别问题。
4.在研究支持向量核函数条件的基础上,构造了一种基于小波核函数的小波支持向量机。分析了算法的收敛性、通用性和泛化能力。该算法扩充较为容易,实验结果表明小波支持向量机算法具有比较理想的函数逼近能力。
5.研究了基于支持向量机的系统辨识理论和方法。提出了一种适用于回转窑烧结温度检测的RS-SVM动态预测新方法,取得了较好的预测效果。对最小二乘
Support vector machine (SVM) based on the Statistical Learning Theory is a new approach and research field in machine learning because of its advantage such as firm mathematic theory foundation, strict theory analysis, complete theory, global optimization as well as good adaptability and generalization. SVM improves the algorithm generalization effectively and minimizes the empirical risk simultaneously by using Structural Risk Minimization and synthesizing the techniques including the statistical learning, machine learning and neural networks, etc. It also has good latent application values and development prospects compared with the conventional machine learning methods.
In this paper, several typical support vector machine algorithms are generalized. Five novel algorithms are proposed. They are PCA support vector machine algorithm which is based on the idea of combination multi-class classification, weighted PCA support vector machine algorithm, wavelet support vector machine borrowed idea from the kernel function, RS-SVM dynamic prediction and fuzzy binary tree support vector machine. The performance and applications of the algorithms are studied in depth. The research is carried out in the following aspects:
1. The solution methods of support vector machine, including quadratic programming method, chunking method, decomposing method, sequential minimization optimization method, iterative solution method named Lagrange support vector machine based on Lagrange function and Newton method based on the smoothing technique, are studied systematically. The methods employs solving convex quadratic programming directly or solving convex quadratic programming after converting the large-scale problem into many sub-problem or utilizing sophisticated optimization techniques after converting the constrained optimization problem into unconstrained ones. The theory foundation for presenting new support vector machine algorithms is laid by means of analyzing those methods.
2. Three support vector machines based on L_p-norm classification margin are studied. The algorithms theory and realization of the L_1-norm SVM under linear and nonlinear conditions are emphatically studied. The optimization problem of L_1-norm SVM which adopts the L_p-norm to measure the classification margin is analyzed. In high dimensions feature space, L_1-norm SVM shows better feature compression
本文分析和总结了现有的几种典型支持向量机算法,提出了基于组合式多类别分类器思想的PCA支持向量机算法、加权PCA支持向量机算法、借鉴核函数方法的小波支持向量机算法、RS-SVM动态预测方法、模糊二叉树支持向量机等算法,对其算法性能和应用作了深入研究。主要工作包括:
1.系统地研究了支持向量机的求解方法。主要有支持向量机的二次规划求解法、选块法、分解法、序列最小优化方法、基于Lagrange函数的迭代求解方法即Lagrange支持向量机、基于Smoothing处理的牛顿求解方法。这些方法是通过求解凸二次规划问题或将大规模问题转化成若干子问题再求解凸二次规划问题,或者是转化为无约束最优化问题再利用比较成熟的最优化方法求解。通过对它们的分析,为提出新的支持向量机算法提供了理论基础。
2.研究了基于L_p范数分类间隔的三种支持向量机。重点研究了L_1范数支持向量机在线性和非线性两种情形下的算法理论和实现,分析了采用L_p范数度量分类间隔的L_p范数支持向量机最优化问题表示方法。在高维特征空间中,L_1范数支持向量机表现出了较好的特征压缩效果,而且可以节省测试计算时间。
3.研究了PCA支持向量机及扩展算法。提出了PCA支持向量机组合式分类方法,解决了传统支持向量机不能进行特征变换的预处理问题。提出了加权PCA支持向量机算法,较好地解决了样本数目的不平衡对分类性能所带来的影响。其次,借鉴核函数思想,构造了一种核PCA(Kernel PCA)方法并用于特征变换,通过与支持向量机组合成为一种新的具有特征变换功能的Kernel PCA支持向量机分类算法。三种新的组合式支持向量机算法均可用于需消除噪声情形的模式识别问题。
4.在研究支持向量核函数条件的基础上,构造了一种基于小波核函数的小波支持向量机。分析了算法的收敛性、通用性和泛化能力。该算法扩充较为容易,实验结果表明小波支持向量机算法具有比较理想的函数逼近能力。
5.研究了基于支持向量机的系统辨识理论和方法。提出了一种适用于回转窑烧结温度检测的RS-SVM动态预测新方法,取得了较好的预测效果。对最小二乘
Support vector machine (SVM) based on the Statistical Learning Theory is a new approach and research field in machine learning because of its advantage such as firm mathematic theory foundation, strict theory analysis, complete theory, global optimization as well as good adaptability and generalization. SVM improves the algorithm generalization effectively and minimizes the empirical risk simultaneously by using Structural Risk Minimization and synthesizing the techniques including the statistical learning, machine learning and neural networks, etc. It also has good latent application values and development prospects compared with the conventional machine learning methods.
In this paper, several typical support vector machine algorithms are generalized. Five novel algorithms are proposed. They are PCA support vector machine algorithm which is based on the idea of combination multi-class classification, weighted PCA support vector machine algorithm, wavelet support vector machine borrowed idea from the kernel function, RS-SVM dynamic prediction and fuzzy binary tree support vector machine. The performance and applications of the algorithms are studied in depth. The research is carried out in the following aspects:
1. The solution methods of support vector machine, including quadratic programming method, chunking method, decomposing method, sequential minimization optimization method, iterative solution method named Lagrange support vector machine based on Lagrange function and Newton method based on the smoothing technique, are studied systematically. The methods employs solving convex quadratic programming directly or solving convex quadratic programming after converting the large-scale problem into many sub-problem or utilizing sophisticated optimization techniques after converting the constrained optimization problem into unconstrained ones. The theory foundation for presenting new support vector machine algorithms is laid by means of analyzing those methods.
2. Three support vector machines based on L_p-norm classification margin are studied. The algorithms theory and realization of the L_1-norm SVM under linear and nonlinear conditions are emphatically studied. The optimization problem of L_1-norm SVM which adopts the L_p-norm to measure the classification margin is analyzed. In high dimensions feature space, L_1-norm SVM shows better feature compression
引文
[1] Vapnik V. The nature of statistical learning theory. New York: Springer-Verlag, 1999
[2] 张学工译.统计学习理论的本质.北京:清华大学出版社, 2000
[3] Rosenblatt F. The perceptron: a probabilistic model for information storage and organization in the brain. Phychological Review, 1958, 65:386-408
[4] Rabiner L .R . A tutorial on hidden Markov models and selected applications in speech recognition. Proceeding of the IEEE, 1989, 77(2):257-286
[5] Naftali Z. T. ishby. On the application of mixture AR hidden Markov models to text independent speaker recognition. IEEE Trans on Signal Processing, 1991, 39(3):563-570
[6] Vapnik V. The nature of statistical learning theory. New York: Springer-Verlag, 1995
[7] 史忠值.知识发现.北京:清华大学出版社,2002,203-207
[8] 邓乃扬,田英杰.数据挖掘中的新方法-支持向量机.北京:科学出版社,2004
[9] 张学工.关于统计学习理论与支持向量机.自动化学报,2000,26(1):32-42
[10] Burges C. A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery,1998,2(2):121-127
[11] C. Cortes, V. Vapnik. Support-vector networks. Machine Learning, 1995, 20 (3): 273-297
[12] Amari S,Wu S.Improving support vector machine classifier by modifying kernel functions.Neural Networks,1999,12(9):783-789
[13] Baesens B.,Viaene S. Van Gestel T. et al. An empirical assessment of kernel type performance for least squares support vector machine classification. In: Proceedings of Fourth International Conference on Knowledge-Based Intelligent Engineer System and Allied Technologies.Brighton,UK, 2000, 313-316
[14] Vapnik V, Chapelle O. Bounds on error expectation for support vector machines. Neural Computation, 2000,12(9):2013-2036
[15] O. Chapelle, V. Vapnik. Choosing multiple parameters for support vector machines. Machine Learning, 2002, 46(1):131-159
[16] Wahba G. Supprot vector machine, reproducing kernel Hibert space and the randomized GACV. Advance in Kernel Methods-supprot vector machine,Cambridge, MA, MIT Press, 1999
[17] Wahba G, Lin Y., Zhang H. GACV for support vector machine. Advance in Larger Margin Classifiers, MIT Press, 1999
[18] 薛毅.最优化原理与方法.北京:北京工业大学出版社,2001
[19] 施光燕, 董加理.最优化方法.北京:高等教育出版社,2002
[20] 飞思科技产品研发中心.Matlab6.5 辅助优化计算与设计.北京:电子工业出版社,2003
[21] Smola A J,Sch?lkopf B, Müller K-R. The connection between regularization operators and support vector kenerl. Neural Networks, 1998,11(4):637-649
[22] Osuna E, Freund R, Girosi F. An improved training algorithm for support vector machine. In: Proceedings of IEEE Workshop on Neural Networks for Signal Processing. Amelea Island, 1997,276-285
[23] Osuna E, Freund R, Girosi F.Training support vector machines:an application to face detection. In: Proceedings of CVPR’97. New York, 1997,130-136
[24] Lin C-J. On the convergence of the decomposition method for support vector machines. IEEE Trans on Neural Networks, 2001, 12(6):1288-1298
[25] Joachims T. Transductive inference for text classification using support vector machine. In: Proceedings of the Sixteenth International Conference on Machine Learning. Morgan Kaufmann, 1999, 148-156
[26] Hsu Chih-Wei,Lin Chih-Jen. A comparison of methods for multi-class support vector machines. IEEE Trans on Neural Networks, 2002, 13(2):415-425
[27] Laskov P. Feasible direction decomposition algorithms for training support vector machines. Machine Learning, 2002, 46(1):315-349
[28] Platt J C. Fast training of support vector machines using sequential minimal optimization. Advance in Kernel method-support vector learning. Cambridge, MA, MIT Press, 1999, 185-208
[29] Platt J C. Probabilistic outputs for support vector machines and comparison to regularized likelihood methods. Advance in Larger Margin Classifiers, MIT Press, 1999
[30] Keerthi S, Gilbert E G. Convergence of a generalized SMO algorithm for SVM classifier design. Machine Learning, 2002, 46(1):351-360
[31] Alistair Shilton, M. Palaniswami. Incremental Training of Support Vector Machines.IEEE Trans on Neural networks,2005, 16(1):114-131
[32] 萧嵘,王继成,孙正兴等. 一种 SVM 增量学习算法. 南京大学学报, 2002, 38(2):152-157
[33] 萧嵘,王继成,孙正兴等. 一种 SVM 增量学习算法 α-ISVM.软件学报, 2001, 12(12):1818-1824
[34] 曾文华,马健. 一种新的支持向量机增量学习算法. 厦门大学学报, 2002, 41(6):687-691
[35] Ahmed S N. Incremental learning with support vector machines. In Proceedings of Workshop on Support Vector Machines. Stockholm, Sweden, 1999, 208-307
[36] Cauwenberghs G,Poggio T. Incremental and decremental support vector machine learning. http://citeseer.nj.nec.com/cauwenberghs00incremental.html, 2004-8-12
[37] Carozza M, Rampone S. Towards an incremental SVM for regression. In Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks, Como, Italy,2000,6,405-410
[38] Xiao Rong, Wang Jicheng,Zhang Fayan. An approach to incremental SVM learning algorithm. In Proceedings of 12th IEEE International Conference on Tools with Artifical Intelligence, ancouver, BC, Canada, 2000, 268-273
[39] Suykens J A K, Vandewalle J, De Moor B. Optimal control by least squres support vector machine. Neural Network, 2001,14(2):23-35
[40] Zhang Xue-gong. Using class-center vectors to build support vector machines. Neural Networks for Signal Processing IX-Proc of the 1999 IEEE Workshop, Wisconsin,1999,33-37.
[41] Keerthi S S. Shavade S K. A fast interative nearest point algorithm for support vector machine classifier design. IEEE Trans on Neural Networks, 2000,11 (1): 124-136
[42] Yang M H, Ahuja N. A geometric approach to train support vector machines. In proceedings of CVPR 2000, Hilton Head Island, 2000, 430-437
[43] De Bastos, De campos. A fast training algorith for unbiased proximal SVM. In Proceedings International Conference on Acoustics Speech and Signal Processing, Philadelphia, 2005, 245-248
[44] Lin Chun-Fu, Wang Sheng-De. Fuzzy support vector machines. IEEE Trans on Neural Networks, 2002, 13(2):464-471
[45] Lin Chih-Jen.On the convergence of the decomposition method for support vector machines. IEEE Trans on neual networks, 2001,12(6):1288-1298
[46] Kuan-Ming Lin, Lin Chih-Jen. A Study on Reduced Support Vector Machines. IEEE Trans on neual networks, 2003,14(6):1449-1459
[47] Suykens J A K, Vandewalle J. Least squares support vector machine classifiers. Neural Processing Letters, 1999, 9(3),293-300
[48] 宋杰 ,唐焕文 .基于线性规划的 v-支持向量机分类器 .大连理工大学学报 , 2005,45(2): 303-307
[49] 王玲,薄列峰,刘,芳等. 最小二乘隐空间支持向量机.计算机学报. 2005, 28 (8): 1301-1307
[50] 林继鹏,刘君华. 基于小波的支持向量机算法研究. 西安交通大学学报, 2005, 39(8):816-819
[51] Li Donghui,Du Shuxin,Wu Tiejun. A weighted support vector machine method and its application.In proceedings of the 5th World Congress on Intelligent Control and Automation, Hangzhou,P.R.China, 2004,1834-1837
[52] Krebel U H.Pairwise classification and support vector machines.Advances in Kernel Methods: Support Vector Learning, Cambridge, MA:MIT Press, 1999, 15:255-268
[53] Platt J, Cristianini N, Shawe-Taylor J.Large margin DAGs for multiclass classification. Advances in Neural Information Processing System,MIT Press, 2000,547-553
[54] Sebald D J,Bucklew J A. Support vector machines and the multiple hypothesis test problem. IEEE Trans on Signal Processing,2001,49(11):344-349
[54] 萧嵘,孙晨,王继成等.一种具有容噪性能的SVM多值分类器.计算机研究与发展,2000 37(9): 1071-1075
[55] 张国云,章兢.一种新的分裂层次聚类SVM多值分类器.控制与决策,2005,20 (8):931-934
[56] 李蓉,叶世伟,史忠植.SVM-KNN分类器-一种提高SVM分类精度的新方法.电子学报, 2002,30(5): 745-748
[57] 李红莲,王春花,袁保宗.一种改进的支持向量机NN-SVM.计算机学报, 2003, 26(8): 1015-1020
[58] 马笑天,黄席樾.基于SVM的二叉树多类分类算法及其在故障诊断中的应用.控制与决策, 2003, 18(3):272-284
[59] Arenas-Garcia J, Perez-Cruz F. Multi-class support vector machines:a new approach. In Proceedings of IEEE International Conference on Acoustics Speech and Signal Processing, New York, 2003,2,781-784
[60] Oren M.,Papageorgiou C., Sinha P. et al. Pedestrain detection using wavelet templates. In Proceedings of CVPR’97,Puerto Rico,1997,193-199
[61] Osuna E.,Freund R., Girosi F.Trainsing support vector machines:an application to face detection. In Proceedings of CVPR’97,Puerto Rico,1997,130-136
[62] 冯元戬,施鹏飞. 基于支持向量机的彩色图像人脸检测方法.上海交通大学学报,2003,37(6):947-951
[63] 燕忠,袁春伟.基于蚁群智能和支持向量机的人脸性别分类方法.电路与系统学报,2004,26(8):1177-1182
[64] 张敏贵,潘泉,张洪才等. 基于支持向量机的人脸分类.计算机工程,2004, 38 (11):110-112
[65] 宋晓宁,束鑫. 一种改进的模糊支持向量机的人脸识别方法.微机发展,2005, 15(3):23-27
[66] 马勇,丁晓青. 基于层次型支持向量机的人脸检测.清华大学学报,2003, 43 (1):35-38
[67] Shuotao Li, Yaonan Wang. Face Recognition using Wavelet Transform and Support Vector Machines. Pattern Recognition and Image Analysis, 2004, 14(3): 471-478
[68] Reyna R. A., Hernandez N., Esteve D. et al. Segmenting images with support vector machines. In Proceedings of 2000 International Conference on Image Processing, Vancover, BC, Canada ,2000,1(1):820-823
[69] Valentini A., Zhang Hongjiang. Automatic image orientation detection. IEEE Trans on Image Processing, 2002,11(7):746-755
[70] Guo Guo-Dong,Jain A. K., Ma Wei-Ying et al. Learning similarity measeure for natural image retrieval with relevance feedback. IEEE Trans on Neural Networks, 2002, 13(4):811-820
[71] Brown M., Lewis H. G.,Gunn S. R..Linear spectral mixture models and support vector machines for remote sensing. IEEE Trans on Geoscience and Remote Sensing, 2000, 38(5):2346-2360
[72] Shutao Li, James T. Yaonan Wang. Texture classication using the support vector machines. Pattern Recognition ,2003,36: 2883-2893
[73] Yongmin Li, Shaogang Gong, Jamie Sherrah,Heather Liddell. Support vector machine based multi-view face detection and recognition. Image and Vision Computing, 2004, 22: 413–427
[74] Jian-xiong Dong, Adam Krzy_zak, Ching Y. Suen . An improved handwritten Chinese character recognition system using support vector machine. Pattern Recognition Letters, 2005, 26 :1849-1856
[75] Valentini and Giorgio.Gene expression data analysis of human lymphoma using support vector machines and output coding ensembles.Artificial Intelligence in Medicine, 2002,26(3):281-304
[76] Cai Yu-dong, Liu Xiao-jun,Xu Xue-biao, et al. Prediction of protein structuralclasses by support vector machines. Computers and Chemistry, 2002, 26(3): 293- 296
[77] Bao L., Sun Z.. Identifying genes related to drug anticancer mechanisms using support vector machine. FFBS letters, 2002, 521(1):109-114
[78] Liu Weiqiang, Shen Peihua, Qu Yingge et al. Fast algorithm of support vector machines in lung cancer diagnosis. In proceedings of International Workshop on Medical Imaging and Augmented Reality, Hong Kong, 2001,188-192
[79] Bhanu P. K. N., Ramakrishnan A. G., Suresh S. et al. Fetal lung maturity analysis using ultrasound image features. IEEE Trans on Information Technology in Biomedicine, 2002, 6(1):38-45
[80] Ruey-Feng Chang, Wen-Jie Wu, Woo Kyung Moon et al. Support Vector Machines for Diagnosis of Breast Tumors on US Images.Academic Radiology, 2003, 10(2):189-197
[81] Jongcheol Kim,Sangchul Won. New fuzzy inference system using a support vector machine.In Proceedings of the 41st IEEE Conference on Decision and Control, Las Vegas, Nevada USA, 2002,1349-1354
[82] Tony Van G. Johan A. K. Suykens. Dirk-Emma Baestaens, et al. Financial time series prediction using least squares support vector machines within the evidence framework. IEEE Trans on Neural Networks, 2001,12(4)809-821
[83] Abhijit Kulkarni,V. K. Jayaraman, B. D. Kulkarni. Control of chaotic dynamical systems using support vector machines.Physics Letters A, 2003, 317:429-435
[84] M. A. Mohandes, T.O. Halawani, S. Rehman, et al. Support vector machines for wind speed prediction. Renewable Energy, 2004,29:939-947
[85] Johan A. K. Suykens. Nolinear Modelling and support vector machines. In Proceedings of International Conference on IEEE Instrumentation and Measurement Technology, Budapest, Hungary, 2001,287-294
[86] Sun Zonghai, Sun Youxian.Optimal control by weighted least squares generalized support vector machines. In Proceedings of the American Control Conference, Denver, Colorado, 2003, 5323-5328
[87] Wei Huang,Yoshiteru Nakamori,Shou-Yang Wang. Forecasting stock market movement direction withsupport vector machine. Computers and Operations Research, 2005,32 : 2513-2522
[88] Kyung-Shik Shin,Taik Soo Lee,Hyun-jung Kim. An application of support vector machines in bankruptcy prediction model. Expert Systems with Applications, 2005, 28: 127–135
[89] Ping-Feng Pai,Chih-Sheng Lin. AhybridARIMAand support vector machines model in stock price forecasting.The International Journal of Management Science, 2005,33: 497-505
[90] 马笑箫,黄席樾,柴毅.基于支持向量机的故障过程趋势预测研究.系统仿真学报,2002,14(11):1548-1551
[91] 王定成,方廷健,高理富等.支持向量机回归在线建模及应用.控制与决策,2003, 18(1):89-95
[92] 赵登福,王蒙,张讲社等.基于支撑向量机的短期负荷预测.中国电机工程学报, 2002, 22(4):26-30
[93] 陈友,张国基,郭国雄.一种改进的SVM算法及其在证券领域中的应用.华南理工大学学报,2003, 31(7):15-18
[94] 李丽娜,候朝桢.基于支持向量机(SVM)的工业过程辨识.北京理工大学学报, 2003,23(5):569-580
[95] 翟永杰,毛继珮.分级聚类支持向量机在汽轮机故障诊断中的应用.华北电力大学学报,2003,30(6):25-29
[96] 张国云,章兢.基于模糊支持向量机的多级二叉树分类器的水轮机调速系统故障诊断.中国电机工程学报,2005,25(8):100-104
[97] 张国云,章兢.基于粗集理论和支持向量机的动态预测新方法及应用.自动化仪表, 2005, 26(2):17-20
[98] Zhang Guoyun,Zhang Jing.A new tree-like fuzzy binary support vector machine for optical character recognition. In Proceedings of SPIE, Beijing,2004, 5637, 212-217
[99] 王宇红,黄德先.基于支持向量机的非线性预测控制技术.信息与控制,2004, 33(2):134-140
[100] 王宇红,黄德先,高东杰等.基于LS-SVM地非线性预测控制技术.控制与决策, 2004,19(4):383-387
[101] 张浩然,韩正之,李昌刚.基于支持向量机的未知非线性系统辨识与控制.上海交通大学学报,2003,37(6):927-930
[102] 高隽.人工神经网络原理及仿真实例.北京:机械工业出版社,2003,76-93
[103] 李国正,王猛,曾华军.支持向量机导论.北京:电子工业出版社,2004
[104] Chunhui Chen,O.L. Mangasarian.Smoothing methods for convex inequalities and linear complementarity problems. Mathematical Programming, 1995, 71:51-69
[105] Glenn Fung.O.L. Mangasarian. Finite Newton method for Lagrangian support vector machine classi!cation. Neurocomputing ,2003,55:39-55
[106] Yuh-Jye Lee.Support vector machines in data mining:[dissertation].Wisconsin: University of Wisconsin-Madison, 2001,19-31
[107] Yuh-Jye Lee,Wen-Feng Hsieh,Chien-Ming Huang .spl epsi/-SSVR: a smooth support vector machine for /spl epsi/-insensitive regression. IEEE Transactions on Knowledge and Data Engineering,2005,17(5)678-585
[108] Joao Pedro, Noboru Murata. Support vector machines with different norms: motivation, formulations and results.Pattern Recognition Letters, 2001, 22: 1263-1272
[109] O.L. Mangasarian. Arbitrary-norm separating plane. Operations Research Letters, 1999, 24:15-23
[110] 蔡春,周博,李杨.一范数支持向量机的特征提取算法及应用.统计与决策, 2004, 172(4):18-19
[111] Glenn M. Fung, O.L. Mangasarian.A feature selection Newton method for support vector machine classification. Computational Optimization and Applications, 2004, 28:185-202
[112] 边肇祺,张学工.模式识别.北京:清华大学出版社,2005,212-310
[113] 温熙森,胡莺庆,邱静.模式识别与状态监控.长沙:国防科技大学出版社, 1997, 180-185
[114] Scholkopf B, Smola A,Muller K R. Nonlinear component analysis as a kernel eigenvalue problems. Neural Computation, 1998, 10(5):299-139
[115] 黄国宏,邵惠鹤.核主元分析及其在人脸识别中的应用.计算机工程, 2004, 30(13):13-15
[116] 王华国,俞金寿.基于核函数主元分析的软测量建模方法及应用.华东理工大学学报, 2004, 30(5):567-570
[117] 刘贵忠,邸双亮. 小波分析及其应用.西安:西安电子科技大学出版社,1992
[118] 飞思科技产品研发中心.Matlab6.5辅助小波分析与应用.北京:电子工业出版社,2003
[119] 崔万照,朱长纯,保文纯等.最小二乘小波支持向量机在非线性系统辨识中的应用.西安交通大学学报, 2004, 38(6):562-586
[120] Yuancheng Li, Bo Li, Tiangjian Fang. Short-term load forecast based on fuzzy wavelete support vector machines. In proceedings of the 5th World Congress on Intelligent Control and Automation, Hangzhou, P.R. China, 2004,5194-5198
[121] 张文修,吴伟志,梁吉业,等.粗造集理论与方法.北京:科学出版社,2001,12-22
[122] 张家树, 肖先赐. 混沌时间序列的Volterra自适应预测.物理学报, 2000, 49 (03): 403-407.
[123] 张家树, 肖先赐. 混沌时间序列的自适应高阶非线性滤波预测. 物理学报, 2000, 49(07): 1221-1227.
[124] 张家树, 肖先赐. 用于混沌时间序列自适应预测的一种少参数二阶Volterra滤波器. 物理学报, 2001, 50(07): 1248-1254.
[125] 谭文, 王耀南. 混沌时间序列的模糊神经网络预测. 物理学报, 2003, 52(04): 797-801.
[126] 陈梁.我国水电建设和科技进步50年.中国电力,1999,32(10):21-24
[127] 王海. 水轮发电机组状态监测、诊断及综合试验分析系统研究:[华中科技大学博士学位论文]. 武汉:华中科技大学水电与数字化工程学院,2001, 8-14
[128] 余晓晖,杜林,陈明英等.基于BP神经网络的水轮机调速系统故障诊断.重庆大学学报, 2001, 24(11):71-74
[129] 王道平,张义忠. 故障智能诊断系统的理论与方法.北京:冶金工业山版社, 2001
[130] N. R. Pal, J. C. Bezdek.On cluster validity for the fuzzy c-means model.IEEE Transaction on Fuzzy Systems,1995,3(3):370-379
[2] 张学工译.统计学习理论的本质.北京:清华大学出版社, 2000
[3] Rosenblatt F. The perceptron: a probabilistic model for information storage and organization in the brain. Phychological Review, 1958, 65:386-408
[4] Rabiner L .R . A tutorial on hidden Markov models and selected applications in speech recognition. Proceeding of the IEEE, 1989, 77(2):257-286
[5] Naftali Z. T. ishby. On the application of mixture AR hidden Markov models to text independent speaker recognition. IEEE Trans on Signal Processing, 1991, 39(3):563-570
[6] Vapnik V. The nature of statistical learning theory. New York: Springer-Verlag, 1995
[7] 史忠值.知识发现.北京:清华大学出版社,2002,203-207
[8] 邓乃扬,田英杰.数据挖掘中的新方法-支持向量机.北京:科学出版社,2004
[9] 张学工.关于统计学习理论与支持向量机.自动化学报,2000,26(1):32-42
[10] Burges C. A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery,1998,2(2):121-127
[11] C. Cortes, V. Vapnik. Support-vector networks. Machine Learning, 1995, 20 (3): 273-297
[12] Amari S,Wu S.Improving support vector machine classifier by modifying kernel functions.Neural Networks,1999,12(9):783-789
[13] Baesens B.,Viaene S. Van Gestel T. et al. An empirical assessment of kernel type performance for least squares support vector machine classification. In: Proceedings of Fourth International Conference on Knowledge-Based Intelligent Engineer System and Allied Technologies.Brighton,UK, 2000, 313-316
[14] Vapnik V, Chapelle O. Bounds on error expectation for support vector machines. Neural Computation, 2000,12(9):2013-2036
[15] O. Chapelle, V. Vapnik. Choosing multiple parameters for support vector machines. Machine Learning, 2002, 46(1):131-159
[16] Wahba G. Supprot vector machine, reproducing kernel Hibert space and the randomized GACV. Advance in Kernel Methods-supprot vector machine,Cambridge, MA, MIT Press, 1999
[17] Wahba G, Lin Y., Zhang H. GACV for support vector machine. Advance in Larger Margin Classifiers, MIT Press, 1999
[18] 薛毅.最优化原理与方法.北京:北京工业大学出版社,2001
[19] 施光燕, 董加理.最优化方法.北京:高等教育出版社,2002
[20] 飞思科技产品研发中心.Matlab6.5 辅助优化计算与设计.北京:电子工业出版社,2003
[21] Smola A J,Sch?lkopf B, Müller K-R. The connection between regularization operators and support vector kenerl. Neural Networks, 1998,11(4):637-649
[22] Osuna E, Freund R, Girosi F. An improved training algorithm for support vector machine. In: Proceedings of IEEE Workshop on Neural Networks for Signal Processing. Amelea Island, 1997,276-285
[23] Osuna E, Freund R, Girosi F.Training support vector machines:an application to face detection. In: Proceedings of CVPR’97. New York, 1997,130-136
[24] Lin C-J. On the convergence of the decomposition method for support vector machines. IEEE Trans on Neural Networks, 2001, 12(6):1288-1298
[25] Joachims T. Transductive inference for text classification using support vector machine. In: Proceedings of the Sixteenth International Conference on Machine Learning. Morgan Kaufmann, 1999, 148-156
[26] Hsu Chih-Wei,Lin Chih-Jen. A comparison of methods for multi-class support vector machines. IEEE Trans on Neural Networks, 2002, 13(2):415-425
[27] Laskov P. Feasible direction decomposition algorithms for training support vector machines. Machine Learning, 2002, 46(1):315-349
[28] Platt J C. Fast training of support vector machines using sequential minimal optimization. Advance in Kernel method-support vector learning. Cambridge, MA, MIT Press, 1999, 185-208
[29] Platt J C. Probabilistic outputs for support vector machines and comparison to regularized likelihood methods. Advance in Larger Margin Classifiers, MIT Press, 1999
[30] Keerthi S, Gilbert E G. Convergence of a generalized SMO algorithm for SVM classifier design. Machine Learning, 2002, 46(1):351-360
[31] Alistair Shilton, M. Palaniswami. Incremental Training of Support Vector Machines.IEEE Trans on Neural networks,2005, 16(1):114-131
[32] 萧嵘,王继成,孙正兴等. 一种 SVM 增量学习算法. 南京大学学报, 2002, 38(2):152-157
[33] 萧嵘,王继成,孙正兴等. 一种 SVM 增量学习算法 α-ISVM.软件学报, 2001, 12(12):1818-1824
[34] 曾文华,马健. 一种新的支持向量机增量学习算法. 厦门大学学报, 2002, 41(6):687-691
[35] Ahmed S N. Incremental learning with support vector machines. In Proceedings of Workshop on Support Vector Machines. Stockholm, Sweden, 1999, 208-307
[36] Cauwenberghs G,Poggio T. Incremental and decremental support vector machine learning. http://citeseer.nj.nec.com/cauwenberghs00incremental.html, 2004-8-12
[37] Carozza M, Rampone S. Towards an incremental SVM for regression. In Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks, Como, Italy,2000,6,405-410
[38] Xiao Rong, Wang Jicheng,Zhang Fayan. An approach to incremental SVM learning algorithm. In Proceedings of 12th IEEE International Conference on Tools with Artifical Intelligence, ancouver, BC, Canada, 2000, 268-273
[39] Suykens J A K, Vandewalle J, De Moor B. Optimal control by least squres support vector machine. Neural Network, 2001,14(2):23-35
[40] Zhang Xue-gong. Using class-center vectors to build support vector machines. Neural Networks for Signal Processing IX-Proc of the 1999 IEEE Workshop, Wisconsin,1999,33-37.
[41] Keerthi S S. Shavade S K. A fast interative nearest point algorithm for support vector machine classifier design. IEEE Trans on Neural Networks, 2000,11 (1): 124-136
[42] Yang M H, Ahuja N. A geometric approach to train support vector machines. In proceedings of CVPR 2000, Hilton Head Island, 2000, 430-437
[43] De Bastos, De campos. A fast training algorith for unbiased proximal SVM. In Proceedings International Conference on Acoustics Speech and Signal Processing, Philadelphia, 2005, 245-248
[44] Lin Chun-Fu, Wang Sheng-De. Fuzzy support vector machines. IEEE Trans on Neural Networks, 2002, 13(2):464-471
[45] Lin Chih-Jen.On the convergence of the decomposition method for support vector machines. IEEE Trans on neual networks, 2001,12(6):1288-1298
[46] Kuan-Ming Lin, Lin Chih-Jen. A Study on Reduced Support Vector Machines. IEEE Trans on neual networks, 2003,14(6):1449-1459
[47] Suykens J A K, Vandewalle J. Least squares support vector machine classifiers. Neural Processing Letters, 1999, 9(3),293-300
[48] 宋杰 ,唐焕文 .基于线性规划的 v-支持向量机分类器 .大连理工大学学报 , 2005,45(2): 303-307
[49] 王玲,薄列峰,刘,芳等. 最小二乘隐空间支持向量机.计算机学报. 2005, 28 (8): 1301-1307
[50] 林继鹏,刘君华. 基于小波的支持向量机算法研究. 西安交通大学学报, 2005, 39(8):816-819
[51] Li Donghui,Du Shuxin,Wu Tiejun. A weighted support vector machine method and its application.In proceedings of the 5th World Congress on Intelligent Control and Automation, Hangzhou,P.R.China, 2004,1834-1837
[52] Krebel U H.Pairwise classification and support vector machines.Advances in Kernel Methods: Support Vector Learning, Cambridge, MA:MIT Press, 1999, 15:255-268
[53] Platt J, Cristianini N, Shawe-Taylor J.Large margin DAGs for multiclass classification. Advances in Neural Information Processing System,MIT Press, 2000,547-553
[54] Sebald D J,Bucklew J A. Support vector machines and the multiple hypothesis test problem. IEEE Trans on Signal Processing,2001,49(11):344-349
[54] 萧嵘,孙晨,王继成等.一种具有容噪性能的SVM多值分类器.计算机研究与发展,2000 37(9): 1071-1075
[55] 张国云,章兢.一种新的分裂层次聚类SVM多值分类器.控制与决策,2005,20 (8):931-934
[56] 李蓉,叶世伟,史忠植.SVM-KNN分类器-一种提高SVM分类精度的新方法.电子学报, 2002,30(5): 745-748
[57] 李红莲,王春花,袁保宗.一种改进的支持向量机NN-SVM.计算机学报, 2003, 26(8): 1015-1020
[58] 马笑天,黄席樾.基于SVM的二叉树多类分类算法及其在故障诊断中的应用.控制与决策, 2003, 18(3):272-284
[59] Arenas-Garcia J, Perez-Cruz F. Multi-class support vector machines:a new approach. In Proceedings of IEEE International Conference on Acoustics Speech and Signal Processing, New York, 2003,2,781-784
[60] Oren M.,Papageorgiou C., Sinha P. et al. Pedestrain detection using wavelet templates. In Proceedings of CVPR’97,Puerto Rico,1997,193-199
[61] Osuna E.,Freund R., Girosi F.Trainsing support vector machines:an application to face detection. In Proceedings of CVPR’97,Puerto Rico,1997,130-136
[62] 冯元戬,施鹏飞. 基于支持向量机的彩色图像人脸检测方法.上海交通大学学报,2003,37(6):947-951
[63] 燕忠,袁春伟.基于蚁群智能和支持向量机的人脸性别分类方法.电路与系统学报,2004,26(8):1177-1182
[64] 张敏贵,潘泉,张洪才等. 基于支持向量机的人脸分类.计算机工程,2004, 38 (11):110-112
[65] 宋晓宁,束鑫. 一种改进的模糊支持向量机的人脸识别方法.微机发展,2005, 15(3):23-27
[66] 马勇,丁晓青. 基于层次型支持向量机的人脸检测.清华大学学报,2003, 43 (1):35-38
[67] Shuotao Li, Yaonan Wang. Face Recognition using Wavelet Transform and Support Vector Machines. Pattern Recognition and Image Analysis, 2004, 14(3): 471-478
[68] Reyna R. A., Hernandez N., Esteve D. et al. Segmenting images with support vector machines. In Proceedings of 2000 International Conference on Image Processing, Vancover, BC, Canada ,2000,1(1):820-823
[69] Valentini A., Zhang Hongjiang. Automatic image orientation detection. IEEE Trans on Image Processing, 2002,11(7):746-755
[70] Guo Guo-Dong,Jain A. K., Ma Wei-Ying et al. Learning similarity measeure for natural image retrieval with relevance feedback. IEEE Trans on Neural Networks, 2002, 13(4):811-820
[71] Brown M., Lewis H. G.,Gunn S. R..Linear spectral mixture models and support vector machines for remote sensing. IEEE Trans on Geoscience and Remote Sensing, 2000, 38(5):2346-2360
[72] Shutao Li, James T. Yaonan Wang. Texture classication using the support vector machines. Pattern Recognition ,2003,36: 2883-2893
[73] Yongmin Li, Shaogang Gong, Jamie Sherrah,Heather Liddell. Support vector machine based multi-view face detection and recognition. Image and Vision Computing, 2004, 22: 413–427
[74] Jian-xiong Dong, Adam Krzy_zak, Ching Y. Suen . An improved handwritten Chinese character recognition system using support vector machine. Pattern Recognition Letters, 2005, 26 :1849-1856
[75] Valentini and Giorgio.Gene expression data analysis of human lymphoma using support vector machines and output coding ensembles.Artificial Intelligence in Medicine, 2002,26(3):281-304
[76] Cai Yu-dong, Liu Xiao-jun,Xu Xue-biao, et al. Prediction of protein structuralclasses by support vector machines. Computers and Chemistry, 2002, 26(3): 293- 296
[77] Bao L., Sun Z.. Identifying genes related to drug anticancer mechanisms using support vector machine. FFBS letters, 2002, 521(1):109-114
[78] Liu Weiqiang, Shen Peihua, Qu Yingge et al. Fast algorithm of support vector machines in lung cancer diagnosis. In proceedings of International Workshop on Medical Imaging and Augmented Reality, Hong Kong, 2001,188-192
[79] Bhanu P. K. N., Ramakrishnan A. G., Suresh S. et al. Fetal lung maturity analysis using ultrasound image features. IEEE Trans on Information Technology in Biomedicine, 2002, 6(1):38-45
[80] Ruey-Feng Chang, Wen-Jie Wu, Woo Kyung Moon et al. Support Vector Machines for Diagnosis of Breast Tumors on US Images.Academic Radiology, 2003, 10(2):189-197
[81] Jongcheol Kim,Sangchul Won. New fuzzy inference system using a support vector machine.In Proceedings of the 41st IEEE Conference on Decision and Control, Las Vegas, Nevada USA, 2002,1349-1354
[82] Tony Van G. Johan A. K. Suykens. Dirk-Emma Baestaens, et al. Financial time series prediction using least squares support vector machines within the evidence framework. IEEE Trans on Neural Networks, 2001,12(4)809-821
[83] Abhijit Kulkarni,V. K. Jayaraman, B. D. Kulkarni. Control of chaotic dynamical systems using support vector machines.Physics Letters A, 2003, 317:429-435
[84] M. A. Mohandes, T.O. Halawani, S. Rehman, et al. Support vector machines for wind speed prediction. Renewable Energy, 2004,29:939-947
[85] Johan A. K. Suykens. Nolinear Modelling and support vector machines. In Proceedings of International Conference on IEEE Instrumentation and Measurement Technology, Budapest, Hungary, 2001,287-294
[86] Sun Zonghai, Sun Youxian.Optimal control by weighted least squares generalized support vector machines. In Proceedings of the American Control Conference, Denver, Colorado, 2003, 5323-5328
[87] Wei Huang,Yoshiteru Nakamori,Shou-Yang Wang. Forecasting stock market movement direction withsupport vector machine. Computers and Operations Research, 2005,32 : 2513-2522
[88] Kyung-Shik Shin,Taik Soo Lee,Hyun-jung Kim. An application of support vector machines in bankruptcy prediction model. Expert Systems with Applications, 2005, 28: 127–135
[89] Ping-Feng Pai,Chih-Sheng Lin. AhybridARIMAand support vector machines model in stock price forecasting.The International Journal of Management Science, 2005,33: 497-505
[90] 马笑箫,黄席樾,柴毅.基于支持向量机的故障过程趋势预测研究.系统仿真学报,2002,14(11):1548-1551
[91] 王定成,方廷健,高理富等.支持向量机回归在线建模及应用.控制与决策,2003, 18(1):89-95
[92] 赵登福,王蒙,张讲社等.基于支撑向量机的短期负荷预测.中国电机工程学报, 2002, 22(4):26-30
[93] 陈友,张国基,郭国雄.一种改进的SVM算法及其在证券领域中的应用.华南理工大学学报,2003, 31(7):15-18
[94] 李丽娜,候朝桢.基于支持向量机(SVM)的工业过程辨识.北京理工大学学报, 2003,23(5):569-580
[95] 翟永杰,毛继珮.分级聚类支持向量机在汽轮机故障诊断中的应用.华北电力大学学报,2003,30(6):25-29
[96] 张国云,章兢.基于模糊支持向量机的多级二叉树分类器的水轮机调速系统故障诊断.中国电机工程学报,2005,25(8):100-104
[97] 张国云,章兢.基于粗集理论和支持向量机的动态预测新方法及应用.自动化仪表, 2005, 26(2):17-20
[98] Zhang Guoyun,Zhang Jing.A new tree-like fuzzy binary support vector machine for optical character recognition. In Proceedings of SPIE, Beijing,2004, 5637, 212-217
[99] 王宇红,黄德先.基于支持向量机的非线性预测控制技术.信息与控制,2004, 33(2):134-140
[100] 王宇红,黄德先,高东杰等.基于LS-SVM地非线性预测控制技术.控制与决策, 2004,19(4):383-387
[101] 张浩然,韩正之,李昌刚.基于支持向量机的未知非线性系统辨识与控制.上海交通大学学报,2003,37(6):927-930
[102] 高隽.人工神经网络原理及仿真实例.北京:机械工业出版社,2003,76-93
[103] 李国正,王猛,曾华军.支持向量机导论.北京:电子工业出版社,2004
[104] Chunhui Chen,O.L. Mangasarian.Smoothing methods for convex inequalities and linear complementarity problems. Mathematical Programming, 1995, 71:51-69
[105] Glenn Fung.O.L. Mangasarian. Finite Newton method for Lagrangian support vector machine classi!cation. Neurocomputing ,2003,55:39-55
[106] Yuh-Jye Lee.Support vector machines in data mining:[dissertation].Wisconsin: University of Wisconsin-Madison, 2001,19-31
[107] Yuh-Jye Lee,Wen-Feng Hsieh,Chien-Ming Huang .spl epsi/-SSVR: a smooth support vector machine for /spl epsi/-insensitive regression. IEEE Transactions on Knowledge and Data Engineering,2005,17(5)678-585
[108] Joao Pedro, Noboru Murata. Support vector machines with different norms: motivation, formulations and results.Pattern Recognition Letters, 2001, 22: 1263-1272
[109] O.L. Mangasarian. Arbitrary-norm separating plane. Operations Research Letters, 1999, 24:15-23
[110] 蔡春,周博,李杨.一范数支持向量机的特征提取算法及应用.统计与决策, 2004, 172(4):18-19
[111] Glenn M. Fung, O.L. Mangasarian.A feature selection Newton method for support vector machine classification. Computational Optimization and Applications, 2004, 28:185-202
[112] 边肇祺,张学工.模式识别.北京:清华大学出版社,2005,212-310
[113] 温熙森,胡莺庆,邱静.模式识别与状态监控.长沙:国防科技大学出版社, 1997, 180-185
[114] Scholkopf B, Smola A,Muller K R. Nonlinear component analysis as a kernel eigenvalue problems. Neural Computation, 1998, 10(5):299-139
[115] 黄国宏,邵惠鹤.核主元分析及其在人脸识别中的应用.计算机工程, 2004, 30(13):13-15
[116] 王华国,俞金寿.基于核函数主元分析的软测量建模方法及应用.华东理工大学学报, 2004, 30(5):567-570
[117] 刘贵忠,邸双亮. 小波分析及其应用.西安:西安电子科技大学出版社,1992
[118] 飞思科技产品研发中心.Matlab6.5辅助小波分析与应用.北京:电子工业出版社,2003
[119] 崔万照,朱长纯,保文纯等.最小二乘小波支持向量机在非线性系统辨识中的应用.西安交通大学学报, 2004, 38(6):562-586
[120] Yuancheng Li, Bo Li, Tiangjian Fang. Short-term load forecast based on fuzzy wavelete support vector machines. In proceedings of the 5th World Congress on Intelligent Control and Automation, Hangzhou, P.R. China, 2004,5194-5198
[121] 张文修,吴伟志,梁吉业,等.粗造集理论与方法.北京:科学出版社,2001,12-22
[122] 张家树, 肖先赐. 混沌时间序列的Volterra自适应预测.物理学报, 2000, 49 (03): 403-407.
[123] 张家树, 肖先赐. 混沌时间序列的自适应高阶非线性滤波预测. 物理学报, 2000, 49(07): 1221-1227.
[124] 张家树, 肖先赐. 用于混沌时间序列自适应预测的一种少参数二阶Volterra滤波器. 物理学报, 2001, 50(07): 1248-1254.
[125] 谭文, 王耀南. 混沌时间序列的模糊神经网络预测. 物理学报, 2003, 52(04): 797-801.
[126] 陈梁.我国水电建设和科技进步50年.中国电力,1999,32(10):21-24
[127] 王海. 水轮发电机组状态监测、诊断及综合试验分析系统研究:[华中科技大学博士学位论文]. 武汉:华中科技大学水电与数字化工程学院,2001, 8-14
[128] 余晓晖,杜林,陈明英等.基于BP神经网络的水轮机调速系统故障诊断.重庆大学学报, 2001, 24(11):71-74
[129] 王道平,张义忠. 故障智能诊断系统的理论与方法.北京:冶金工业山版社, 2001
[130] N. R. Pal, J. C. Bezdek.On cluster validity for the fuzzy c-means model.IEEE Transaction on Fuzzy Systems,1995,3(3):370-379