最小二乘支持向量机算法及应用研究
|
摘要
支持向量机(Support Vector Machines, SVM)是基于统计学习理论,借助于最优化方法来解决机器学习问题的新工具,它由Vapnik和Cortes于1995年提出,已成为近年来机器学习研究的一项重大成果。SVM在解决小样本、高维模式识别等问题中有着独特的优势,有效地克服了神经网络分类等传统分类方法的许多缺点,具有较高的泛化性能。最小二乘支持向量机(Least Squares Support Vector Machines, LS-SVM)是标准SVM的推广,具有标准SVM的诸多优点,同时,LS-SVM将模型的训练过程归结为一个线性方程组的求解问题,避免了标准SVM中的约束凸二次规划问题,从而简化了计算过程,大大降低了计算复杂度,提高了求解速度。SVM作为一门新兴技术,还需要在理论和应用的深度和广度上加以完善。本文主要在LS-SVM模型优化和特殊数据分类上进行了较深入的研究和实践,用以提高分类器的分类性能和泛化能力。
本文的主要研究工作如下:
1.在深入分析了SVM和LS-SVM的分类机理的基础上,研究了LS-SVM模型参数选择对模型分类的影响,指出只有选择合适的规范系数和核函数参数组合时,LS-SVM分类器才具有最优的分类性能和泛化能力。
2.提出了基于多样性保持的分布估计算法(EDA-DP)的LS-SVM模型参数优化选择的方法,阐明了该方法的基本思路和主要步骤,并将算法应用于UCI基准数据集和雷达目标高分辨距离像(HRRP)的识别。实验表明,基于EDA-DP算法优化的LS-SVM模型具有良好的分类性能和泛化能力。
3.在回顾不平衡数据欠抽样技术的基础上,研究了在核特征空间内对不平衡数据进行欠抽样的可行性,得出结论:基于SVM的不平衡数据分类,若使用径向基核函数,当输入空间中样本间距离较小时,在特征空间内对多数类利用有关最近邻的技术进行欠抽样,可以取得较优的效果;而若输入空间中样本间距离较大,则在特征空间中对多数类欠抽样,其最近邻技术可能会失去期望的效果。
4.在深入分析了不平衡数据的分类特性的基础上,提出了基于聚类-邻域清理(SBC-NCL)的最小二乘支持向量机不平衡数据分类算法,并将算法应用于UCI基准数据集和雷达目标HRRP识别实验,证明了算法具有较高的泛化能力。
Support Vector Machines(SVM) is a new developed machine learning method based on the Statistical Learning Theory(SLT) and optimization theory. It was proposed firstly by Vapnik and Cortes in 1995, it has become an important research direction in the field of machine learning. SVM has many advantages in pattern recognition, such as the superiority in small-sample and high-dimension problems, and it resolves the shortcomings of neural networks and other traditional classification methods effectively for its good performance and high generalization ability. Least Squares Support Vector Machines(LS-SVM) is developed from SVM. LS-SVM has all the advantages which SVM has, and it trains the model by linear equation group resolving, which reduces computing complexity and increases the solving speed, but not the quadratic programming problem as in SVM. However, as a new technique, researches on SVM still need be worked in theories and applications. The paper researches on SVM in model optimal selection and classification of special datasets in order to improve classification ability and generalization capacity of the classifier. The following parts are main works:
1. Effects from LS-SVM model hyperparameter selection to classifiers were disscussed based on classification principle of SVM and LS-SVM. Then the facts were proposed that classifier with minimum structural risk is got just when penalty coefficient and kernel parameter are all appropriate.
2. A method using estimation of distribution algorithms with diversity preservation (EDA-DP) to optimally select model parameters of LS-SVM was proposed, and the method was applied in recognition on UCI benchmark datasets and radar target high resolution range profile(HRRP) datasets. Experimental results showed that the classification model based on the algorithm had good ability.
3. Under-sampling approaches for imbalanced data in feature space were discussed, and only when the distances between samples were smaller, some good effects of the classification on imbalance data would got by SVM which RBF was using.
4. Cluster-based under-sampling and neighborhood cleaning approach(SBC-NCL) for imbalanced data classification by LS-SVM was proposed. The algorithm was applied in recognition on UCI benchmark datasets and radar target high resolution range profile(HRRP) datasets, and the results showed that the classification model based on the algorithm had good generalization capacity.
本文的主要研究工作如下:
1.在深入分析了SVM和LS-SVM的分类机理的基础上,研究了LS-SVM模型参数选择对模型分类的影响,指出只有选择合适的规范系数和核函数参数组合时,LS-SVM分类器才具有最优的分类性能和泛化能力。
2.提出了基于多样性保持的分布估计算法(EDA-DP)的LS-SVM模型参数优化选择的方法,阐明了该方法的基本思路和主要步骤,并将算法应用于UCI基准数据集和雷达目标高分辨距离像(HRRP)的识别。实验表明,基于EDA-DP算法优化的LS-SVM模型具有良好的分类性能和泛化能力。
3.在回顾不平衡数据欠抽样技术的基础上,研究了在核特征空间内对不平衡数据进行欠抽样的可行性,得出结论:基于SVM的不平衡数据分类,若使用径向基核函数,当输入空间中样本间距离较小时,在特征空间内对多数类利用有关最近邻的技术进行欠抽样,可以取得较优的效果;而若输入空间中样本间距离较大,则在特征空间中对多数类欠抽样,其最近邻技术可能会失去期望的效果。
4.在深入分析了不平衡数据的分类特性的基础上,提出了基于聚类-邻域清理(SBC-NCL)的最小二乘支持向量机不平衡数据分类算法,并将算法应用于UCI基准数据集和雷达目标HRRP识别实验,证明了算法具有较高的泛化能力。
Support Vector Machines(SVM) is a new developed machine learning method based on the Statistical Learning Theory(SLT) and optimization theory. It was proposed firstly by Vapnik and Cortes in 1995, it has become an important research direction in the field of machine learning. SVM has many advantages in pattern recognition, such as the superiority in small-sample and high-dimension problems, and it resolves the shortcomings of neural networks and other traditional classification methods effectively for its good performance and high generalization ability. Least Squares Support Vector Machines(LS-SVM) is developed from SVM. LS-SVM has all the advantages which SVM has, and it trains the model by linear equation group resolving, which reduces computing complexity and increases the solving speed, but not the quadratic programming problem as in SVM. However, as a new technique, researches on SVM still need be worked in theories and applications. The paper researches on SVM in model optimal selection and classification of special datasets in order to improve classification ability and generalization capacity of the classifier. The following parts are main works:
1. Effects from LS-SVM model hyperparameter selection to classifiers were disscussed based on classification principle of SVM and LS-SVM. Then the facts were proposed that classifier with minimum structural risk is got just when penalty coefficient and kernel parameter are all appropriate.
2. A method using estimation of distribution algorithms with diversity preservation (EDA-DP) to optimally select model parameters of LS-SVM was proposed, and the method was applied in recognition on UCI benchmark datasets and radar target high resolution range profile(HRRP) datasets. Experimental results showed that the classification model based on the algorithm had good ability.
3. Under-sampling approaches for imbalanced data in feature space were discussed, and only when the distances between samples were smaller, some good effects of the classification on imbalance data would got by SVM which RBF was using.
4. Cluster-based under-sampling and neighborhood cleaning approach(SBC-NCL) for imbalanced data classification by LS-SVM was proposed. The algorithm was applied in recognition on UCI benchmark datasets and radar target high resolution range profile(HRRP) datasets, and the results showed that the classification model based on the algorithm had good generalization capacity.
引文
[1]孙即祥.现代模式识别[M].长沙:国防科技大学出版社,2003.
[2]李介谷,蔡国康.计算机模式识别技术[M].上海:上海交通大学出版社,1986.
[3]岳素林.基于高分辨距离像的雷达目标识别方法研究[D].西安电子科技大学硕士学位论文,2005. 1.
[4]闫锦.基于高距离分辨像的雷达目标识别研究[D].中国航天第二研究院硕士学位论文,2004, 1.
[5] Lunts A., Brailovskiy V. Evaluation of Attributes Obtained in Statistical Decision Rules[J]. Engineering Cybernetics, 1967, Vol.3, No.1: 98-109.
[6]范明,柴玉梅,昝红英等译.统计学习基础-数据挖掘、推理与预测[M].北京:电子工业出版社,2004.
[7] Vapnik v., Chapelle O. Bounds on Error Expectation for Support Vector Machine[M]. Smola, Bartlett, et a1.(eds.), Advances in Large Margin Classifiers, Cambridge, MA: MIT Press, 2000: 5-6.
[8] Opper M., Winther O. Ganssian Processes and SVM: Mean Field and Leave-one-out[M]. Smola, Bartlett, et a1.(eds.), Andvances in Large Marge Classifiers. Cambridge, MA: MIT Press. 2000: 311-326.
[9] Joachims T. Estimating the Generalization Performance of a SVM Efficiently[C]. Proc. of the International Conference on Machine Learning. Morgan Kaufman, 2000.
[10] Jaakkola T., Haussler D. Probabilistic Kernel Regression Models[C]. Proc. of the 7th Workshop on AI and Statistics, San Francisco, 1999: 26-34.
[11] Chung K-M, et a1. Radius Margin Bounds for Support Vector Machines with the RBF Kernel[J]. Neural Computation, 2003.
[12] Wahba G., Lin Y., Zhang H. Generalized Approximate Cross-validation for Support Vector Machines: Another Way to Look at Margin-like Quantities[M]. Smola, Bartlett, et a1.(ed.), Andvances in Large Marge Classifiers, Cambridge, MA: MIT Press, 2000: 397-309.
[13] Cortes C., Vapnik V. Support Vector Networks[J]. Machine Learning, 1995, Vol.20: 273-297.
[14] Osuna E., Freund R. An Improved Training Algorithm for Support Vector Machines[C]. Proc. of the IEEE Workshop on Neural Networks for Signal Processing, 1997
[15] Joachims T. Making Large-scale SVM Learning Practical[M]. Scholkopf B., Burges C.J., et a1.(eds.), Advances in Kernel Methods: Support Vector Learning, MITPress, Cambridge, MA, 1999.
[16] Platt J.C. Fast Training of Support Vector Machines using Sequential Minimal Optimization[M]. Scholkopf B., Burges C.J., et a1.(eds.). Advances in Kernel Methods: Support Vector Learning, MIT Press, Cambridge, MA, 1999.
[17] Scholkopf B., Smola A.Bartlett P. New Support Vector Algorithms[J]. Neural Computation, 2000, Vol.12: 1207-1245.
[18] Cao L.J., Keerthi S.S., et a1. Parallel Sequential Minimal Optimization for the Training of Support Vector Machines[J]. IEEE Trans. on Neural Networks, 2006, Vol.l7, No.4: 1039-1049.
[19] Dong J.X., Krzyzak A., et a1. Fast SVM Training Algorithm with Decomposition on Very Large Data Sets[J]. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2005, Vol.27, No.4: 603-618.
[20] Mangasarian O.L., A Finite Newton Method for Classification[J]. Optimization Methods and Software, 2002, Vol.17: 913-929.
[21] Valentini G., Dietterieh T. Bias-Variance Analysis of Support Vector Machines for the Development of SVM-Based Ensemble Methods[J]. Journal of Machine Learning Research, 2004, Vol.5: 725-775.
[22]张翔,肖小玲,徐光祜.基于样本之间紧密度的模糊支持向量机方法[J].软件学报,2006,Vol.l7, No.5: 951-958.
[23] SUYKENS J A K, VANDEWALIE J, Least Squares Support Vector Machine Classifiers[J], Neural Processing Letters, 1999,9(3):293-300
[24] Valyon J., Horvath G. A Sparse Least Squares Support Vector Machines Classifier[C]. Proc. of 2004 IEEE International Joint Conference on Neural Network, Budapest, Hungary, 2004.
[25] Mangasarian O.L. Generalized Support Vector Machines[M]. Alexander J.S., Bartlett P., et al.(eds.), Advances in Large Margin Classifiers, MIT Press, Cambridge, 2000: 135-146.
[26] Mangasarian O.L., Musicant D.R. Active Support Vector Machine Classification[M]. Lee Todd K., Dietterich Thomas G., Volker Tresp(eds.), Advances in Neural Information Processing Systems, MIT Press, 2001,:577-583.
[27] Mangasarian O.L., Musicant D.R. Lagrangian Support Vector Machines[J]. Journal of Machine Learning Reasearch, 2001(3):161-177.
[28] Lee Yuh-Jye, Mangasarian O.L. RSVM: Reduced Support Vector Machines[C]. Proc. of the First SIAM International Conference on Data Ming, Chicago, 2001.4:00-07.
[29] Fung Glenn, Mangasarian O.L. Proximal Support Vector Machine[C]. Proc. of the 7th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, California,2001.
[30] Fung Glenn, Mangasarian O.L. Incremental Support Vector Machine Classification[C]. Proc. of the Second SIAM International Conference on Data Ming, Arlington, USA, 2002.
[31] Song Qing, Hu Wenjie, Xie Wenfan. Robust Support Vector Machine for Bullet Hole Image Classification[J]. IEEE Trans. on Systems, Man and Cybernetics, 2002, 32(4): 440-448.
[32] Lee KiYoung , Kim Dae-Won, et al. Possibilistic Support Vector Machines[J]. Pattern Recognition, 2005, 38(8): 1325-1327.
[33] Bcnneth K.P., Demirez A. Semi-Supervised Support Vector Machines[C]. Kristin P. Bennett, Ayhan Demiriz, eds. Proc. of the 1998 Conference on Advances in Neural Information Processing Systems, MIT Press, Cambridge, MA, USA, 1998: 368-374.
[34] Maryanne Domm, Andrew Engel, Peguy Pierre-Louis. An Integer Support Vector Machine[C]. The sixth International Conference on SNPD/SAWN, Hainan, China, 2005.
[35] Scholkopf B, Platt J., Shawe-Taylor J. Estimating the Support of a high-dimensional distribution[J]. Neural Computation, 2001, 13(7): 1443-1471.
[36] Q. Zhao, J. C. Pineiple, Support vector machines for SAR automatic larger recognition[J]. IEEE Trans. AES, 2001, 37(2): 643-654
[37] J.U,J. C. Principe. Statistical pattern recognition for SAR/ATR[J]. USAFRL Technical Report, July, 2001
[38] R.E. Karisen, el. al. Target classification via support vector machines[J]. Optical Engineering, 2000, 39(3): 704-711
[39]宋晓宁,束鑫.一种改进的模糊SVM的人脸识别方法[J].微机发展, 2005, 15(3): 23-25,28.
[40]熊浩勇,基于SVM的中文文本分类算法研究与实现[D].武汉理工大学硕士学位论文,2008.
[41]谢秋玲.应用于心电图分类的KNN和SVM分类器研究[D].华东师范大学硕士学位论文, 2004.
[42]李雅琴. SVM在手写数字识别中的应用研究[D].华中师范大学硕士学位论文, 2007.
[43]马君国,赵宏钟,王微.基于最优参数支持向量机的目标识别算法[J]. IEEE, 2009.
[44]马永军,李孝忠,王希雷.基于模糊SVM和核方法的目标检测方法研究[J].天津科技大学学报,2005, 20(3): 29-32.
[45]李亚楠,耿伯英,杨武.基于SVM和辐射噪声的舰船目标识别[J].舰船电子工程,2006, 21(1): 58-60.
[46]万力,盘善荣,傅明.基于SVM的P2P流量识别[J].计算机技术与自动化,2009(3), Vol.28, No.1: 112-115.
[47]白鹏,张喜斌,张斌等.支持向量机理论及工程应用实例.西安:西安电子科技大学出版社,2008.
[48] Kehai Xin,Xuegong Zhang.Editing support vector machines[C]. Proc. of International Joint Conference on Neural Networks, Washington, USA, 2001(2): 1464-1467.
[49]李红莲,王春花,袁保宗.一种改进的支持向量机NNSVM[J].计算机学报,2003, 8.
[50] Chew Hong-Gunn, Bogner R.E., Lim Cheng-Chew. Dual-Support Vector Machine with Error Rter and Training Size Biasing[C]. Proc. of Acoustics, Speech, and Signal Processing, SaltLake City, Utah, USA, 2001.
[51]郭雷,肖怀铁,付强.非均衡数据目标识别中的SVM模型多参数优化选择方法[J].红外与毫米波学报,2009(4), Vol.28. No.2.
[52] Raskutti B., Kowalczyk A. Extreme rebalancing for SVMs: a cage study[J]. News letter of the ACM Special Interest Group on Knowledge Discovery and Data Mining, 2004, 6(1): 61-69.
[53] Lee Y., Lin Y., Wahba G. Multicategory support vector machines: theory and application to the classification of microarray data and satellite radiance data[R]. Wisconsin: University of Wisconsin, 2002.
[54] Rosenblatt F. Principles of Neurodinamics: Perceptron and Theory of Brain Mechanisms[M]. Spartan Books, Washington D.C., 1962.
[55]李国正等译.支持向量机导论[M](Cristianini N. Shawe-Taylor J. An Introduction to Support Vector Machines and Other Kernel-based Learning Methods).北京:电子工业出版社,2004, 3.
[56]丁爱玲,刘芳,姚霞.基于支撑矢量机的智能目标识别方法[J].西安电子科技大学学报(自然科学版),2001, Vol . 28, No. 6: 743-746.
[57]张莉,周伟达,焦李成.用于一维图像识别的支撑矢量机方法[J].红外与毫米波学报,2002, 4, Vol. 21, No. 2: 119-123
[58] Vapnik V. The Nature of Statistical Learning Theory [M]. New York, USA: Springer, 1995.
[59] LeCun Y, et a1. Backpropagation applied to handwritten zip code recognition[J]. Neural Comput., 1999, l: 541-551.
[60] Kubat M., Matwin S. Addressing the Course of Imbalanced Training Sets: One-Sided Selection[C]. International Conference on Machine Learning, 1997: 179-186.
[61]李京华,张聪颖,倪宁.基于参数优化的支持向量机战场多目标声识别[J].探测与控制学报,2010, Vol.32, No.1: 1-5.
[62] Chawla N.V., Lazarevic A, Hall LO., Bowyer K. SMOTEBoost:Improving prediction of the Minority Class in Boosting[C]. 7th European Conference on Principles and Practice of Knowledge Discovery in Databases, 2003: 107-119.
[63] Chapelle O., Vapnik V. Choosing Multiple Parameters for Support Vector Machines[J]. Machine Learning, 2002, 46: 131-159.
[64]王克奇,杨少春,戴天虹等.采用遗传算法优化最小二乘支持向量机参数的方法[J].计算机应用与软件,2009(7), Vol.26. No.7: 109-111.
[65] Yang Hong, Lou Fei, Xu Yuge, Liang Jin. GA Based LS-SVM Classifier for Waste Water Treatment Process[C]. Proc. of the 27th Chinese Control Conference, 2008(7): 436-439.
[66]张培林,钱林方,曹建军等.基于蚁群算法的支持向量机参数优化[J].南京理工大学学报(自然科学版),2009, Vol.33, No.4: 464-468.
[67]王东霞,张楠,路晓丽.基于育种算法的SVM参数优化[J].安徽大学学报(自然科学版),2009, Vol.33, No.4: 26-28.
[68] Adankon M. M., Cheriet M. Model Selection for the LS-SVM. Application to Handwriting Recognition[J]. Pattern Recognition, 2009, 42: 3264-3270.
[69] Suykens J.A.K, Vandewalle J. Least Squares Support Vector Machine Classifiers[J]. Neural Processing Letters(S1573-773X), 1999, 9(3): 293-300.
[70]刘东辉,卞建鹏,付平等.支持向量机最优参数选择的研究[J].河北科技大学学报,2009(3), Vol.30, No.1: 58-61.
[71]史忠植.高级人工智能(第二版)[M].北京:科学出版社,2006.
[72]王雪松,程玉虎,等.机器学习理论、方法及应用[M].北京:科学出版社,2009.
[73]周树德,孙增圻.分布估计算法综述[J].自动化学报,2007(2), V0l.33, No.2: 113-124.
[74] WU G. CHANG E. Class-boundary alignment for imbalanced dataset learning[C]. Workshop on Learning from Imbalanced Datasets(ICML 2003), Washington, DC, 2003: 49-56.
[75] Batuwita R, Palade V. FSVM-CIL: Fuzzy Support Vector Machines for Class Imbalance Learning[J]. IEEE Trans. on Fuzzy Systems, 2010(6),Vol. 18, No. 3: 558-571.
[76] Imam T, Ming Ting K, Kamruzzaman J. z-SVM: An SVM for Improved Classification of Imbalanced Data[C]. AI 2006, LNAI 4304: 264-273.
[77]薛贞霞,刘三阳,刘万里.不平衡最小二乘支持向量机[J].系统仿真学报, 2009(7), Vol. 21, No. 14: 4324-4327.
[78] Chawla N, Bowyer K, Hall L, et a1. SMOTE: synthetic minorityover-sampling technique[J]. Journal of Artificial Intelligence Research, 2002, 16: 321-357.
[79] Zhi-Qiang Zeng, Ji Gao. Improving SVM Classification with Imbalance Data Set[C]. ICONII 2009, LNCS 5863: 389-398.
[80] Laurikkala J. Improving Identification of Difficult Small Classes by Balancing Class Distribution[C]. Proc of the 8th Conference on AI in Medicine, Europe: Artificial Intelligence Medicine, 2001: 63-66.
[81] Show-Jane Yen, Yue-Shi Lee. Cluster-based Under-sampling Approaches for Imbalanced Data Distributions[J]. Expert Systems with Applications, 2009, 36: 5718-5727.
[82] KUBAT M, MATWIN S. Addressing the curse of imbalanced training sets:one-sided selection[C]. Proc. of the 14th International Conference on Machine Learning. San Francisco, CA: Morgan Kaufmann 1997: 217-225.
[2]李介谷,蔡国康.计算机模式识别技术[M].上海:上海交通大学出版社,1986.
[3]岳素林.基于高分辨距离像的雷达目标识别方法研究[D].西安电子科技大学硕士学位论文,2005. 1.
[4]闫锦.基于高距离分辨像的雷达目标识别研究[D].中国航天第二研究院硕士学位论文,2004, 1.
[5] Lunts A., Brailovskiy V. Evaluation of Attributes Obtained in Statistical Decision Rules[J]. Engineering Cybernetics, 1967, Vol.3, No.1: 98-109.
[6]范明,柴玉梅,昝红英等译.统计学习基础-数据挖掘、推理与预测[M].北京:电子工业出版社,2004.
[7] Vapnik v., Chapelle O. Bounds on Error Expectation for Support Vector Machine[M]. Smola, Bartlett, et a1.(eds.), Advances in Large Margin Classifiers, Cambridge, MA: MIT Press, 2000: 5-6.
[8] Opper M., Winther O. Ganssian Processes and SVM: Mean Field and Leave-one-out[M]. Smola, Bartlett, et a1.(eds.), Andvances in Large Marge Classifiers. Cambridge, MA: MIT Press. 2000: 311-326.
[9] Joachims T. Estimating the Generalization Performance of a SVM Efficiently[C]. Proc. of the International Conference on Machine Learning. Morgan Kaufman, 2000.
[10] Jaakkola T., Haussler D. Probabilistic Kernel Regression Models[C]. Proc. of the 7th Workshop on AI and Statistics, San Francisco, 1999: 26-34.
[11] Chung K-M, et a1. Radius Margin Bounds for Support Vector Machines with the RBF Kernel[J]. Neural Computation, 2003.
[12] Wahba G., Lin Y., Zhang H. Generalized Approximate Cross-validation for Support Vector Machines: Another Way to Look at Margin-like Quantities[M]. Smola, Bartlett, et a1.(ed.), Andvances in Large Marge Classifiers, Cambridge, MA: MIT Press, 2000: 397-309.
[13] Cortes C., Vapnik V. Support Vector Networks[J]. Machine Learning, 1995, Vol.20: 273-297.
[14] Osuna E., Freund R. An Improved Training Algorithm for Support Vector Machines[C]. Proc. of the IEEE Workshop on Neural Networks for Signal Processing, 1997
[15] Joachims T. Making Large-scale SVM Learning Practical[M]. Scholkopf B., Burges C.J., et a1.(eds.), Advances in Kernel Methods: Support Vector Learning, MITPress, Cambridge, MA, 1999.
[16] Platt J.C. Fast Training of Support Vector Machines using Sequential Minimal Optimization[M]. Scholkopf B., Burges C.J., et a1.(eds.). Advances in Kernel Methods: Support Vector Learning, MIT Press, Cambridge, MA, 1999.
[17] Scholkopf B., Smola A.Bartlett P. New Support Vector Algorithms[J]. Neural Computation, 2000, Vol.12: 1207-1245.
[18] Cao L.J., Keerthi S.S., et a1. Parallel Sequential Minimal Optimization for the Training of Support Vector Machines[J]. IEEE Trans. on Neural Networks, 2006, Vol.l7, No.4: 1039-1049.
[19] Dong J.X., Krzyzak A., et a1. Fast SVM Training Algorithm with Decomposition on Very Large Data Sets[J]. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2005, Vol.27, No.4: 603-618.
[20] Mangasarian O.L., A Finite Newton Method for Classification[J]. Optimization Methods and Software, 2002, Vol.17: 913-929.
[21] Valentini G., Dietterieh T. Bias-Variance Analysis of Support Vector Machines for the Development of SVM-Based Ensemble Methods[J]. Journal of Machine Learning Research, 2004, Vol.5: 725-775.
[22]张翔,肖小玲,徐光祜.基于样本之间紧密度的模糊支持向量机方法[J].软件学报,2006,Vol.l7, No.5: 951-958.
[23] SUYKENS J A K, VANDEWALIE J, Least Squares Support Vector Machine Classifiers[J], Neural Processing Letters, 1999,9(3):293-300
[24] Valyon J., Horvath G. A Sparse Least Squares Support Vector Machines Classifier[C]. Proc. of 2004 IEEE International Joint Conference on Neural Network, Budapest, Hungary, 2004.
[25] Mangasarian O.L. Generalized Support Vector Machines[M]. Alexander J.S., Bartlett P., et al.(eds.), Advances in Large Margin Classifiers, MIT Press, Cambridge, 2000: 135-146.
[26] Mangasarian O.L., Musicant D.R. Active Support Vector Machine Classification[M]. Lee Todd K., Dietterich Thomas G., Volker Tresp(eds.), Advances in Neural Information Processing Systems, MIT Press, 2001,:577-583.
[27] Mangasarian O.L., Musicant D.R. Lagrangian Support Vector Machines[J]. Journal of Machine Learning Reasearch, 2001(3):161-177.
[28] Lee Yuh-Jye, Mangasarian O.L. RSVM: Reduced Support Vector Machines[C]. Proc. of the First SIAM International Conference on Data Ming, Chicago, 2001.4:00-07.
[29] Fung Glenn, Mangasarian O.L. Proximal Support Vector Machine[C]. Proc. of the 7th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, California,2001.
[30] Fung Glenn, Mangasarian O.L. Incremental Support Vector Machine Classification[C]. Proc. of the Second SIAM International Conference on Data Ming, Arlington, USA, 2002.
[31] Song Qing, Hu Wenjie, Xie Wenfan. Robust Support Vector Machine for Bullet Hole Image Classification[J]. IEEE Trans. on Systems, Man and Cybernetics, 2002, 32(4): 440-448.
[32] Lee KiYoung , Kim Dae-Won, et al. Possibilistic Support Vector Machines[J]. Pattern Recognition, 2005, 38(8): 1325-1327.
[33] Bcnneth K.P., Demirez A. Semi-Supervised Support Vector Machines[C]. Kristin P. Bennett, Ayhan Demiriz, eds. Proc. of the 1998 Conference on Advances in Neural Information Processing Systems, MIT Press, Cambridge, MA, USA, 1998: 368-374.
[34] Maryanne Domm, Andrew Engel, Peguy Pierre-Louis. An Integer Support Vector Machine[C]. The sixth International Conference on SNPD/SAWN, Hainan, China, 2005.
[35] Scholkopf B, Platt J., Shawe-Taylor J. Estimating the Support of a high-dimensional distribution[J]. Neural Computation, 2001, 13(7): 1443-1471.
[36] Q. Zhao, J. C. Pineiple, Support vector machines for SAR automatic larger recognition[J]. IEEE Trans. AES, 2001, 37(2): 643-654
[37] J.U,J. C. Principe. Statistical pattern recognition for SAR/ATR[J]. USAFRL Technical Report, July, 2001
[38] R.E. Karisen, el. al. Target classification via support vector machines[J]. Optical Engineering, 2000, 39(3): 704-711
[39]宋晓宁,束鑫.一种改进的模糊SVM的人脸识别方法[J].微机发展, 2005, 15(3): 23-25,28.
[40]熊浩勇,基于SVM的中文文本分类算法研究与实现[D].武汉理工大学硕士学位论文,2008.
[41]谢秋玲.应用于心电图分类的KNN和SVM分类器研究[D].华东师范大学硕士学位论文, 2004.
[42]李雅琴. SVM在手写数字识别中的应用研究[D].华中师范大学硕士学位论文, 2007.
[43]马君国,赵宏钟,王微.基于最优参数支持向量机的目标识别算法[J]. IEEE, 2009.
[44]马永军,李孝忠,王希雷.基于模糊SVM和核方法的目标检测方法研究[J].天津科技大学学报,2005, 20(3): 29-32.
[45]李亚楠,耿伯英,杨武.基于SVM和辐射噪声的舰船目标识别[J].舰船电子工程,2006, 21(1): 58-60.
[46]万力,盘善荣,傅明.基于SVM的P2P流量识别[J].计算机技术与自动化,2009(3), Vol.28, No.1: 112-115.
[47]白鹏,张喜斌,张斌等.支持向量机理论及工程应用实例.西安:西安电子科技大学出版社,2008.
[48] Kehai Xin,Xuegong Zhang.Editing support vector machines[C]. Proc. of International Joint Conference on Neural Networks, Washington, USA, 2001(2): 1464-1467.
[49]李红莲,王春花,袁保宗.一种改进的支持向量机NNSVM[J].计算机学报,2003, 8.
[50] Chew Hong-Gunn, Bogner R.E., Lim Cheng-Chew. Dual-Support Vector Machine with Error Rter and Training Size Biasing[C]. Proc. of Acoustics, Speech, and Signal Processing, SaltLake City, Utah, USA, 2001.
[51]郭雷,肖怀铁,付强.非均衡数据目标识别中的SVM模型多参数优化选择方法[J].红外与毫米波学报,2009(4), Vol.28. No.2.
[52] Raskutti B., Kowalczyk A. Extreme rebalancing for SVMs: a cage study[J]. News letter of the ACM Special Interest Group on Knowledge Discovery and Data Mining, 2004, 6(1): 61-69.
[53] Lee Y., Lin Y., Wahba G. Multicategory support vector machines: theory and application to the classification of microarray data and satellite radiance data[R]. Wisconsin: University of Wisconsin, 2002.
[54] Rosenblatt F. Principles of Neurodinamics: Perceptron and Theory of Brain Mechanisms[M]. Spartan Books, Washington D.C., 1962.
[55]李国正等译.支持向量机导论[M](Cristianini N. Shawe-Taylor J. An Introduction to Support Vector Machines and Other Kernel-based Learning Methods).北京:电子工业出版社,2004, 3.
[56]丁爱玲,刘芳,姚霞.基于支撑矢量机的智能目标识别方法[J].西安电子科技大学学报(自然科学版),2001, Vol . 28, No. 6: 743-746.
[57]张莉,周伟达,焦李成.用于一维图像识别的支撑矢量机方法[J].红外与毫米波学报,2002, 4, Vol. 21, No. 2: 119-123
[58] Vapnik V. The Nature of Statistical Learning Theory [M]. New York, USA: Springer, 1995.
[59] LeCun Y, et a1. Backpropagation applied to handwritten zip code recognition[J]. Neural Comput., 1999, l: 541-551.
[60] Kubat M., Matwin S. Addressing the Course of Imbalanced Training Sets: One-Sided Selection[C]. International Conference on Machine Learning, 1997: 179-186.
[61]李京华,张聪颖,倪宁.基于参数优化的支持向量机战场多目标声识别[J].探测与控制学报,2010, Vol.32, No.1: 1-5.
[62] Chawla N.V., Lazarevic A, Hall LO., Bowyer K. SMOTEBoost:Improving prediction of the Minority Class in Boosting[C]. 7th European Conference on Principles and Practice of Knowledge Discovery in Databases, 2003: 107-119.
[63] Chapelle O., Vapnik V. Choosing Multiple Parameters for Support Vector Machines[J]. Machine Learning, 2002, 46: 131-159.
[64]王克奇,杨少春,戴天虹等.采用遗传算法优化最小二乘支持向量机参数的方法[J].计算机应用与软件,2009(7), Vol.26. No.7: 109-111.
[65] Yang Hong, Lou Fei, Xu Yuge, Liang Jin. GA Based LS-SVM Classifier for Waste Water Treatment Process[C]. Proc. of the 27th Chinese Control Conference, 2008(7): 436-439.
[66]张培林,钱林方,曹建军等.基于蚁群算法的支持向量机参数优化[J].南京理工大学学报(自然科学版),2009, Vol.33, No.4: 464-468.
[67]王东霞,张楠,路晓丽.基于育种算法的SVM参数优化[J].安徽大学学报(自然科学版),2009, Vol.33, No.4: 26-28.
[68] Adankon M. M., Cheriet M. Model Selection for the LS-SVM. Application to Handwriting Recognition[J]. Pattern Recognition, 2009, 42: 3264-3270.
[69] Suykens J.A.K, Vandewalle J. Least Squares Support Vector Machine Classifiers[J]. Neural Processing Letters(S1573-773X), 1999, 9(3): 293-300.
[70]刘东辉,卞建鹏,付平等.支持向量机最优参数选择的研究[J].河北科技大学学报,2009(3), Vol.30, No.1: 58-61.
[71]史忠植.高级人工智能(第二版)[M].北京:科学出版社,2006.
[72]王雪松,程玉虎,等.机器学习理论、方法及应用[M].北京:科学出版社,2009.
[73]周树德,孙增圻.分布估计算法综述[J].自动化学报,2007(2), V0l.33, No.2: 113-124.
[74] WU G. CHANG E. Class-boundary alignment for imbalanced dataset learning[C]. Workshop on Learning from Imbalanced Datasets(ICML 2003), Washington, DC, 2003: 49-56.
[75] Batuwita R, Palade V. FSVM-CIL: Fuzzy Support Vector Machines for Class Imbalance Learning[J]. IEEE Trans. on Fuzzy Systems, 2010(6),Vol. 18, No. 3: 558-571.
[76] Imam T, Ming Ting K, Kamruzzaman J. z-SVM: An SVM for Improved Classification of Imbalanced Data[C]. AI 2006, LNAI 4304: 264-273.
[77]薛贞霞,刘三阳,刘万里.不平衡最小二乘支持向量机[J].系统仿真学报, 2009(7), Vol. 21, No. 14: 4324-4327.
[78] Chawla N, Bowyer K, Hall L, et a1. SMOTE: synthetic minorityover-sampling technique[J]. Journal of Artificial Intelligence Research, 2002, 16: 321-357.
[79] Zhi-Qiang Zeng, Ji Gao. Improving SVM Classification with Imbalance Data Set[C]. ICONII 2009, LNCS 5863: 389-398.
[80] Laurikkala J. Improving Identification of Difficult Small Classes by Balancing Class Distribution[C]. Proc of the 8th Conference on AI in Medicine, Europe: Artificial Intelligence Medicine, 2001: 63-66.
[81] Show-Jane Yen, Yue-Shi Lee. Cluster-based Under-sampling Approaches for Imbalanced Data Distributions[J]. Expert Systems with Applications, 2009, 36: 5718-5727.
[82] KUBAT M, MATWIN S. Addressing the curse of imbalanced training sets:one-sided selection[C]. Proc. of the 14th International Conference on Machine Learning. San Francisco, CA: Morgan Kaufmann 1997: 217-225.