支持向量机和支持向量域描述的若干问题研究
|
摘要
支持向量机(Support Vector Machine, SVM)是基于统计学习理论提出的一种新型有监督模式识别方法.采用结构风险最小化原则,SVM在最大化分类间隔和最小化分类误差之间取得折衷,以控制分类器的泛化能力.SVM较好地解决了小样本、高维数及非线性等实际问题,具有拟合精度高、选择参数少、推广能力强和全局最优等特点.SVM成为机器学习领域新的研究热点,并被广泛应用到模式识别、函数拟合和密度估计等领域.本文针对大规模样本集的SVM训练问题、SVM的集成学习问题、变形SVM问题、支持向量域描述(Support Vector Domain Description,SVDD)的快速训练问题等进行研究,主要研究工作如下:
1.研究了大规模样本集的SVM训练算法.SVM在训练大规模样本集时面临着占用内存多、计算代价大的问题,这也成为实际应用的瓶颈问题.基于并行学习中“分而治之”的理念和“支持向量与全体训练样本等价”的结论,提出一种同心超球面支持向量机(Hypersphere Support Vector Machine, HSVM).HSVM以相同层数的两组同心超球面组对正负两类样本进行分割,对分层间隔内的样本采用SVM训练,合并所有分层间隔中支持向量的并集以参与最终的SVM训练.HSVM既保持了SVM的分类精度又降低了SVM的训练时间.
2.研究了SVM的集成学习算法.运用集成学习的理念构造了一种空间支持向量域分类器(Space Support Vector Domain Classifier,SSVDC).选取支持向量域分类器(Support Vector Domain Classifier,SVDC)和K近邻(K Nearest Neighbor,KNN)作为子分类器,SSVDC以选择集成的策略对两者的预测结果进行集成.SSVDC首先采用SVDD求得两类样本的最小包围超球,并以超球的描述边界将训练样本划分为互不相交的几个区域;其次计算待测样本到两个最小包围超球球心的距离,根据其与两个最小包围超球半径的大小关系判断待测样本所处区域;最后选择相应的子分类器得到最终的分类结果.由于子分类器均针对样本的某个子集进行训练,SSVDC具有较短的训练时间.由于根据样本的分布选取相应的子分类器,SSVDC具有较高的分类精度且其分类精度受核参数变化的影响不大.数值实验验证了SSVDC的有效性以及对比SVM和SVDC的优越性.
3.研究了变形SVM算法.通过改变原始优化问题的函数项、变量或系数得到了一系列变形SVM算法,这在一定程度拓宽了SVM的应用范围.针对变形SVM中的二次损失函数SVM,构造一种光滑对角加权支持向量机(Smooth Diagonal Weighted Support Vector Machine,SDWSVM).在线性空间中,直接运用光滑技术得到光滑模型,也即采取Sigmoid函数的积分函数来逼近正号函数形式的松弛.在非线性空间中,先分别利用Lagrange乘子向量来代替分类超平面的权值向量和原-对偶规划隐含的分类超平面的权值向量表达式对原规划的目标函数进行转换,再利用光滑技术构造光滑模型.对线性空间和非线性空间中得到的光滑模型,均采用Newton法求解,具有较高的效率.
4.提出了一种约简支持向量域描述算法(Reduced Support Vector Domain Description,RSVDD).SVDD的训练即为求解一个所含未知数个数等于全体训练样本个数的凸二次规划.为了提高SVDD的训练速度,RSVDD对每个样本定义一种自中心距离比值,定义此值为该样本到中心的距离与所有样本的平均中心距离的比值,并以此值作为判断该样本成为支持向量的可能性度量.RSVDD选取自中心距离比值较大的部分样本参与SVDD训练,从而减少了待解QP的规模.该算法引入参数少,易于实现且保持了目标类精度.
5.提出了一种信赖支持向量域描述算法(Confidence Support Vector Domain Description,CSVDD).由于支持向量往往分布在SVDD的描述边界附近,基于这个几何特性,定义了一种信赖抽样方法以提取部分样本参与SVDD训练.依次以每个样本为中心,做一个以某定长为半径的超球.统计落入该超球中的训练样本数目,并将该值作为判断中心样本属于支持向量的信赖度量(Confidence Measure).根据信赖度量的值将训练样本升序排列,CSVDD提取排在前面的部分样本作为边界向量集参与最终的SVDD训练,缩短了SVDD的训练时间并保持了SVDD的目标类精度.
Support Vector Machine (SVM) is a new supervised pattern recognition method based on Statistical Learning Theory (SLT). SVM adopts Structural Risk Minimization (SRM) principle; it makes compromise between the margin maximization and the error minimization, so as to control the generalization ability of the obtained classifier. SVM can better deal with problems, such as small sample, high dimension and nonlinearity. SVM has advantages such as high classification accuracies, few parameters, global optimums and strong generalization performances; it becomes a new research area in the field of machine learning research and it has been widely applied into various areas, such as pattern recognition, function regression and density estimation. Focusing on the existing problems, such as SVM training for large-scale samples, SVM ensemble learning, reformation of SVM, Support Vector Domain Description (SVDD), etc, detailed studies are made and all the researches can be summarized into the following five parts:
1.The research on the algorithms of SVM training for large-scale samples. The training of SVM becomes difficult in practical applications for large-scale samples; it takes up large memory and enormous computation time. Based on the ideology of“divide and conquer”in parallel learning and the conclustion of“the equivalence between support vectors and the whole training samples”, concentric hypersphere support vector machine is proposed and is called HSVM for short. Dividing the positive and negative samples using groups of concentric hyperspheres with the same layers, HSVM trains the samples between the margins using SVM and takes the combination of the obtained support vectors in corresponding margins to participate in the final SVM training. HSVM guarantees the classification accuracies of SVM while it reduces the training time of SVM.
2.The research on ensemble learning of SVM. Ensemble learning ideology is utilized to construct a Space Support Vector Domain Classifier (SSVDC). Taking SVDC (Support Vector Domain Classifier) and KNN (K Nearest Neighbor) as the sub classifiers, SSVDC uses selective ensemble strategy to obtain the final classification results. Firstly, SSVDC uses SVDD to obtain the minimal enclosing hyperspheres of the positive and negative samples, and divides the training samples into several disconnected regions using boundaries of the two hyperspheres; then SSVDC calculates the distances of one test sample to the centers of the two minimal enclosing spheres of the positive and negative samples, and specifies the region that the test sample belongs to based on the relationship between the distances and the radiuses of the two minimal enclosing spheres; finally SSVDC takes the corresponding sub classifier to judge its label. Since sub classifiers are carried out on subsets of the data, SSVDC has short training times. Since different sub classifiers are erected based on the sample distribution, SSVDC has high target accuracies which vary little with kernel parameters. Numerical experiments demonstrate the effectiveness of SSVDC and the superiority over SVM and SVDC.
3.The research on reformation of SVM. Reformation of SVM broadens the application areas of SVM, which can be obtained by changing the terms, variables or coefficients in the objective function in the original optimization problem. Based on quadratic loss function SVM, smoothing technique is adopted to construct Smooth Diagonal Weighted Support Vector Machine (SDWSVM). In the linear space, smooth model is directly obtained by using smoothing technique, which takes the integral function of sigmoid function to approximate the slack in the plus function form. In the kernel space, two methods are used to transform the original objective function before applying smoothing technique. One uses the method adopted by traditional SSVM which takes the Lagrange multiplier vector to substitute the weight vector of the separating hyperplane; the other uses dual technique between the primal and dual program to find the expression of the weight vector of the separating hyperplane. For the obtained smooth models in the linear and kernel space, Newton method is proposed to find the global optimums which guarantees high efficiency.
4.Reduced Support Vector Domain Description (RSVDD) is presented. The training of SVDD involves solving a convex quadratic program, whose number of variables is equal to the number of training samples. To accelerate the training speed, RSVDD defines a self central distance ratio for each sample, which is the ratio of the central distance divided by the mean central distance, and takes it as a probability measure to judge whether one sample is a support vector or not. RSVDD selects some samples with high self central distance ratios to participate in the final SVDD training, thus reducing the training scale. RSVDD is easy to implement, has few parameters and maintains high target accuracies and short training time.
5.Confidence Support Vector Domain Description (CSVDD) is proposed. Based on the geometry characteristic that support vectors usually distribute around the description boundary, a confidence sampling strategy is proposed so as to select some samples to participate in the SVDD training. Centered at each sample, a sphere with fixed radius is drawn. Counting the number of samples in the user-defined sphere, CSVDD takes this number as a confidence measure to judge whether the center sample is one support vector or not. Ranking the training samples in ascending order according to the confidence measure, CSVDD extracts a certain portion of those former ranked samples as boundary vectors to participate in the SVDD training. The training time of CSVDD are shorter than that of SVDD, while the target accuracies of the former are identical with those of the latter.
1.研究了大规模样本集的SVM训练算法.SVM在训练大规模样本集时面临着占用内存多、计算代价大的问题,这也成为实际应用的瓶颈问题.基于并行学习中“分而治之”的理念和“支持向量与全体训练样本等价”的结论,提出一种同心超球面支持向量机(Hypersphere Support Vector Machine, HSVM).HSVM以相同层数的两组同心超球面组对正负两类样本进行分割,对分层间隔内的样本采用SVM训练,合并所有分层间隔中支持向量的并集以参与最终的SVM训练.HSVM既保持了SVM的分类精度又降低了SVM的训练时间.
2.研究了SVM的集成学习算法.运用集成学习的理念构造了一种空间支持向量域分类器(Space Support Vector Domain Classifier,SSVDC).选取支持向量域分类器(Support Vector Domain Classifier,SVDC)和K近邻(K Nearest Neighbor,KNN)作为子分类器,SSVDC以选择集成的策略对两者的预测结果进行集成.SSVDC首先采用SVDD求得两类样本的最小包围超球,并以超球的描述边界将训练样本划分为互不相交的几个区域;其次计算待测样本到两个最小包围超球球心的距离,根据其与两个最小包围超球半径的大小关系判断待测样本所处区域;最后选择相应的子分类器得到最终的分类结果.由于子分类器均针对样本的某个子集进行训练,SSVDC具有较短的训练时间.由于根据样本的分布选取相应的子分类器,SSVDC具有较高的分类精度且其分类精度受核参数变化的影响不大.数值实验验证了SSVDC的有效性以及对比SVM和SVDC的优越性.
3.研究了变形SVM算法.通过改变原始优化问题的函数项、变量或系数得到了一系列变形SVM算法,这在一定程度拓宽了SVM的应用范围.针对变形SVM中的二次损失函数SVM,构造一种光滑对角加权支持向量机(Smooth Diagonal Weighted Support Vector Machine,SDWSVM).在线性空间中,直接运用光滑技术得到光滑模型,也即采取Sigmoid函数的积分函数来逼近正号函数形式的松弛.在非线性空间中,先分别利用Lagrange乘子向量来代替分类超平面的权值向量和原-对偶规划隐含的分类超平面的权值向量表达式对原规划的目标函数进行转换,再利用光滑技术构造光滑模型.对线性空间和非线性空间中得到的光滑模型,均采用Newton法求解,具有较高的效率.
4.提出了一种约简支持向量域描述算法(Reduced Support Vector Domain Description,RSVDD).SVDD的训练即为求解一个所含未知数个数等于全体训练样本个数的凸二次规划.为了提高SVDD的训练速度,RSVDD对每个样本定义一种自中心距离比值,定义此值为该样本到中心的距离与所有样本的平均中心距离的比值,并以此值作为判断该样本成为支持向量的可能性度量.RSVDD选取自中心距离比值较大的部分样本参与SVDD训练,从而减少了待解QP的规模.该算法引入参数少,易于实现且保持了目标类精度.
5.提出了一种信赖支持向量域描述算法(Confidence Support Vector Domain Description,CSVDD).由于支持向量往往分布在SVDD的描述边界附近,基于这个几何特性,定义了一种信赖抽样方法以提取部分样本参与SVDD训练.依次以每个样本为中心,做一个以某定长为半径的超球.统计落入该超球中的训练样本数目,并将该值作为判断中心样本属于支持向量的信赖度量(Confidence Measure).根据信赖度量的值将训练样本升序排列,CSVDD提取排在前面的部分样本作为边界向量集参与最终的SVDD训练,缩短了SVDD的训练时间并保持了SVDD的目标类精度.
Support Vector Machine (SVM) is a new supervised pattern recognition method based on Statistical Learning Theory (SLT). SVM adopts Structural Risk Minimization (SRM) principle; it makes compromise between the margin maximization and the error minimization, so as to control the generalization ability of the obtained classifier. SVM can better deal with problems, such as small sample, high dimension and nonlinearity. SVM has advantages such as high classification accuracies, few parameters, global optimums and strong generalization performances; it becomes a new research area in the field of machine learning research and it has been widely applied into various areas, such as pattern recognition, function regression and density estimation. Focusing on the existing problems, such as SVM training for large-scale samples, SVM ensemble learning, reformation of SVM, Support Vector Domain Description (SVDD), etc, detailed studies are made and all the researches can be summarized into the following five parts:
1.The research on the algorithms of SVM training for large-scale samples. The training of SVM becomes difficult in practical applications for large-scale samples; it takes up large memory and enormous computation time. Based on the ideology of“divide and conquer”in parallel learning and the conclustion of“the equivalence between support vectors and the whole training samples”, concentric hypersphere support vector machine is proposed and is called HSVM for short. Dividing the positive and negative samples using groups of concentric hyperspheres with the same layers, HSVM trains the samples between the margins using SVM and takes the combination of the obtained support vectors in corresponding margins to participate in the final SVM training. HSVM guarantees the classification accuracies of SVM while it reduces the training time of SVM.
2.The research on ensemble learning of SVM. Ensemble learning ideology is utilized to construct a Space Support Vector Domain Classifier (SSVDC). Taking SVDC (Support Vector Domain Classifier) and KNN (K Nearest Neighbor) as the sub classifiers, SSVDC uses selective ensemble strategy to obtain the final classification results. Firstly, SSVDC uses SVDD to obtain the minimal enclosing hyperspheres of the positive and negative samples, and divides the training samples into several disconnected regions using boundaries of the two hyperspheres; then SSVDC calculates the distances of one test sample to the centers of the two minimal enclosing spheres of the positive and negative samples, and specifies the region that the test sample belongs to based on the relationship between the distances and the radiuses of the two minimal enclosing spheres; finally SSVDC takes the corresponding sub classifier to judge its label. Since sub classifiers are carried out on subsets of the data, SSVDC has short training times. Since different sub classifiers are erected based on the sample distribution, SSVDC has high target accuracies which vary little with kernel parameters. Numerical experiments demonstrate the effectiveness of SSVDC and the superiority over SVM and SVDC.
3.The research on reformation of SVM. Reformation of SVM broadens the application areas of SVM, which can be obtained by changing the terms, variables or coefficients in the objective function in the original optimization problem. Based on quadratic loss function SVM, smoothing technique is adopted to construct Smooth Diagonal Weighted Support Vector Machine (SDWSVM). In the linear space, smooth model is directly obtained by using smoothing technique, which takes the integral function of sigmoid function to approximate the slack in the plus function form. In the kernel space, two methods are used to transform the original objective function before applying smoothing technique. One uses the method adopted by traditional SSVM which takes the Lagrange multiplier vector to substitute the weight vector of the separating hyperplane; the other uses dual technique between the primal and dual program to find the expression of the weight vector of the separating hyperplane. For the obtained smooth models in the linear and kernel space, Newton method is proposed to find the global optimums which guarantees high efficiency.
4.Reduced Support Vector Domain Description (RSVDD) is presented. The training of SVDD involves solving a convex quadratic program, whose number of variables is equal to the number of training samples. To accelerate the training speed, RSVDD defines a self central distance ratio for each sample, which is the ratio of the central distance divided by the mean central distance, and takes it as a probability measure to judge whether one sample is a support vector or not. RSVDD selects some samples with high self central distance ratios to participate in the final SVDD training, thus reducing the training scale. RSVDD is easy to implement, has few parameters and maintains high target accuracies and short training time.
5.Confidence Support Vector Domain Description (CSVDD) is proposed. Based on the geometry characteristic that support vectors usually distribute around the description boundary, a confidence sampling strategy is proposed so as to select some samples to participate in the SVDD training. Centered at each sample, a sphere with fixed radius is drawn. Counting the number of samples in the user-defined sphere, CSVDD takes this number as a confidence measure to judge whether the center sample is one support vector or not. Ranking the training samples in ascending order according to the confidence measure, CSVDD extracts a certain portion of those former ranked samples as boundary vectors to participate in the SVDD training. The training time of CSVDD are shorter than that of SVDD, while the target accuracies of the former are identical with those of the latter.
引文
[1] V. N. Vapnik. The nature of statistical learning theory. Springer: New York, 1995.
[2] F. Mulier. Vapnik-Chervonenkis (VC) learning theory and its applications. IEEE Transaction on Neural Networks, 1999, 10(5):985-987.
[3] F. Rosenblatt. Principles of neurodynamics: perceptrons and the theory of brain mechanisms. Spartans Books: Washington D.C., 1962.
[4] A. Novikoff. On convergence proofs for perceptrons. In Proceedings of the Symposium on the Mathematical Theory of Automata, 1962, 12:615-622.
[5] A. N. Kolmogorov. Three approaches to the quantitative definition of information. Problem of Information Transmission, 1965, 1(1):1-7.
[6] G. J. Chaitin. On the length of programs for computing finite binary sequences. Journal of Association with Computer Machine, 1966, 13(4): 547-569.
[7] Y. L. Cun. Learning process in an asymmetric threshold network. NATO ASI series. Series F: Computer and System Sciences, 1986, 20:233-240.
[8] D. E. Rumelhart, G. E. Hinton, R. J. Williams. Learning internal representations by error propagation. In Parallel Distributed Processing: Explorations in The Microstructure of Cognition, 1986, 1:318-362.
[9]高隽.人工神经网络原理及仿真实例.北京:机械工业出版社,2003.
[10]张学工译.统计学习理论的本质.北京:清华大学出版社,2000.
[11] V. N. Vapnik. An overview of statistical learning theory. IEEE Transaction on Neural Networks, 1999, 10(5): 988-999.
[12] V. N. Vapnik. Three fundamental concepts of the capacity of learning machines. Physica A, 1993, 200: 538-544.
[13]邓乃杨,田英杰.数据挖掘中的新方法-支持向量机.北京:科学出版社,2005.
[14] C. J. C. Burges. A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery, 1998, 2(2): 121-167.
[15] A. J. Smola, B. Sch?lkopf. A tutorial on support vector regression. Statistics and Computing, 2004, 14(3): 199-222.
[16] R. Mohamed, A. El-Baz, A. Farag. Probability density estimation using advanced support vector machines and the expectation maximization algorithm. Proceedings of World Academy of Science, Engineering and Technology, 2005, 2: 137-140.
[17]王国胜,钟义信.支持向量机的若干新进展.电子学报,2001,29(10): 1397-1400.
[18] S. Amari, S. Wu. Improving support machine classifier by modifying kernel function. Neural Networks, 1999, 12(6): 783-789.
[19] H. Lodhi, C. Saunders, J. Shawe-Taylor, et al. Text classification using string kernels. Journal of Machine Learning Research, 2002, 2(2): 419-444.
[20] N. Cancedda, E. Gaussier, C. Goutte, et al. Word sequence kernels. Journal of Machine Learning Research, 2003, 3(6): 1059-1082.
[21] G. Camps-Valls, L. Geomez-Chova, J. Vila-Frances, et al. Composite kernels for hyperspectral image classification. IEEE Geosciences and Remote Sensing Letters, 2006, 3(1): 93-98.
[22] K. Duan, S. S. Keerthi, A. N. Poo. Evaluation of simple performance measures for tuning SVM hyperparameters. Neurocomputing, 2003, 51: 41-59.
[23] G. Wahba, Y. Lin, H. Zhang, et al. Generalized approximate cross-validation for support vector machines: another way to look at margin-like quantities. Advances in Large Margin Classifier, Cambridge, MA: MIT Press, 2000, 297-309.
[24] A. Lunts, V. Brailovskiy. Evaluation of attributes obtained in statistical decision rules. Engineering Cybernetics, 1967, 3(1): 98-109.
[25] V. N. Vapnik, O. Chapell. Bounds on error expectation for support machine. Neural Computation, 2000, 12(9): 2013-2036.
[26] T. S. Joaakkola, D. Haussler. Probabilistic kernel regression models. In Proceedings of the 7th workshop on AI and Statistics, San Francisco, 1999, 26-34.
[27] M. Opper, O. Winther. Gaussian processes for classification: mean-field algoithms. Neural Computation, 2000, 12(11): 2655-2684.
[28] B. Sch?lkopf and A. Smola. Learning with Kernel. MA: MIT press, 2002.
[29] D. J. C. Machay. Probable networks and plausible predictions-a review of practical bayesian methods for supervised neural networks. Network Computation in Neural Systems, 1995, 6(3): 469-505.
[30] J. T. Kowk. The evidence framework applied to support vector machines. IEEE Transactions on Neural Network, 2000, 11(5): 1162-1173.
[31] B. Sch?lkopf, A. Smola, R. Williamson, et al. New support vector algorithms. Neural Computation, 2000, 12(5): 1207-1245.
[32] D. M. J. Tax, R. P. W. Duin. Support vector domain description. Pattern Recognition Letters, 1999, 20(11-13): 1191-1199.
[33] J. A. K. Suykens, T. Van Gestel, J. De Brabanter, et al. Least squares support vector machines. World Scientific: Singapore, 2002.
[34] T. V. Gestel, J. A. K. Suykens, B. Baesens, et al. Benchmarking least squares support vector machine classifiers. Machine Learning, 2004, 54(1): 5-32.
[35] Danny Roobaert. DirectSVM: a fast and simple support vector machine perceptron. Journal of VLSI Signal Processing Systems, 2002, 32(1-2): 147-156.
[36] V. Kecman, L. Hadzic. Support vectors selection by linear programming. Proceedings of the International Joint Conference on Neural Networks (IJCNN’2000), Como, 2000, 5: 193-198.
[37] L. Hadzic, V. Kecman. Support vector machines trained by linear programming: theory and application in image compression and data classification. The Fifth IEEE Seminar on Neural Network Applications inElectrical Engineering, Belgrade, 2000: 18-23.
[38] Zhou Wei-da, Zhang Li, Jiao Li-cheng. Linear programming support vector machines. Pattern Recognition, 2002, 35(12): 2927-2936.
[39] O. L. Mangasarian. Generalized support vector machines. Advances in Large Margin Classifiers, MA: MIT Press, 1998.
[40] O. L. Mangasarian, D. R. Musicant. Data discrimination via nonlinear generalized support vector machines. Complementarity: Applications, Agorithms and Extensions, Kluwer academic publishers, 1999: 233-251.
[41] O. L. Mangasarian, D. R. Musicant. Successive over relaxation for support vector machines. IEEE Transactions on Neural Networks, 1999, 10(5): 1032-1037.
[42] D. R. Musicant, O. L. Mangasarian. LSVM: Lagrangian support vector machine. Journal of Machine Learning Research, 2001, 1: 161-177.
[43] O. L. Mangasarian, D. R. Musicant. Active set support vector machine classification. Advances in Neural Information Processing Systems 2000, MA: MIT Press, 2001: 577-583.
[44] D. M. J. Tax, R.P.W. Duin. Support vector data description. Machine Learning, 2004, 54(1): 45-66.
[45] J. Munoz-Mari, L. Bruzzone, G. Camps-Valls. A support vector domain description approach to supervised classification of remote sensing images. IEEE Transaction on Geosciences and Remote Sensing, 2007, 45(8): 2683-2692.
[46] Sang-Woong Lee, Seong-Whang Lee. SVDD based illumination compensation for face recognition. International Joint Conference on Advances in Biometrics, 2007, Korea.
[47] Chao Yuan, David Casasent. A novel support vector classifier with better rejection performance. 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003, 1: 419-424.
[48] J. Park, D. Kang, J. Kim, et al. SVDD-based pattern denoising. Neural Computation, 2007, 19(7): 1919-1938.
[49]梁锦锦,刘三阳,吴德.空间支持向量域分类器.西安电子科技大学学报,2008,35(6):1080-1083转1088.
[50]陆从德,张太镒,胡金艳.基于乘性规则的支持向量域分类器.计算机学报,2004,27(5):690-694.
[51] D. M. J. Tax, R. P. W. Duin. Uniform object generation for optimizing one-class classifiers. Journal of Machine Learning Research, 2001, 2: 155-173.
[52] K.Crammer, Y. Singer. On the learnability and design of output codes for multiclass problems. Machine Learning, 2002, 47(2-3): 201-233.
[53] T. G. Dietterich, G. Bakiri. Solving multiclass learning problems via error-correcting output codes. Journal of Artificial Intelligence Research, 1995, 2: 263-286.
[54] C.W. Hsu, C. J. Lin. A comparison of methods for multiclass support vector machines. IEEE transaction on Neural Networks, 2002, 13(2): 415-425.
[55] J. Cervantes, L. Xiaoou, Y. Wen, et al. Multi-class support vector machines for large data sets via minimum enclosing ball clustering. IEEE 4th International Conference on Electrical and Electronics Engineering, Mexico, 2007: 146-149.
[56] P. Lingras, C. J. Butz. Rough set based 1-v-1 and 1-v-r approaches to support vector machine multi-classification. Information Sciences, 2007, 177(18): 3782-3798.
[57] J. C. Platt, N. Cristianini, J. Shawe-Taylor. Large margin DAGs for multiclass classification. Advances in Neural Information Processing System, 2000: 547-553.
[58] C. Cortes, V. N. Vapnik. Support vector networks. Machine Learning, 1995, 20 (3): 273-297.
[59] E. Osuna, R. Freund, F. Girosi. An improved training algorithm for support vector machines. Proceedings of the 1997 IEEE workshop on Neural Networks for signal processing, New York: IEEE press, 1997: 276-285.
[60] T. Joachims. Making large-scale SVM learning practical. Advances in Kernel Methods: Support Vector Learning. Cambridge, MA: MIT press, 1999: 169-184.
[61] C. J. Lin. On the convergence of the decomposition method for support vector machines. IEEE Transactions on Neural Networks, 2001, 12(6): 1288-1298.
[62] C. W. Hsu, C. J. Lin. A simple decomposition method for support vector machines. Machine learning, 2002, 46(1): 291-314.
[63] J. C. Platt. Fast training of support vector machines using sequential minimal optimization. Advances in Kernel Methods: Support Vector Learning, Cambridge, MA: MIT press, 1999, 185-208.
[64] Chih-Jen Lin. Asymptotic convergence of an SMO algorithm without any assumptions. IEEE transactions on Neural Networks, 2002, 13(1): 248-250.
[65] S. S. Keerthi, S. K. Shevade, C. Bhattacharyya, et al. Improvements to Platt’s SMO algorithm for SVM classifier design. Neural Computation, 2001, 13(3): 637-649.
[66] G. Cauwenberghs, T. Poggio. Incremental and decremental support vector machine learning. Advances in Neural Information Processing Systems, Cambridge, MA: MIT press, 2001, 13: 409-415.
[67]箫嵘,王继成,孙正兴等.一种SVM增量学习算法α-ISVM.软件学报,2001,12(12):1818-1824.
[68] J. X. Dong, A. Kyzyzak, C. Y. Suen. A fast parallel optimization for training support vector machine. Proceedings of 3rd International Conference on Machine Learning and Data Mining, 2003, 2734: 96-105.
[69] H. P. Graf, E. Cosato, L. Botou, et al. Parallel support vector machines: the cascade SVM. Advances in Neural Information Processing Systems, Cambridge, MA: MIT Press, 2004: 521-528.
[70] R. Collobert, S. Bengio, Y. Bengio. A parallel mixture of SVMs for Large scale problems. Neural Computation, 2002, 14(5): 1105-1114.
[71] G. Zanghirati, L. Zanni. A parallel solver for large quadratic programs intraining support vector machine. Parallel computing, 2003, 29(4):535-551.
[72] L. J. Cao, S. S. Keerthi, C. J. Ong, et al. Parallel sequential minimal optimization for the training of support vector machines. IEEE transaction on neural networks, 2006, 17(4): 1039-1049.
[73] G. Valentini, T. G. Dietterich. Bias-variance analysis of support vector machines for the development of SVM based ensemble methods. Journal of Machine Learning Research, 2004, 5: 725-775.
[74] B. Robert, G. Ricardo, Q. Francis. Attribute bagging: Improving accuracy of classifier ensembles by using random feature subsets. Pattern recognition, 2003, 36(6): 1291-1302.
[75] Kim H., Pang S., Je H., et al. Constructing support vector machine ensemble. Pattern recognition, 2003, 36(12): 2757-2767.
[76] X. C. Li, L. Wang, E. Sung. A study of AdaBoost with SVM weak learners. International Joint Conference on Neural Networks, Montreal, Canada, 2005, 1: 196-201.
[77] Y. S. Dong, Han K. S. A comparison of several ensemble methods for text categorization. 2004 IEEE International Conference on Services Computing, Shanghai, China, 2004: 419-422.
[78] D. C Tao., X. O. Tang, X. L. Li, et al. Asymmetric bagging and random subspace for support vector machines-based relevance feedback in image retrieval. IEEE Transaction on Pattern Analysis and Machine Intelligence, 2006, 28(7): 1088-1099.
[79]焦李成,张莉,周伟达.支持向量预选取的中心距离比值法.电子学报,2001,29(3):383-386.
[80] M. H. Yang, N. Ahuja. A geometric approach to training support vector machines. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Hilton Head Island, USA, 2000, 1: 430-437.
[81] M. Rychetsky, S. Ortmann, M. Ullmann, et al. Accelerated training of support vector machines. International Joint Conference on Neural Networks, Washington, USA, 1999, 2: 998-1003.
[82]李青,焦李成,周伟达.基于向量投影的支持向量预选取.计算机学报,2005,28(2):145-152.
[83] H. Shin, S. Cho. Fast pattern selection for support vector classifiers. Lecture Notes in Computer Science, Springer, 2003, 2637: 376-387.
[84] H. Yu, J. Yang, J. Han. Classifying large data sets using SVM hierarchical clusters. Proceedings of the Ninth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Washington, USA, 2003, 306-315.
[85] Zhang Li, Zhou Weida, Jiao Licheng. Pre-extracting support vectors for support vector machines. Signal Processing Proceedings, Beijing, China, 2000, 3: 1432-1435.
[86] C. F. Lin, S. D. Wang. Fuzzy support vector machines. IEEE Transaction on Neural Network, 2002, 13(3): 466-471.
[87] H. P. Huang, Y. H. Liu. Fuzzy support vector machines for patternrecognition and data mining. International Journal of Fuzzy Systems, 2002, 4(3): 826-835.
[88]张翔,肖小玲,徐光祐.基于样本之间紧密度的模糊支持向量机方法.软件学报,2006,17(5):951-958.
[89] S. Mika, B. Sch?lkopf, A. Smola, et al. Kernel PCA and de-nosing in feature spaces. Advances in Neural Information Processing Systems, MA: MIT Press, Cambridge, 1999, 11: 536-542.
[90] D. Nguyen, T. Ho. A bottom-up method for simplifying support vector solutions. IEEE Transactions on Neural Networks, 2006, 17(3): 792-796.
[91] H. Shin, S. Cho. Neighborhood property based pattern selection for support vector machines. Neural Computation, 2007, 19(3): 816-855.
[92] Y. F. Shi, Y. P. Zhao. Comparison of text categorization algorithms. Wuhan University Journal of Natural Sciences, 2004, 9(5): 798-804.
[93] D. A. Reynolds, R. C. Rose. Robust text-independent speaker identification using Gaussian mixture speaker models. IEEE Transactions on Speech and Audio Processing, 1995, 3(1): 72-83.
[94] Bernd Heisele, Thomas Serre, Sam Prentice, et al. Hierarchical classification and feature reduction for fast face detection with support vector machines. Pattern Recognition, 2003, 36(9): 2007-2017.
[95] A. David, B. Lerner. Support vector machine-based image classification for genetic syndrome diagnosis. Pattern Recognition Letters, 2005, 26(8): 1029-1038.
[96] M. Felgani, B. Lorenzo. Classification of hyperspectral remote sensing images with support vector machines. IEEE Transactions on Geoscience Remote Sensing, 2004, 42(8): 1778-1790.
[97] G. Camps-Valls, L. Gomez-Chova, J. Calpe-Maravilla, et al. Robust support vector method for hyperspectral data classification and knowledge discovery. IEEE Transactions on Geoscience and Remote Sensing, 2004, 42(7): 1530-1542.
[98] F. Perez-Cruz, O. Bousquet. Kernel methods and their potential use in Signal processing. IEEE Signal Processing Magazine, 2004, 21(3): 57-65.
[99] M. Rychetsky, S. Ortmann, M. Glesner. Support vector approaches for engine knock detection. International Joint Conference on Neural Networks, 1999, 2: 969-974.
[100] P. M. L. Drezet, R. F. Harrison. Support vector machines for system identification. UKACC International Conference on Control, 1998, 1: 688- 692.
[101] B. J. De Kruif, T. J. A. De Vries. On using a support vector machine in learning feed-forward control. International Conference on Advanced Intelligent Mechatronics, Como, Italy, 2001, 1: 272-277.
[102] L. B. Jack, A. K. Nandi. Fault detection using support vector machines and artificial neural networks. Mechanical Systems and Signal Processing, 2002, 16(2-3): 373-390.
[103] M. Ge, R. Du, G. C. Zhang, et al. Fault diagnosis using support vectormachine with an application in sheet metal stamping operations. Mechanical Systems and Signal Processing, 2004, 18(1): 143-159.
[104] F. Mike, R. James. Computer intrusion detection with classification and anomaly detection using SVMs. International Journal of Pattern Recognition and Artificial Intelligence, 2003, 17(3): 441-458.
[105] F. E. H. Tay, L. Cao. Application of support vector machines in financial time series forecasting. Omega, 2001, 29(4): 309-317.
[106] Bhanu Prakash K. N., Ramakrishnan A. G., Suresh S., et al. Fetal lung maturity analysis using ultrasound image features. IEEE Transactions on Information Technology in Biomedicine, 2002, 6(1): 38-45.
[107] Liu Weiqiang, Shen Peihua, Qu Yingge, et al. Fast algorithm of support vector machines in lung cancer diagnosis. Proceedings of International Workshop on Medical Imaging and Augmented Reality, Hong Kong, China, 2001: 188-192.
[108] L. Bao, Z. Sun. Identifying genes related to drug anticancer mechanisms using support vector machine. FEBS Letters, 2002, 521(1-3): 109-114.
[109] Mark G, Ronald J. The current excitement in bioinformatics-analysis of whole-genome expression data: how does it relate to protein structure and function? Current Opinion in Structural Biology, 2000, 10(5): 574-584.
[110]孙德山,吴今培,肖健华. SVR在混沌时间序列预测中的应用.系统仿真学报,2004,16(3):519-524.
[111] L. J. Cao, F. E. H. Tay. Support vector machine with adaptive parameters in financial time series forecasting. IEEE Transactions on Neural Networks, 2003, 14(6):1506-1518.
[112] S. Chen, A. K. Samingan, L. Hanzo. Support vector machine multiuser receiver for DS-CDMA Signals in Multipath Channels. IEEE Transactions on Neural Networks, 2001, 12(3): 604-611.
[113]王磊.支持向量机学习算法的若干问题研究.成都:电子科技大学博士学位论文,2007.
[114] V. N. Vapnik, A. Y. Chervonenkis. On the uniform convergence of relative frequencies of events to their probabilities. Theory of Probability and its Applications, 1971, 16(2): 264-280.
[115] V. N. Vapnik. Statistical learning theory. Wiley: New York, 1998.
[116]袁亚湘,孙文瑜著.最优化理论与方法.北京:科学出版社,1997.
[117] N. Cristianini, J. Shawe-Taylor. An introduction to support vector machines and other kernel-based learning methods. Cambridge: Cambridge University Press, 2000.
[118]薛毅.最优化原理与方法.北京:北京工业大学出版社,2003.
[119]范明,柴玉梅等译.统计学习基础-数据挖掘、推理与预测.北京:电子工业出版社,2004.
[120] C. A. Micchelli. Interpolation of scattered data: distance matrices and conditionally positive definite functions. Constructive Approximation, 1986, 2(1): 11-12.
[121] D. J. C. Mackay. Introduction to Gaussian processes. NATO ASI Series.Series F: Computer and System Sciences, 1998, 168: 133-165.
[122] F. Girosi. An equivalence between sparse approximation and support vector machines. Neural Computation, 1997, 10(6): 1455-1480.
[123] B. Sch?lkopf, C. J. C. Burges, A. J. Smola. Advances in kernel methods support vector learning. Cambridge: MIT Press, 1998.
[124] D. Haussler. Convolution kernels on discrete structures. Technical Report UCSC-CRL-99-10, University of Californian Santa Cruz, Computer Science Department, 1999.
[125] C. Watkins. Dynamic alignment kernels. In: Advances in Large Margin Classifiers, ed. By A. J. Smola, P. L. Bartlett, et al, Cambridge, MA: MIT Press, 2000.
[126] M. Carozza, S. Rampone. Towards an incremental SVM for regression. In Procreedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks, Como, Italy, 2000, 16: 405-410.
[127]孔锐,张冰.一种快速支持向量机增量学习算法.控制与决策,2005,20(10):1129-1132转1136.
[128] Lu Bao-liang, Wang Kai-an, M. Utiyama, et al. A part-verus part method for massively parallel training of support vector machines. 2004 IEEE International Joint Conference on Neural Networks, 2004, 1: 735-740.
[129] Anton Schwaighofer, Vloker Tresp. The bayesian committee support vector machine. Lecture Notes in computer Science, 2001, 2130: 411-417.
[130]文益民,廖洪元,周立华.一种可减少训练时间的分层并行支持向量机方法.南京师范大学学报,2005,5(1):8-11.
[131]许建华,张学工,李衍达.支持向量机的新发展.控制与决策,2004,19(5):481-484.
[132] Matheny ME, Resnic Fs, Arora N, Ohno-Machado L. Effects of SVM parameter optimization on discrimination and calibration for post-procedural PCI mortality. Journal of Biomedical Informatics, 2007, 40(6): 688-697.
[133] Qi Wang, Zhigang Feng, SHDA Katsunori. Parameter optimization of LS-SVM for regression using NGA. SPIE-The International Society for Optical Engineering, Orlando, USA, 2007, 6560.
[134]董春曦,饶鲜,杨绍全,徐松涛.支持向量机参数选择方法研究.系统工程与电子技术,2004,26(8):117-120.
[135] M. Boardman, T. Trappenberq. A heuristic for free optimization with support vector machines. International Joint Conference on Neural Networks, 2006: 610-617.
[136]李蓉,叶世伟,史忠植.SVM-KNN分类器-一种提高SVM分类精度的新方法.电子学报,2002,30(5):745-748.
[137]刘万里,刘三阳,薛贞霞.基于支持向量域的分离超平面.系统工程与电子技术,2008,30(4):748-751.
[138] P. Sollich, A. Krogh. Learning with ensembles: how over-fitting can be useful. In Advances in Neural Information Processing Systems, Cambridge, MA: MIT Press, 1996, 8: 190-196.
[139] L. K. Hansen, P. Salamon. Neural Network ensembles. IEEE Transaction onPattern Analysis and Machine Intelligence, 1990, 12(10): 993-1001.
[140] Breiman L. Bagging preditors. Machine Learning, 1996, 24(2): 123-140.
[141] Y. Freund, R. E. Schapire. Experiments with a new boosting algorithm. Proceedings of the 13th Conference on Machine Learning, Bari, Italy, 1996: 148-156.
[142] L. Breiman. Arching classifiers. The Annals of Statistics, 1998, 26(3): 801-824.
[143] L. Breiman. Random forests. Machine Learning, 2001, 45(1): 5-32.
[144] T. G. Dietterich. Machine Learning Research: four current directions. The AI Magazine, 1997, 18(4): 97-136.
[145] Z. H. Zhou, J. X. Wu, W. Tang. Ensembling neural networks: many could be better than all. Artificial Intelligence, 2002, 137(1-2): 239-263.
[146] D. H. Wolpert. Stacked generalization. Neural Networks, 1992, 5(2): 241-259.
[147] T. Cover, P. Hart. Nearest neighbor pattern classification. IEEE Transaction on Information Theory, 1967, 13(1): 21-27.
[148] E. Fix, J. L. Hodges. Discriminatory analysis, non-parametric discrimination. USAF school of aviation medicine Randolph Field, Tex., Project 21-49-004, Report Number 4, 1951, 261-279.
[149]朱永生,王成栋,张优云.二次损失函数支持向量机性能的研究.计算机学报,2003,26(8):982-989.
[150] Yuh-Jye Lee, O. L. Mangasarian. SSVM: a smooth support vector machine for classification. Computational Optimization and Applications, 2001, 22(1): 5-21.
[151] Chunhui Chen, O. L. Mangasarian. A class of smoothing functions for nonlinear and mixed complementarity problems. Computational Optimization and Applications, 1996, 5(2): 97-138.
[152] J. E. Dennis, Robert B. Schnabel. Numerical methods for unconstrained optimization and nonlinear equations, Prentice-Hall, Englewood Cliffs, N. J., 1983.
[153] S. M. Lu, X. Z. Wang. A comparison among four SVM classification methods: LSVM, NLSVM, SSVM and NSVM. Proceedings of the Third International Conference on Machine Learning and Cybernetics, Shanghai, China, 2004: 4277-4282.
[154]袁玉波,严杰,徐成贤.多项式光滑的支持向量机.计算机学报,2005,28 (1):9-17.
[155] Min Jiang, Zhiqing Meng, Gengui Zhou. A smoothing support vector machine based on quarter penalty function. Proceedings of the International Conference on Computational Intelligence and Security, Harbin, China, 2007: 57-59.
[156] YuhJye Lee, WenFeng Hsieh, ChienMing Huang.ε-SSVR: A smooth support vector machine forε-insensitive regression. IEEE Transactions on Knowledge and Data Engineering, 2005, 17 (5): 678-685.
[157] Yubo Yuan. Spline function smooth support vector machine for classification.Journal of Industrial and Management Optimization, 2007, 3(3): 529-542.
[158] Jinzhi Xiong, Tianming Hu, Guangming Li, et al. A comparative study of three smooth SVM classifiers. Proceedings of the Sixth World Congress on Intelligent Control and Automation, Dalian, China, 2006: 5962-5966.
[159]熊金志,胡金莲,袁华强等.一类光滑支持向量机新函数的研究.电子学报,2007,35 (2):366-370.
[160] K. P. Bennett, O. L. Mangasarian. Robust linear programming discrimination of two linearly inseparable sets. Optimization Methods and Software, 1992, 1: 23-34.
[161] Lee KiYoung, Kim Dae-Won, Lee Kwang H., et al. Density-induced support vector data description. IEEE Transactions on Neural Networks, 2007, 18(1): 284-289.
[162]武小红,周建江.一种基于类中心最大间隔的支持向量机.信息与控制,2007,36(1):63-67.
[163] A. Ben-Hur, D. Horn, H. T. Siegelmann, et al. Support vector clustering. Journal of Machine Learning Research, 2001, 2:125-137.
[164] Chun-Hong Zheng, Li-cheng Jiao. Fuzzy Pre-extracting method for support vector machine. Proceedings of the First International Conference on Machine Learning and Cybernetics, Beijing, China, 2002, 4: 2026-2030.
[165] G. Fung, O. L. Mangasarian. Proximal support vector machine classifiers. Proceedings KDD-2001: Knowledge Discovery and Data Mining, San Francisco, California, 2001: 77-86.
[166] Jigang Wang, Predrag Neskovic, Leon N Cooper. Selecting data for fast support vector machines training. Chapter 3 in Studies in Computation Intelligence, Springer-Verlag, 2007, 35: 61-84.
[167] S. Abe, T. Inoue. Fast training of support vector machines by extracting boundary data. Lecture Notes in Computer Sciences, International Conference on Artificial Neural Networks, Berlin: Springer, 2001, 2130: 308-313.
[168] P. Juszczak, R. P. W. Duin. Selective sampling methods in one-class classification problems. Lecture Notes in Computer Sciences, Artificial Neural Networks and Neural Information Processing, Berlin: Springer, 2003, 2714: 140-148.
[2] F. Mulier. Vapnik-Chervonenkis (VC) learning theory and its applications. IEEE Transaction on Neural Networks, 1999, 10(5):985-987.
[3] F. Rosenblatt. Principles of neurodynamics: perceptrons and the theory of brain mechanisms. Spartans Books: Washington D.C., 1962.
[4] A. Novikoff. On convergence proofs for perceptrons. In Proceedings of the Symposium on the Mathematical Theory of Automata, 1962, 12:615-622.
[5] A. N. Kolmogorov. Three approaches to the quantitative definition of information. Problem of Information Transmission, 1965, 1(1):1-7.
[6] G. J. Chaitin. On the length of programs for computing finite binary sequences. Journal of Association with Computer Machine, 1966, 13(4): 547-569.
[7] Y. L. Cun. Learning process in an asymmetric threshold network. NATO ASI series. Series F: Computer and System Sciences, 1986, 20:233-240.
[8] D. E. Rumelhart, G. E. Hinton, R. J. Williams. Learning internal representations by error propagation. In Parallel Distributed Processing: Explorations in The Microstructure of Cognition, 1986, 1:318-362.
[9]高隽.人工神经网络原理及仿真实例.北京:机械工业出版社,2003.
[10]张学工译.统计学习理论的本质.北京:清华大学出版社,2000.
[11] V. N. Vapnik. An overview of statistical learning theory. IEEE Transaction on Neural Networks, 1999, 10(5): 988-999.
[12] V. N. Vapnik. Three fundamental concepts of the capacity of learning machines. Physica A, 1993, 200: 538-544.
[13]邓乃杨,田英杰.数据挖掘中的新方法-支持向量机.北京:科学出版社,2005.
[14] C. J. C. Burges. A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery, 1998, 2(2): 121-167.
[15] A. J. Smola, B. Sch?lkopf. A tutorial on support vector regression. Statistics and Computing, 2004, 14(3): 199-222.
[16] R. Mohamed, A. El-Baz, A. Farag. Probability density estimation using advanced support vector machines and the expectation maximization algorithm. Proceedings of World Academy of Science, Engineering and Technology, 2005, 2: 137-140.
[17]王国胜,钟义信.支持向量机的若干新进展.电子学报,2001,29(10): 1397-1400.
[18] S. Amari, S. Wu. Improving support machine classifier by modifying kernel function. Neural Networks, 1999, 12(6): 783-789.
[19] H. Lodhi, C. Saunders, J. Shawe-Taylor, et al. Text classification using string kernels. Journal of Machine Learning Research, 2002, 2(2): 419-444.
[20] N. Cancedda, E. Gaussier, C. Goutte, et al. Word sequence kernels. Journal of Machine Learning Research, 2003, 3(6): 1059-1082.
[21] G. Camps-Valls, L. Geomez-Chova, J. Vila-Frances, et al. Composite kernels for hyperspectral image classification. IEEE Geosciences and Remote Sensing Letters, 2006, 3(1): 93-98.
[22] K. Duan, S. S. Keerthi, A. N. Poo. Evaluation of simple performance measures for tuning SVM hyperparameters. Neurocomputing, 2003, 51: 41-59.
[23] G. Wahba, Y. Lin, H. Zhang, et al. Generalized approximate cross-validation for support vector machines: another way to look at margin-like quantities. Advances in Large Margin Classifier, Cambridge, MA: MIT Press, 2000, 297-309.
[24] A. Lunts, V. Brailovskiy. Evaluation of attributes obtained in statistical decision rules. Engineering Cybernetics, 1967, 3(1): 98-109.
[25] V. N. Vapnik, O. Chapell. Bounds on error expectation for support machine. Neural Computation, 2000, 12(9): 2013-2036.
[26] T. S. Joaakkola, D. Haussler. Probabilistic kernel regression models. In Proceedings of the 7th workshop on AI and Statistics, San Francisco, 1999, 26-34.
[27] M. Opper, O. Winther. Gaussian processes for classification: mean-field algoithms. Neural Computation, 2000, 12(11): 2655-2684.
[28] B. Sch?lkopf and A. Smola. Learning with Kernel. MA: MIT press, 2002.
[29] D. J. C. Machay. Probable networks and plausible predictions-a review of practical bayesian methods for supervised neural networks. Network Computation in Neural Systems, 1995, 6(3): 469-505.
[30] J. T. Kowk. The evidence framework applied to support vector machines. IEEE Transactions on Neural Network, 2000, 11(5): 1162-1173.
[31] B. Sch?lkopf, A. Smola, R. Williamson, et al. New support vector algorithms. Neural Computation, 2000, 12(5): 1207-1245.
[32] D. M. J. Tax, R. P. W. Duin. Support vector domain description. Pattern Recognition Letters, 1999, 20(11-13): 1191-1199.
[33] J. A. K. Suykens, T. Van Gestel, J. De Brabanter, et al. Least squares support vector machines. World Scientific: Singapore, 2002.
[34] T. V. Gestel, J. A. K. Suykens, B. Baesens, et al. Benchmarking least squares support vector machine classifiers. Machine Learning, 2004, 54(1): 5-32.
[35] Danny Roobaert. DirectSVM: a fast and simple support vector machine perceptron. Journal of VLSI Signal Processing Systems, 2002, 32(1-2): 147-156.
[36] V. Kecman, L. Hadzic. Support vectors selection by linear programming. Proceedings of the International Joint Conference on Neural Networks (IJCNN’2000), Como, 2000, 5: 193-198.
[37] L. Hadzic, V. Kecman. Support vector machines trained by linear programming: theory and application in image compression and data classification. The Fifth IEEE Seminar on Neural Network Applications inElectrical Engineering, Belgrade, 2000: 18-23.
[38] Zhou Wei-da, Zhang Li, Jiao Li-cheng. Linear programming support vector machines. Pattern Recognition, 2002, 35(12): 2927-2936.
[39] O. L. Mangasarian. Generalized support vector machines. Advances in Large Margin Classifiers, MA: MIT Press, 1998.
[40] O. L. Mangasarian, D. R. Musicant. Data discrimination via nonlinear generalized support vector machines. Complementarity: Applications, Agorithms and Extensions, Kluwer academic publishers, 1999: 233-251.
[41] O. L. Mangasarian, D. R. Musicant. Successive over relaxation for support vector machines. IEEE Transactions on Neural Networks, 1999, 10(5): 1032-1037.
[42] D. R. Musicant, O. L. Mangasarian. LSVM: Lagrangian support vector machine. Journal of Machine Learning Research, 2001, 1: 161-177.
[43] O. L. Mangasarian, D. R. Musicant. Active set support vector machine classification. Advances in Neural Information Processing Systems 2000, MA: MIT Press, 2001: 577-583.
[44] D. M. J. Tax, R.P.W. Duin. Support vector data description. Machine Learning, 2004, 54(1): 45-66.
[45] J. Munoz-Mari, L. Bruzzone, G. Camps-Valls. A support vector domain description approach to supervised classification of remote sensing images. IEEE Transaction on Geosciences and Remote Sensing, 2007, 45(8): 2683-2692.
[46] Sang-Woong Lee, Seong-Whang Lee. SVDD based illumination compensation for face recognition. International Joint Conference on Advances in Biometrics, 2007, Korea.
[47] Chao Yuan, David Casasent. A novel support vector classifier with better rejection performance. 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003, 1: 419-424.
[48] J. Park, D. Kang, J. Kim, et al. SVDD-based pattern denoising. Neural Computation, 2007, 19(7): 1919-1938.
[49]梁锦锦,刘三阳,吴德.空间支持向量域分类器.西安电子科技大学学报,2008,35(6):1080-1083转1088.
[50]陆从德,张太镒,胡金艳.基于乘性规则的支持向量域分类器.计算机学报,2004,27(5):690-694.
[51] D. M. J. Tax, R. P. W. Duin. Uniform object generation for optimizing one-class classifiers. Journal of Machine Learning Research, 2001, 2: 155-173.
[52] K.Crammer, Y. Singer. On the learnability and design of output codes for multiclass problems. Machine Learning, 2002, 47(2-3): 201-233.
[53] T. G. Dietterich, G. Bakiri. Solving multiclass learning problems via error-correcting output codes. Journal of Artificial Intelligence Research, 1995, 2: 263-286.
[54] C.W. Hsu, C. J. Lin. A comparison of methods for multiclass support vector machines. IEEE transaction on Neural Networks, 2002, 13(2): 415-425.
[55] J. Cervantes, L. Xiaoou, Y. Wen, et al. Multi-class support vector machines for large data sets via minimum enclosing ball clustering. IEEE 4th International Conference on Electrical and Electronics Engineering, Mexico, 2007: 146-149.
[56] P. Lingras, C. J. Butz. Rough set based 1-v-1 and 1-v-r approaches to support vector machine multi-classification. Information Sciences, 2007, 177(18): 3782-3798.
[57] J. C. Platt, N. Cristianini, J. Shawe-Taylor. Large margin DAGs for multiclass classification. Advances in Neural Information Processing System, 2000: 547-553.
[58] C. Cortes, V. N. Vapnik. Support vector networks. Machine Learning, 1995, 20 (3): 273-297.
[59] E. Osuna, R. Freund, F. Girosi. An improved training algorithm for support vector machines. Proceedings of the 1997 IEEE workshop on Neural Networks for signal processing, New York: IEEE press, 1997: 276-285.
[60] T. Joachims. Making large-scale SVM learning practical. Advances in Kernel Methods: Support Vector Learning. Cambridge, MA: MIT press, 1999: 169-184.
[61] C. J. Lin. On the convergence of the decomposition method for support vector machines. IEEE Transactions on Neural Networks, 2001, 12(6): 1288-1298.
[62] C. W. Hsu, C. J. Lin. A simple decomposition method for support vector machines. Machine learning, 2002, 46(1): 291-314.
[63] J. C. Platt. Fast training of support vector machines using sequential minimal optimization. Advances in Kernel Methods: Support Vector Learning, Cambridge, MA: MIT press, 1999, 185-208.
[64] Chih-Jen Lin. Asymptotic convergence of an SMO algorithm without any assumptions. IEEE transactions on Neural Networks, 2002, 13(1): 248-250.
[65] S. S. Keerthi, S. K. Shevade, C. Bhattacharyya, et al. Improvements to Platt’s SMO algorithm for SVM classifier design. Neural Computation, 2001, 13(3): 637-649.
[66] G. Cauwenberghs, T. Poggio. Incremental and decremental support vector machine learning. Advances in Neural Information Processing Systems, Cambridge, MA: MIT press, 2001, 13: 409-415.
[67]箫嵘,王继成,孙正兴等.一种SVM增量学习算法α-ISVM.软件学报,2001,12(12):1818-1824.
[68] J. X. Dong, A. Kyzyzak, C. Y. Suen. A fast parallel optimization for training support vector machine. Proceedings of 3rd International Conference on Machine Learning and Data Mining, 2003, 2734: 96-105.
[69] H. P. Graf, E. Cosato, L. Botou, et al. Parallel support vector machines: the cascade SVM. Advances in Neural Information Processing Systems, Cambridge, MA: MIT Press, 2004: 521-528.
[70] R. Collobert, S. Bengio, Y. Bengio. A parallel mixture of SVMs for Large scale problems. Neural Computation, 2002, 14(5): 1105-1114.
[71] G. Zanghirati, L. Zanni. A parallel solver for large quadratic programs intraining support vector machine. Parallel computing, 2003, 29(4):535-551.
[72] L. J. Cao, S. S. Keerthi, C. J. Ong, et al. Parallel sequential minimal optimization for the training of support vector machines. IEEE transaction on neural networks, 2006, 17(4): 1039-1049.
[73] G. Valentini, T. G. Dietterich. Bias-variance analysis of support vector machines for the development of SVM based ensemble methods. Journal of Machine Learning Research, 2004, 5: 725-775.
[74] B. Robert, G. Ricardo, Q. Francis. Attribute bagging: Improving accuracy of classifier ensembles by using random feature subsets. Pattern recognition, 2003, 36(6): 1291-1302.
[75] Kim H., Pang S., Je H., et al. Constructing support vector machine ensemble. Pattern recognition, 2003, 36(12): 2757-2767.
[76] X. C. Li, L. Wang, E. Sung. A study of AdaBoost with SVM weak learners. International Joint Conference on Neural Networks, Montreal, Canada, 2005, 1: 196-201.
[77] Y. S. Dong, Han K. S. A comparison of several ensemble methods for text categorization. 2004 IEEE International Conference on Services Computing, Shanghai, China, 2004: 419-422.
[78] D. C Tao., X. O. Tang, X. L. Li, et al. Asymmetric bagging and random subspace for support vector machines-based relevance feedback in image retrieval. IEEE Transaction on Pattern Analysis and Machine Intelligence, 2006, 28(7): 1088-1099.
[79]焦李成,张莉,周伟达.支持向量预选取的中心距离比值法.电子学报,2001,29(3):383-386.
[80] M. H. Yang, N. Ahuja. A geometric approach to training support vector machines. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Hilton Head Island, USA, 2000, 1: 430-437.
[81] M. Rychetsky, S. Ortmann, M. Ullmann, et al. Accelerated training of support vector machines. International Joint Conference on Neural Networks, Washington, USA, 1999, 2: 998-1003.
[82]李青,焦李成,周伟达.基于向量投影的支持向量预选取.计算机学报,2005,28(2):145-152.
[83] H. Shin, S. Cho. Fast pattern selection for support vector classifiers. Lecture Notes in Computer Science, Springer, 2003, 2637: 376-387.
[84] H. Yu, J. Yang, J. Han. Classifying large data sets using SVM hierarchical clusters. Proceedings of the Ninth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Washington, USA, 2003, 306-315.
[85] Zhang Li, Zhou Weida, Jiao Licheng. Pre-extracting support vectors for support vector machines. Signal Processing Proceedings, Beijing, China, 2000, 3: 1432-1435.
[86] C. F. Lin, S. D. Wang. Fuzzy support vector machines. IEEE Transaction on Neural Network, 2002, 13(3): 466-471.
[87] H. P. Huang, Y. H. Liu. Fuzzy support vector machines for patternrecognition and data mining. International Journal of Fuzzy Systems, 2002, 4(3): 826-835.
[88]张翔,肖小玲,徐光祐.基于样本之间紧密度的模糊支持向量机方法.软件学报,2006,17(5):951-958.
[89] S. Mika, B. Sch?lkopf, A. Smola, et al. Kernel PCA and de-nosing in feature spaces. Advances in Neural Information Processing Systems, MA: MIT Press, Cambridge, 1999, 11: 536-542.
[90] D. Nguyen, T. Ho. A bottom-up method for simplifying support vector solutions. IEEE Transactions on Neural Networks, 2006, 17(3): 792-796.
[91] H. Shin, S. Cho. Neighborhood property based pattern selection for support vector machines. Neural Computation, 2007, 19(3): 816-855.
[92] Y. F. Shi, Y. P. Zhao. Comparison of text categorization algorithms. Wuhan University Journal of Natural Sciences, 2004, 9(5): 798-804.
[93] D. A. Reynolds, R. C. Rose. Robust text-independent speaker identification using Gaussian mixture speaker models. IEEE Transactions on Speech and Audio Processing, 1995, 3(1): 72-83.
[94] Bernd Heisele, Thomas Serre, Sam Prentice, et al. Hierarchical classification and feature reduction for fast face detection with support vector machines. Pattern Recognition, 2003, 36(9): 2007-2017.
[95] A. David, B. Lerner. Support vector machine-based image classification for genetic syndrome diagnosis. Pattern Recognition Letters, 2005, 26(8): 1029-1038.
[96] M. Felgani, B. Lorenzo. Classification of hyperspectral remote sensing images with support vector machines. IEEE Transactions on Geoscience Remote Sensing, 2004, 42(8): 1778-1790.
[97] G. Camps-Valls, L. Gomez-Chova, J. Calpe-Maravilla, et al. Robust support vector method for hyperspectral data classification and knowledge discovery. IEEE Transactions on Geoscience and Remote Sensing, 2004, 42(7): 1530-1542.
[98] F. Perez-Cruz, O. Bousquet. Kernel methods and their potential use in Signal processing. IEEE Signal Processing Magazine, 2004, 21(3): 57-65.
[99] M. Rychetsky, S. Ortmann, M. Glesner. Support vector approaches for engine knock detection. International Joint Conference on Neural Networks, 1999, 2: 969-974.
[100] P. M. L. Drezet, R. F. Harrison. Support vector machines for system identification. UKACC International Conference on Control, 1998, 1: 688- 692.
[101] B. J. De Kruif, T. J. A. De Vries. On using a support vector machine in learning feed-forward control. International Conference on Advanced Intelligent Mechatronics, Como, Italy, 2001, 1: 272-277.
[102] L. B. Jack, A. K. Nandi. Fault detection using support vector machines and artificial neural networks. Mechanical Systems and Signal Processing, 2002, 16(2-3): 373-390.
[103] M. Ge, R. Du, G. C. Zhang, et al. Fault diagnosis using support vectormachine with an application in sheet metal stamping operations. Mechanical Systems and Signal Processing, 2004, 18(1): 143-159.
[104] F. Mike, R. James. Computer intrusion detection with classification and anomaly detection using SVMs. International Journal of Pattern Recognition and Artificial Intelligence, 2003, 17(3): 441-458.
[105] F. E. H. Tay, L. Cao. Application of support vector machines in financial time series forecasting. Omega, 2001, 29(4): 309-317.
[106] Bhanu Prakash K. N., Ramakrishnan A. G., Suresh S., et al. Fetal lung maturity analysis using ultrasound image features. IEEE Transactions on Information Technology in Biomedicine, 2002, 6(1): 38-45.
[107] Liu Weiqiang, Shen Peihua, Qu Yingge, et al. Fast algorithm of support vector machines in lung cancer diagnosis. Proceedings of International Workshop on Medical Imaging and Augmented Reality, Hong Kong, China, 2001: 188-192.
[108] L. Bao, Z. Sun. Identifying genes related to drug anticancer mechanisms using support vector machine. FEBS Letters, 2002, 521(1-3): 109-114.
[109] Mark G, Ronald J. The current excitement in bioinformatics-analysis of whole-genome expression data: how does it relate to protein structure and function? Current Opinion in Structural Biology, 2000, 10(5): 574-584.
[110]孙德山,吴今培,肖健华. SVR在混沌时间序列预测中的应用.系统仿真学报,2004,16(3):519-524.
[111] L. J. Cao, F. E. H. Tay. Support vector machine with adaptive parameters in financial time series forecasting. IEEE Transactions on Neural Networks, 2003, 14(6):1506-1518.
[112] S. Chen, A. K. Samingan, L. Hanzo. Support vector machine multiuser receiver for DS-CDMA Signals in Multipath Channels. IEEE Transactions on Neural Networks, 2001, 12(3): 604-611.
[113]王磊.支持向量机学习算法的若干问题研究.成都:电子科技大学博士学位论文,2007.
[114] V. N. Vapnik, A. Y. Chervonenkis. On the uniform convergence of relative frequencies of events to their probabilities. Theory of Probability and its Applications, 1971, 16(2): 264-280.
[115] V. N. Vapnik. Statistical learning theory. Wiley: New York, 1998.
[116]袁亚湘,孙文瑜著.最优化理论与方法.北京:科学出版社,1997.
[117] N. Cristianini, J. Shawe-Taylor. An introduction to support vector machines and other kernel-based learning methods. Cambridge: Cambridge University Press, 2000.
[118]薛毅.最优化原理与方法.北京:北京工业大学出版社,2003.
[119]范明,柴玉梅等译.统计学习基础-数据挖掘、推理与预测.北京:电子工业出版社,2004.
[120] C. A. Micchelli. Interpolation of scattered data: distance matrices and conditionally positive definite functions. Constructive Approximation, 1986, 2(1): 11-12.
[121] D. J. C. Mackay. Introduction to Gaussian processes. NATO ASI Series.Series F: Computer and System Sciences, 1998, 168: 133-165.
[122] F. Girosi. An equivalence between sparse approximation and support vector machines. Neural Computation, 1997, 10(6): 1455-1480.
[123] B. Sch?lkopf, C. J. C. Burges, A. J. Smola. Advances in kernel methods support vector learning. Cambridge: MIT Press, 1998.
[124] D. Haussler. Convolution kernels on discrete structures. Technical Report UCSC-CRL-99-10, University of Californian Santa Cruz, Computer Science Department, 1999.
[125] C. Watkins. Dynamic alignment kernels. In: Advances in Large Margin Classifiers, ed. By A. J. Smola, P. L. Bartlett, et al, Cambridge, MA: MIT Press, 2000.
[126] M. Carozza, S. Rampone. Towards an incremental SVM for regression. In Procreedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks, Como, Italy, 2000, 16: 405-410.
[127]孔锐,张冰.一种快速支持向量机增量学习算法.控制与决策,2005,20(10):1129-1132转1136.
[128] Lu Bao-liang, Wang Kai-an, M. Utiyama, et al. A part-verus part method for massively parallel training of support vector machines. 2004 IEEE International Joint Conference on Neural Networks, 2004, 1: 735-740.
[129] Anton Schwaighofer, Vloker Tresp. The bayesian committee support vector machine. Lecture Notes in computer Science, 2001, 2130: 411-417.
[130]文益民,廖洪元,周立华.一种可减少训练时间的分层并行支持向量机方法.南京师范大学学报,2005,5(1):8-11.
[131]许建华,张学工,李衍达.支持向量机的新发展.控制与决策,2004,19(5):481-484.
[132] Matheny ME, Resnic Fs, Arora N, Ohno-Machado L. Effects of SVM parameter optimization on discrimination and calibration for post-procedural PCI mortality. Journal of Biomedical Informatics, 2007, 40(6): 688-697.
[133] Qi Wang, Zhigang Feng, SHDA Katsunori. Parameter optimization of LS-SVM for regression using NGA. SPIE-The International Society for Optical Engineering, Orlando, USA, 2007, 6560.
[134]董春曦,饶鲜,杨绍全,徐松涛.支持向量机参数选择方法研究.系统工程与电子技术,2004,26(8):117-120.
[135] M. Boardman, T. Trappenberq. A heuristic for free optimization with support vector machines. International Joint Conference on Neural Networks, 2006: 610-617.
[136]李蓉,叶世伟,史忠植.SVM-KNN分类器-一种提高SVM分类精度的新方法.电子学报,2002,30(5):745-748.
[137]刘万里,刘三阳,薛贞霞.基于支持向量域的分离超平面.系统工程与电子技术,2008,30(4):748-751.
[138] P. Sollich, A. Krogh. Learning with ensembles: how over-fitting can be useful. In Advances in Neural Information Processing Systems, Cambridge, MA: MIT Press, 1996, 8: 190-196.
[139] L. K. Hansen, P. Salamon. Neural Network ensembles. IEEE Transaction onPattern Analysis and Machine Intelligence, 1990, 12(10): 993-1001.
[140] Breiman L. Bagging preditors. Machine Learning, 1996, 24(2): 123-140.
[141] Y. Freund, R. E. Schapire. Experiments with a new boosting algorithm. Proceedings of the 13th Conference on Machine Learning, Bari, Italy, 1996: 148-156.
[142] L. Breiman. Arching classifiers. The Annals of Statistics, 1998, 26(3): 801-824.
[143] L. Breiman. Random forests. Machine Learning, 2001, 45(1): 5-32.
[144] T. G. Dietterich. Machine Learning Research: four current directions. The AI Magazine, 1997, 18(4): 97-136.
[145] Z. H. Zhou, J. X. Wu, W. Tang. Ensembling neural networks: many could be better than all. Artificial Intelligence, 2002, 137(1-2): 239-263.
[146] D. H. Wolpert. Stacked generalization. Neural Networks, 1992, 5(2): 241-259.
[147] T. Cover, P. Hart. Nearest neighbor pattern classification. IEEE Transaction on Information Theory, 1967, 13(1): 21-27.
[148] E. Fix, J. L. Hodges. Discriminatory analysis, non-parametric discrimination. USAF school of aviation medicine Randolph Field, Tex., Project 21-49-004, Report Number 4, 1951, 261-279.
[149]朱永生,王成栋,张优云.二次损失函数支持向量机性能的研究.计算机学报,2003,26(8):982-989.
[150] Yuh-Jye Lee, O. L. Mangasarian. SSVM: a smooth support vector machine for classification. Computational Optimization and Applications, 2001, 22(1): 5-21.
[151] Chunhui Chen, O. L. Mangasarian. A class of smoothing functions for nonlinear and mixed complementarity problems. Computational Optimization and Applications, 1996, 5(2): 97-138.
[152] J. E. Dennis, Robert B. Schnabel. Numerical methods for unconstrained optimization and nonlinear equations, Prentice-Hall, Englewood Cliffs, N. J., 1983.
[153] S. M. Lu, X. Z. Wang. A comparison among four SVM classification methods: LSVM, NLSVM, SSVM and NSVM. Proceedings of the Third International Conference on Machine Learning and Cybernetics, Shanghai, China, 2004: 4277-4282.
[154]袁玉波,严杰,徐成贤.多项式光滑的支持向量机.计算机学报,2005,28 (1):9-17.
[155] Min Jiang, Zhiqing Meng, Gengui Zhou. A smoothing support vector machine based on quarter penalty function. Proceedings of the International Conference on Computational Intelligence and Security, Harbin, China, 2007: 57-59.
[156] YuhJye Lee, WenFeng Hsieh, ChienMing Huang.ε-SSVR: A smooth support vector machine forε-insensitive regression. IEEE Transactions on Knowledge and Data Engineering, 2005, 17 (5): 678-685.
[157] Yubo Yuan. Spline function smooth support vector machine for classification.Journal of Industrial and Management Optimization, 2007, 3(3): 529-542.
[158] Jinzhi Xiong, Tianming Hu, Guangming Li, et al. A comparative study of three smooth SVM classifiers. Proceedings of the Sixth World Congress on Intelligent Control and Automation, Dalian, China, 2006: 5962-5966.
[159]熊金志,胡金莲,袁华强等.一类光滑支持向量机新函数的研究.电子学报,2007,35 (2):366-370.
[160] K. P. Bennett, O. L. Mangasarian. Robust linear programming discrimination of two linearly inseparable sets. Optimization Methods and Software, 1992, 1: 23-34.
[161] Lee KiYoung, Kim Dae-Won, Lee Kwang H., et al. Density-induced support vector data description. IEEE Transactions on Neural Networks, 2007, 18(1): 284-289.
[162]武小红,周建江.一种基于类中心最大间隔的支持向量机.信息与控制,2007,36(1):63-67.
[163] A. Ben-Hur, D. Horn, H. T. Siegelmann, et al. Support vector clustering. Journal of Machine Learning Research, 2001, 2:125-137.
[164] Chun-Hong Zheng, Li-cheng Jiao. Fuzzy Pre-extracting method for support vector machine. Proceedings of the First International Conference on Machine Learning and Cybernetics, Beijing, China, 2002, 4: 2026-2030.
[165] G. Fung, O. L. Mangasarian. Proximal support vector machine classifiers. Proceedings KDD-2001: Knowledge Discovery and Data Mining, San Francisco, California, 2001: 77-86.
[166] Jigang Wang, Predrag Neskovic, Leon N Cooper. Selecting data for fast support vector machines training. Chapter 3 in Studies in Computation Intelligence, Springer-Verlag, 2007, 35: 61-84.
[167] S. Abe, T. Inoue. Fast training of support vector machines by extracting boundary data. Lecture Notes in Computer Sciences, International Conference on Artificial Neural Networks, Berlin: Springer, 2001, 2130: 308-313.
[168] P. Juszczak, R. P. W. Duin. Selective sampling methods in one-class classification problems. Lecture Notes in Computer Sciences, Artificial Neural Networks and Neural Information Processing, Berlin: Springer, 2003, 2714: 140-148.
© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号 地址：北京市海淀区学院路29号 邮编：100083 电话：办公室：(+86 10)66554848；文献借阅、咨询服务、科技查新：66554700 |