支持向量机算法的研究及应用
|
摘要
基于统计学习理论的支持向量机是一种新型的学习方法,它采用结构风险最小化原则,是一个凸二次优化问题,能够保证找到的极值解就是全局最优解,从而在统计样本量较少的情况下获得良好的统计规律和更好的泛化能力,为解决小样本、非线性、高维数等学习问题提供了一个框架,帮助解决了许多其他学习方法难以解决的问题。本文针对支持向量机的理论和应用做了如下研究:
在详细分析SVM算法及其属性的基础上,利用SVM的解具有稀疏性的特点,提出了一种基于模糊核聚类的数据约简型支持向量机算法。该算法利用非线性映射和核技巧,通过模糊核聚类方法将数据映射到高维特征空间后聚类,以此来寻找靠近最优分类面的数据,从而进行数据的约简,在保证推广能力不受太大影响的前提下,缩小SVM的求解规模,从而提高其学习速度。实验的结果证实了该数据约简算法的可行性和有效性。
为了进一步提高SVM的推广性能,本文提出了一种基于改进Adaboost的ε不敏感支持向量回归集成算法。该算法使用多个支持向量机,按照某种学习规则协调各支持向量机的输出,从而提高其泛化性能。将该方法应用于双酚A生产过程中质量指标的软测量建模,仿真结果表明了该集成算法的可行性和有效性。
参数选择是支持向量机研究领域的重要问题之一。针对SVR参数对模型的推广能力影响较大,但目前又无完善的理论指导参数选取这一问题,本文提出了一种基于二分法的核参数解路径算法。在该算法中,随着参数的更新,在已有参数解的基础上进行推导计算以求得当前参数的最优解,而其目标函数的极值所对应的参数值即为最优参数解。数值函数和实际应用例子表明该方法可以快速地求得推广能力最佳的模型所对应的参数。
Support Vector Machine (SVM) is a kind of novel machine learning methods, theoretically based on statistic learning theory. It employs the criteria of structural risk minimization. And it’s a quadratic programming problem which can make sure that the extreme solution found is the optimal one. So it can use limited information to obtain statistic principles and high generalization, and can also provide a framework for the small samples, nonlinearity and high dimension problems which most traditional learning methods can’t solve. In this paper, a series of work on the theory and application of support vector machine was discussed.
After analyzing the SVM theory in detail and using the characteristic of solution sparseness, a data reducing algorithm of support vector machine based on fuzzy kernel clustering was proposed. Through nonlinear mapping and kernel trick, the data which were mapped into a high dimensional feature space from the original space can cluster in the feature space by using fuzzy kernel clustering algorithm. So the data which were most likely to be support vectors, can be found from the sub-clusters that were located near the optimal classification hyperplane. And the size of sample-data for SVM training turns to be small. Meanwhile the training time was reduced greatly without compromising the generalization capability. The simulations show that this new method was effective.
In order to further improve the generalization of SVM, an improved support vector regression ensemble algorithm was proposed. Learning by a series of support vector regressions and combining all the results in accordance with some rule, the algorithm improves its regression performance greatly. Moreover, the proposed algorithm was used in a soft–sensor model for the Bisphenol-A productive process. Simulations using artificial and real data also demonstrated that the algorithm was effective.
Parameter selection was one of the most important issues in the research of support vector machines. The previous researches show that the SVM’s generalization capacity was greatly affected by its parameters. But there have been few theoretical methods to choose the SVR’s parameters so far. A solution path algorithm with respect to kernel parameter based on the bisection method was proposed. With the update of the parameter, the current solution can be computed based on an already obtained one, and the value of the parameter which is correlated with the extreme value of the target function is the optimal one. Simulations using artificial and real data show that this algorithm can quickly get the model which has better generalization.
在详细分析SVM算法及其属性的基础上,利用SVM的解具有稀疏性的特点,提出了一种基于模糊核聚类的数据约简型支持向量机算法。该算法利用非线性映射和核技巧,通过模糊核聚类方法将数据映射到高维特征空间后聚类,以此来寻找靠近最优分类面的数据,从而进行数据的约简,在保证推广能力不受太大影响的前提下,缩小SVM的求解规模,从而提高其学习速度。实验的结果证实了该数据约简算法的可行性和有效性。
为了进一步提高SVM的推广性能,本文提出了一种基于改进Adaboost的ε不敏感支持向量回归集成算法。该算法使用多个支持向量机,按照某种学习规则协调各支持向量机的输出,从而提高其泛化性能。将该方法应用于双酚A生产过程中质量指标的软测量建模,仿真结果表明了该集成算法的可行性和有效性。
参数选择是支持向量机研究领域的重要问题之一。针对SVR参数对模型的推广能力影响较大,但目前又无完善的理论指导参数选取这一问题,本文提出了一种基于二分法的核参数解路径算法。在该算法中,随着参数的更新,在已有参数解的基础上进行推导计算以求得当前参数的最优解,而其目标函数的极值所对应的参数值即为最优参数解。数值函数和实际应用例子表明该方法可以快速地求得推广能力最佳的模型所对应的参数。
Support Vector Machine (SVM) is a kind of novel machine learning methods, theoretically based on statistic learning theory. It employs the criteria of structural risk minimization. And it’s a quadratic programming problem which can make sure that the extreme solution found is the optimal one. So it can use limited information to obtain statistic principles and high generalization, and can also provide a framework for the small samples, nonlinearity and high dimension problems which most traditional learning methods can’t solve. In this paper, a series of work on the theory and application of support vector machine was discussed.
After analyzing the SVM theory in detail and using the characteristic of solution sparseness, a data reducing algorithm of support vector machine based on fuzzy kernel clustering was proposed. Through nonlinear mapping and kernel trick, the data which were mapped into a high dimensional feature space from the original space can cluster in the feature space by using fuzzy kernel clustering algorithm. So the data which were most likely to be support vectors, can be found from the sub-clusters that were located near the optimal classification hyperplane. And the size of sample-data for SVM training turns to be small. Meanwhile the training time was reduced greatly without compromising the generalization capability. The simulations show that this new method was effective.
In order to further improve the generalization of SVM, an improved support vector regression ensemble algorithm was proposed. Learning by a series of support vector regressions and combining all the results in accordance with some rule, the algorithm improves its regression performance greatly. Moreover, the proposed algorithm was used in a soft–sensor model for the Bisphenol-A productive process. Simulations using artificial and real data also demonstrated that the algorithm was effective.
Parameter selection was one of the most important issues in the research of support vector machines. The previous researches show that the SVM’s generalization capacity was greatly affected by its parameters. But there have been few theoretical methods to choose the SVR’s parameters so far. A solution path algorithm with respect to kernel parameter based on the bisection method was proposed. With the update of the parameter, the current solution can be computed based on an already obtained one, and the value of the parameter which is correlated with the extreme value of the target function is the optimal one. Simulations using artificial and real data show that this algorithm can quickly get the model which has better generalization.
引文
1.黄凤岗,宁克欧.模式识别[M].黑龙江:哈尔滨工业大学出版社, 1998
2.韩力群.人工神经网络理论、设计与应用[M].北京:化学工业出版社, 2002
3. N. Vladimir, V. Vapnik. An overview of statistical learning theory[J]. IEEE Transaction on Neural Networks, 1999, 10(5):988-999
4.张学工.关于统计学习理论与支持向量机[J].自动化学报, 2000, 26(1):32-42
5.邓乃杨,田英杰.数据挖掘中的新方法——支持向量机[M].北京:科学出版社, 2004
6. V. Vapnik. The nature of statistical learning theory[M]. 2nd edition. New York: Springer-Verlag, 1999
7. V. Vapnik.统计学习理论的本质[M].第2版.张学工译.北京:清华大学出版社, 2000
8.邵信光.基于回归型支持向量机的数据建模方法研究[D]:硕士学位论文.无锡:江南大学通信与控制工程学院, 2006
9. E. Osuna, R. Freund, F Girosi. An improved training algorithm for support vector machine[C]. In: Proceedings of the 1997 IEEE Workshop on Neural Networks and Signal Processing, 1997, 276-285
10. J. Platt. Fast training of support vector machines using sequential minimal optimization[C]. In: Advances in Kernel Method: Support Vector Learning, 1999, 185-208
11. J. Suykens, J. Vandewalle. Least squares support vector machines[J]. Neural Processing Letters, 1999, 9(3):293-300
12. Y. J. Lee, O. L. Mangasarian. SSVM:A smooth support vector machine for classification[J]. Computational Optimization and Applications, 2001, 20(1):5-22
13. Y. J. Lee, O. L. Mangasarian. RSVM:Reduced support vector machines[C]. In: Proceedings of the 1st SIAM International Conference on Data Mining, 2001
14. M. B. Almeida, A. P. Braga, J. P. Braga. SVM-KM:speeding SVMs learning with a priori cluster selection and k-means[C]. In: Proceedings of the 6th Brazilian Symposium on Neural Networks, 2000, 162-167
15. N. Ducdung, T. B. Ho. A bottom-up method for simplifying support vector solution[J]. IEEE Transactions on Neural Networks, 2006, 17(3):792-796
16. G. Ratsch, T. Onoda, K. Muller. Soft margins for AdaBoost[J]. Machine Learning, 2001, 42(3):287-320
17. O. Chappelle, V. Vapnik, O. Bousquet, et al. Choosing multiple parameters for support vector machines[J]. Machine Learning, 2002, 46(1):131-160
18. P. W. Chen, J. Y. Wang, H. M. Lee. Model selection of SVMs using GA approach[A]. In: Proceedings of2004 IEEE International Joint Conference on Neural Networks, 2004, 3:2035-2040
19. C. H. Zheng, L. C. Jiao. Automatic parameters selection for SVM based on GA[C]. In: Proceedings of the 5th World Congress on Intelligent Control and Automation, 2004, 1869-1872
20.邵信光,杨慧中,陈刚.基于粒子群优化算法的支持向量机参数选择及应用[J].控制理论及应用, 2006, 23(5):740-743
21. B. Efron, T. Hastie, I. Johnstone, et al. Least angle regression[J]. Annals of Statistics, 2004, 32(2):407–499
22. T. Hastie, S. Rosset, R. Tibshirani. The entire regularization path for the support vector machine[J]. Journal of Machine Learning Research, 2004, (5):1391–1415
23. G. Wang, D. Yeung, F. Lochovsky. Two-dimensional solution path for support vector regression[C]. In: Proceedings of the 23th International Conference on Machine Learning, 2006, 993-1000
24. G. Wang, D. Yeung, F. Lochovsky. A kernel path algorithm for support vector machines[C]. In: Proceedings of the 24th International Conference on Machine Learning, 2007, 951-958
25. B. Sch?lkoph, A. Smola, K. Muller. Nonlinear component analysis as a kernel eigenvalue problem[J]. Neural Computation, 1998, 10(5):1299-1319
26. H. Gábor. Kernel CMAC with improved capability[C]. In: Proceedings of the International Joint Conference on Neural Networks, 2004, 681-586
27.张莉,周传达,焦李成.核聚类算法[J].计算机学报, 2002, 25(6):587-590
28. M. Girolami. Mercer kernel-based clustering in feature space[J]. Neural Networks, 2002, 13(3):780-784
29. S. S. Keerthi, K. B. Duan, S. K. Shevade, et al. A fast dual algorithm for kernel logistic regression[J]. Machine Learning, 2005, 61(15):151-165
30. M. Browne. Cross-validation methods[J]. Journal of Mathematical Psychology, 2000, 44(1):108-132
31.邵信光,杨慧中.一种基于信息几何的模型选择新标准[J].江南大学学报, 2006, 5(4):379-382
32. K. Kobayashi, F. Komaki. Information criteria for support vector machines[J]. Neural Networks, 2006, 17(3):571-577
33. Y. Q. Zhan, D. G. Shen. Design efficient support vector machine for fast classification[J]. Pattern Recognition, 2005, 38(1):157-161
34. Y. Q. Zhan, D. G. Shen. An adaptive error penalization method for training an efficient and generalized SVM[J]. Pattern Recognition, 2006, 39(3):342-350
35. E. Osuua, R. Freund, F. Girosi. Training support vector machines: An application to face detection[C]. In: Proceedings of IEEE Conference on Compute Vision, 1998, 555-562
36. C. J. Burges. A tutorial on support vector machines for pattern recognition[J]. Data Mining and Knowledge Discovery, 1998, 2(2):121-167
37. N. Matic, I. Guyon, J. Denker. Writer adaptation for online handwritten character recognition[C]. In: Proceedings of the 2nd International Conference on Document Analysis and Recognition, 1993, 187-191
38. V. Vapnik, S. Golowich, A. Smola. Support vector method for function approximation, regression estimation, and signal processing[C]. In: Neural information Processing Systems, 1997, 9-16
39. R. Burbidge, M. Trotter. Drug design by machine learning:support vector machine for pharmaceutical data analysis[J]. Computer and Chemistry, 2001, 24:5-14
40. J. B. Bi. Support vector regression with application in automated drug discovery [D]: Ph.D. Thesis of Rensselaer Polytechnic Institute, Troy, New York, 2003
41. H. Yang, L. Chan, I. King. Support vector machines for regression for volayile stock market prediction[C]. In: Proceedings of Intelligent Data Engineering and Automated Learning, 2002, 319-396
42. T. Francis and L. J. Cao.ε-Descending support vector machine for financial time series forecasting[J]. Neural Processing Letters, 2002, 15(2):179-195
43. L. J. Cao, T. Francis. Financial forecasting using support vector machines[J]. Neural Computing & Applications, 2001, 10(2):184-192
44.王定成,方廷健.支持向量机回归在线建模及应用[J].控制与决策, 2003, 18(1):89-95
45.冯端,沈伟,张艳珠,邵惠鹤.基于F-SVMs的多模型建模方法[J].控制与决策, 2003, 18(6):646-650
46.邵信光,杨慧中,石晨曦.ε不敏感支持向量回归在化工数据建模中的应用[J].东南大学学报, 2004, 34(sup.):215-218
47.朱国强,刘士荣,俞金寿.支持向量机及其在函数逼近中的应用[J].华东理工大学学报, 2002, 28(5):555-559
48. J. Shawe, N. Cristianini,赵玲玲,翁苏明,曾华军等译.模式识别的核方法[M].北京:机械工业出版社, 2006
49. P. Laskov. Feasible direction decomposition algorithms for training support vector machines[J]. Machine Learning, 2002, 46(1):315-349
50. R. Collobert. SVMTorch:support vector machines for large-scale regression problems[J]. Journal of Machine Learning Research, 2001, (1):143-160
51. P. Laskov. An improved decomposition algorithm for regression support vector machines[C]. In: Advances in Neural Information Processing System, 2000, 484-490
52. S. Keerithi, S. Shevade, C, Bhattacharyya, et al. A fast iterative nearest point algorithm for support vector machine classifier design[R]. In: Technical report of Indian Institute of Science, Department of Compute Science and Automation, 1999
53.张文生,丁辉,王珏.基于领域原理计算海量数据支持向量的研究[J].软件学报, 2001, 12(5):711-720
54.汪西莉,焦李成.一种基于马氏距离的支持向量快速提取算法[J].西安电子科技大学学报, 2004, 31(4):639-643
55.张金泽,单甘霖.模糊超球支持向量机[J].军械工程学院学报, 2005, 17(3):65-67
56. D. Newman, J. Hettich, C. L. Blake, et al. UCI machine learning repository. www.ics.uci.edu/~learn/MLRepository.html.1998
57.阎威武,朱宏栋,邵惠鹤.基于最小二乘支持向量机的软测量建模[J].系统仿真学报, 2003, 15(10):1494-1496
58. Y. Freund, R. E. Schapire. A decision-theoretic generalization of online learning and an application to boosting[J]. Journal of Computer and system Science, 1997, 55(1):119-139
59. Y. Freund, R. E. Schapire. Experiments with a new boosting algorithm[C]. In: Proceeding of the 13th International Conference on Machine Learning, 1996, 148-156
60. G.. Ridgeway, D. Madigan, T. Richardson. Boosting methodology for regression problems[C]. In: Proceedings of the 7th International Workshop on Artificial Intelligence and Statistics. San Francisco, 1999, 152-161
61. D. P. Solomatine, D. L. Shrestha. Adaboost.RT: a boosting algorithm for regression problems[C]. In: 2004 IEEE International Joint Conference on Neural Networks, 2004, 1163-1168
62. J. Q. Liu, T. Q. Zhang. A method determining parameters of SVR model based on Probability and Statistics[C]. In: Proceedings of the 6th World Congress on Intelligent Control and Automation, 2006, 1553-1557
63. S. Keerthi, V. Sindhwani, O. Chapelle. An efficient method for gradient-based adaption of hyperparameter in SVM models[C]. In: 20th Annual Conference on Neural Information Processing Systems, 2006
64. C. Chang, C. Lin.. LIBSVM: a library for support vector machines. http://www.csie.ntu.edu.tw/~cjlin/libsvm/
2.韩力群.人工神经网络理论、设计与应用[M].北京:化学工业出版社, 2002
3. N. Vladimir, V. Vapnik. An overview of statistical learning theory[J]. IEEE Transaction on Neural Networks, 1999, 10(5):988-999
4.张学工.关于统计学习理论与支持向量机[J].自动化学报, 2000, 26(1):32-42
5.邓乃杨,田英杰.数据挖掘中的新方法——支持向量机[M].北京:科学出版社, 2004
6. V. Vapnik. The nature of statistical learning theory[M]. 2nd edition. New York: Springer-Verlag, 1999
7. V. Vapnik.统计学习理论的本质[M].第2版.张学工译.北京:清华大学出版社, 2000
8.邵信光.基于回归型支持向量机的数据建模方法研究[D]:硕士学位论文.无锡:江南大学通信与控制工程学院, 2006
9. E. Osuna, R. Freund, F Girosi. An improved training algorithm for support vector machine[C]. In: Proceedings of the 1997 IEEE Workshop on Neural Networks and Signal Processing, 1997, 276-285
10. J. Platt. Fast training of support vector machines using sequential minimal optimization[C]. In: Advances in Kernel Method: Support Vector Learning, 1999, 185-208
11. J. Suykens, J. Vandewalle. Least squares support vector machines[J]. Neural Processing Letters, 1999, 9(3):293-300
12. Y. J. Lee, O. L. Mangasarian. SSVM:A smooth support vector machine for classification[J]. Computational Optimization and Applications, 2001, 20(1):5-22
13. Y. J. Lee, O. L. Mangasarian. RSVM:Reduced support vector machines[C]. In: Proceedings of the 1st SIAM International Conference on Data Mining, 2001
14. M. B. Almeida, A. P. Braga, J. P. Braga. SVM-KM:speeding SVMs learning with a priori cluster selection and k-means[C]. In: Proceedings of the 6th Brazilian Symposium on Neural Networks, 2000, 162-167
15. N. Ducdung, T. B. Ho. A bottom-up method for simplifying support vector solution[J]. IEEE Transactions on Neural Networks, 2006, 17(3):792-796
16. G. Ratsch, T. Onoda, K. Muller. Soft margins for AdaBoost[J]. Machine Learning, 2001, 42(3):287-320
17. O. Chappelle, V. Vapnik, O. Bousquet, et al. Choosing multiple parameters for support vector machines[J]. Machine Learning, 2002, 46(1):131-160
18. P. W. Chen, J. Y. Wang, H. M. Lee. Model selection of SVMs using GA approach[A]. In: Proceedings of2004 IEEE International Joint Conference on Neural Networks, 2004, 3:2035-2040
19. C. H. Zheng, L. C. Jiao. Automatic parameters selection for SVM based on GA[C]. In: Proceedings of the 5th World Congress on Intelligent Control and Automation, 2004, 1869-1872
20.邵信光,杨慧中,陈刚.基于粒子群优化算法的支持向量机参数选择及应用[J].控制理论及应用, 2006, 23(5):740-743
21. B. Efron, T. Hastie, I. Johnstone, et al. Least angle regression[J]. Annals of Statistics, 2004, 32(2):407–499
22. T. Hastie, S. Rosset, R. Tibshirani. The entire regularization path for the support vector machine[J]. Journal of Machine Learning Research, 2004, (5):1391–1415
23. G. Wang, D. Yeung, F. Lochovsky. Two-dimensional solution path for support vector regression[C]. In: Proceedings of the 23th International Conference on Machine Learning, 2006, 993-1000
24. G. Wang, D. Yeung, F. Lochovsky. A kernel path algorithm for support vector machines[C]. In: Proceedings of the 24th International Conference on Machine Learning, 2007, 951-958
25. B. Sch?lkoph, A. Smola, K. Muller. Nonlinear component analysis as a kernel eigenvalue problem[J]. Neural Computation, 1998, 10(5):1299-1319
26. H. Gábor. Kernel CMAC with improved capability[C]. In: Proceedings of the International Joint Conference on Neural Networks, 2004, 681-586
27.张莉,周传达,焦李成.核聚类算法[J].计算机学报, 2002, 25(6):587-590
28. M. Girolami. Mercer kernel-based clustering in feature space[J]. Neural Networks, 2002, 13(3):780-784
29. S. S. Keerthi, K. B. Duan, S. K. Shevade, et al. A fast dual algorithm for kernel logistic regression[J]. Machine Learning, 2005, 61(15):151-165
30. M. Browne. Cross-validation methods[J]. Journal of Mathematical Psychology, 2000, 44(1):108-132
31.邵信光,杨慧中.一种基于信息几何的模型选择新标准[J].江南大学学报, 2006, 5(4):379-382
32. K. Kobayashi, F. Komaki. Information criteria for support vector machines[J]. Neural Networks, 2006, 17(3):571-577
33. Y. Q. Zhan, D. G. Shen. Design efficient support vector machine for fast classification[J]. Pattern Recognition, 2005, 38(1):157-161
34. Y. Q. Zhan, D. G. Shen. An adaptive error penalization method for training an efficient and generalized SVM[J]. Pattern Recognition, 2006, 39(3):342-350
35. E. Osuua, R. Freund, F. Girosi. Training support vector machines: An application to face detection[C]. In: Proceedings of IEEE Conference on Compute Vision, 1998, 555-562
36. C. J. Burges. A tutorial on support vector machines for pattern recognition[J]. Data Mining and Knowledge Discovery, 1998, 2(2):121-167
37. N. Matic, I. Guyon, J. Denker. Writer adaptation for online handwritten character recognition[C]. In: Proceedings of the 2nd International Conference on Document Analysis and Recognition, 1993, 187-191
38. V. Vapnik, S. Golowich, A. Smola. Support vector method for function approximation, regression estimation, and signal processing[C]. In: Neural information Processing Systems, 1997, 9-16
39. R. Burbidge, M. Trotter. Drug design by machine learning:support vector machine for pharmaceutical data analysis[J]. Computer and Chemistry, 2001, 24:5-14
40. J. B. Bi. Support vector regression with application in automated drug discovery [D]: Ph.D. Thesis of Rensselaer Polytechnic Institute, Troy, New York, 2003
41. H. Yang, L. Chan, I. King. Support vector machines for regression for volayile stock market prediction[C]. In: Proceedings of Intelligent Data Engineering and Automated Learning, 2002, 319-396
42. T. Francis and L. J. Cao.ε-Descending support vector machine for financial time series forecasting[J]. Neural Processing Letters, 2002, 15(2):179-195
43. L. J. Cao, T. Francis. Financial forecasting using support vector machines[J]. Neural Computing & Applications, 2001, 10(2):184-192
44.王定成,方廷健.支持向量机回归在线建模及应用[J].控制与决策, 2003, 18(1):89-95
45.冯端,沈伟,张艳珠,邵惠鹤.基于F-SVMs的多模型建模方法[J].控制与决策, 2003, 18(6):646-650
46.邵信光,杨慧中,石晨曦.ε不敏感支持向量回归在化工数据建模中的应用[J].东南大学学报, 2004, 34(sup.):215-218
47.朱国强,刘士荣,俞金寿.支持向量机及其在函数逼近中的应用[J].华东理工大学学报, 2002, 28(5):555-559
48. J. Shawe, N. Cristianini,赵玲玲,翁苏明,曾华军等译.模式识别的核方法[M].北京:机械工业出版社, 2006
49. P. Laskov. Feasible direction decomposition algorithms for training support vector machines[J]. Machine Learning, 2002, 46(1):315-349
50. R. Collobert. SVMTorch:support vector machines for large-scale regression problems[J]. Journal of Machine Learning Research, 2001, (1):143-160
51. P. Laskov. An improved decomposition algorithm for regression support vector machines[C]. In: Advances in Neural Information Processing System, 2000, 484-490
52. S. Keerithi, S. Shevade, C, Bhattacharyya, et al. A fast iterative nearest point algorithm for support vector machine classifier design[R]. In: Technical report of Indian Institute of Science, Department of Compute Science and Automation, 1999
53.张文生,丁辉,王珏.基于领域原理计算海量数据支持向量的研究[J].软件学报, 2001, 12(5):711-720
54.汪西莉,焦李成.一种基于马氏距离的支持向量快速提取算法[J].西安电子科技大学学报, 2004, 31(4):639-643
55.张金泽,单甘霖.模糊超球支持向量机[J].军械工程学院学报, 2005, 17(3):65-67
56. D. Newman, J. Hettich, C. L. Blake, et al. UCI machine learning repository. www.ics.uci.edu/~learn/MLRepository.html.1998
57.阎威武,朱宏栋,邵惠鹤.基于最小二乘支持向量机的软测量建模[J].系统仿真学报, 2003, 15(10):1494-1496
58. Y. Freund, R. E. Schapire. A decision-theoretic generalization of online learning and an application to boosting[J]. Journal of Computer and system Science, 1997, 55(1):119-139
59. Y. Freund, R. E. Schapire. Experiments with a new boosting algorithm[C]. In: Proceeding of the 13th International Conference on Machine Learning, 1996, 148-156
60. G.. Ridgeway, D. Madigan, T. Richardson. Boosting methodology for regression problems[C]. In: Proceedings of the 7th International Workshop on Artificial Intelligence and Statistics. San Francisco, 1999, 152-161
61. D. P. Solomatine, D. L. Shrestha. Adaboost.RT: a boosting algorithm for regression problems[C]. In: 2004 IEEE International Joint Conference on Neural Networks, 2004, 1163-1168
62. J. Q. Liu, T. Q. Zhang. A method determining parameters of SVR model based on Probability and Statistics[C]. In: Proceedings of the 6th World Congress on Intelligent Control and Automation, 2006, 1553-1557
63. S. Keerthi, V. Sindhwani, O. Chapelle. An efficient method for gradient-based adaption of hyperparameter in SVM models[C]. In: 20th Annual Conference on Neural Information Processing Systems, 2006
64. C. Chang, C. Lin.. LIBSVM: a library for support vector machines. http://www.csie.ntu.edu.tw/~cjlin/libsvm/