摘要
统计方法是从观测自然现象或者专门安排的实验所得到的数据去推断该事物可能的规律性。统计学习理论是在研究小样本统计估计和预测的过程中发展起来的一种新兴理论，它试图从更本质上来研究机器学习问题，因此引起了人们越来越多的重视。
支持向量机(SVM)是在统计学习理论基础上发展起来的一种新的模式识别方法，它是统计学习理论中的结构风险最小化思想在实际中的一种体现。SVM的基本思想是通过非线性变换将输入空间变换到一个高维空间，然后在这个新的空间中求取最优分类超平面。它在解决小样本、非线性及高维模式识别问题中表现出许多特有的优势，并能够推广应用到函数拟合等其他机器学习问题中。本文选取SVM及其如何在控制领域取得应用加以研究，希望理论与应用并重。因此本文主要由两大部分构成：其一是关于SVM本身的研究，提出了新的SVM；其二是关于应用的研究，包括如何在最优控制中取得应用、如何构造SVM软测量仪及在故障诊断中取得应用。
本文的主要贡献如下：
1．回顾了统计学习理论研究的基本问题及主要内容。为了更好地说明统计学习理论在实际中的实现问题，我们回顾了SVM的基本概念及基本理论。然后介绍了SVM的发展和国内外研究现状，主要从SVM本身发展，SVM的算法，SVM的应用三方面进行了回顾，对每一方面目前存在的问题进行了总结。
2．针对回归估计问题提出了模糊SVM。模糊SVM主要用于解决标准SVM不能很好处理噪声污染的数据样本问题。把针对分类问题的广义SVM及最小二乘广义SVM用来处理回归估计问题，并和模糊SVM结合起来形成了基于模糊加权的广义SVM和基于模糊的多层最小二乘广义SVM。
3．针对多类回归模型估计问题提出了基于SVM的模糊C聚类算法，这种方法可以对样本进行聚类的同时实现对多个回归模型的估计；同时提出了基于最小二乘支持向量机(LS-SVM)的模糊C聚类算法。在讨论多类回归模型估
浙江大学博士学位论文
计问题的基础上又针对多个输出的问题讨论了SVM和LS一SVM如何实现的
问题。
4.加权LS一SVM可以对回归估计问题实现鲁棒估计，把它同基于LS一SVM的N
步最优控制想法相结合，将其扩展到基于加权最小二乘广义SVM，并应用
于N步最优控制问题，确定加权因子采用了模糊运算的办法。
5.针对微生物发酵中需要有大量软测量仪表的问题，提出了基于SVM的软测
量仪，针对离线和在线问题分别讨论了SVM软测量仪和LS一SVM软测量
仪的实现问题。sVM和Ls一sVM软测量仪实际上也是一种智能软测量仪
表。
6.研究了svM如何在故障诊断中应用的问题，提出了一种微生物发酵的故障
诊断新方法，即两个关联向量机分别作为观测器和分类器。观测器用于估计
二氧化碳释放率以便得到残差序列，分类器用于对残差序列进行分类。为了
减少染菌所造成的影响和损失，对异常工况进行及时诊断显得尤为重要。在
这里采用了另一种故障诊断方法，即把主元分析同支持向量机结合起来，这
样既可以从过多的监测变量中提取出主要的监测变量，又可以从有限的故障
样本得到具有较强推广能力的决策函数。
7.最后对全文进行了概括性总结，并指出了理论和应用上有待进一步研究的方
向。
Statistics is to inference the law of nature according to observation data. Statistical learning theory is a newly developed theory for studying the statistical estimation and prediction problem based on small number of samples. It studies the nature of machine learning, so more and more people are interested in it.
Support vector machine (SVM) is a new method for pattern recognition based on the statistical learning theory. It is an implementation of structure risk minimization principle in the statistical learning theory. By mapping input data into a high dimensional characteristic space in which an optimal separating hyperplane is built, SVM presents a lot of advantages for resolving the small samples, nonlinear and high dimensional pattern recognition, as well as other machine-learning problems such as function fitting. This thesis studies SVM and its applications in control. It consists of two parts, one is the study on SVM in which new SVMs are proposed, and the other is the application in optimal control, soft sensor construction and fault diagnosis.
In detail, the major contributions of this thesis are as following:
1. Research contributions and major problems in statistical learning theory study are reviewed. In order to explain the implementation problem of statistical learning theory, basic concepts and theory of the SVM ?including the developments and research situation of SVM itself, SVM algorithms and applications ~ are summarized, and problems in the research for each aspect are put forward.
2. Fuzzy SVMs for regression estimation are developed in Chapter 2. They are used to resolve the problem that ordinary SVM could not deal with contaminated samples well. By applying generalized SVM and least square SVM of classification to regression estimation problem, fuzzy generalized weighted SVM and fuzzy multiplayer least square generalized SVM are proposed.
3. Fuzzy C-clustering SVM and least square SVM for multiclass regression
estimations is presented in Chapter 3, This method can estimate multiple regressions while clustering the samples. Multi-output SVM and multi-output least square SVM are discussed for multiclass regression estimations.
4. Chapter 4 introduces the use of weighted least square generalized SVM for optimal control systems. Weighted least square SVM for robust regression estimation is generalized and then used to solve N-step optimal control problem based on the SVM. The weights are determined by fuzzy method.
5. To meet the needs of various soft sensors in microbiological fermentation process, new kind of soft sensors based on SVM are proposed in Chapter 5. SVM based soft sensor and least square SVM based soft sensor for off-line and on-line estimation are discussed respectively.
6. Chapter 6 studies how SVMs are applied to fault diagnosis. A new method of fault diagnosis for microbiological fermentation process is presented, two relevant vector machines act as observer and classifier respectively. The observer is applied to estimate release rate of carbon dioxide to get residual sequence. The classifier is applied to classify the residual sequence. In order to reduce the loss arisen by polluted mycelia, it is very important to diagnose abnormal states in time. We adopt another fault diagnosis method for the checking of polluted mycelia in the microbiological fermentation process. The method combines nonlinear principal components analysis technology with support vector machines to construct multi-layer support vector machines. The multi-layer support vector machines are able to extract main monitoring variables from many process variables, also obtain decision function with excellent generalization performance from limited samples of fault.
Finally, a brief review of this thesis is given. Some future research directions are highlighted.
引文
Alexander D. Ioffe, Alexander J. Zaslavski. Variational principles and well-posedness in optimization and calculus of Variations. SIAM J. Control and Optimization, Vol.38 (2): pp. 566-581, 2000.
Anguita A., S. Ridella, S. Rovetta. Circuital implementation of support vector machines. Electronics Letters, 1998, 34(16): p1596-1597.
Assis A. J. D., Filho R. M.. Soft sensors development for on-line bioreactor state estimation. Computers and Chemical Engeering, Vol. 24:pp1099-1103, 2000.
Basu A., Watters C.. Shepherd, M.; Support vector machines for text categorization.System Sciences, 2003. Proceedings of the 36th Annual Hawaii International Conference on, 6-9 Jan 2003 Page(s): 103-109.
Bennett K. P., J. A. Blue. A support vector machine approach to decision trees. In Proceedings of IJCNN'98, pages 2396-2401, Anchorage, Alaska, 1997.
Berger J. O. Statistical Decision Theory and Bayesian Anaysis. Second edition,Spring-Verlag, 1985.
Bezdek J. C.. Pattern recognition with fuzzy object function algorithms. Plenum,New York, 1981.
Bishop C. M. and M. E. Tipping. Variational relevance vector machines. In C.Boutilier and M. Goldszmidt, editors, Proceedings of the 16th Conference on Uncertainty in Articial Intelligence, pages 46-53. Morgan Kaufmann, 2000.
Blanz V., B. Schlkopf, H. Bülthoff, C. J. C. Burges, V. Vapnik, T. Vetter.Comparison of view-based object recognition algorithms using realistic 3D models. In: vonder Malsburg C, von Seelen W, J. C. Vorbrüggen, B. Sendhoff, editors, Artificial Neural Networks—ICANN'96, Springers Lecture Notes in Computer Science, Vol. 1112:p251-256, Berlin, 1996.
Bradley E S., O. L. Mangasarian. Massive data discrimination via linear support vector machines. Technical Report 98-05, Computer Sciences Department,
University of Wisconsin, Madison, Wisconsin, May 1998.
Brown M., Gunn S.R.. Empirical data modelling algorithms: additive spline models and support vector machines. Control'98, UKACC International Conference on (Conf. Publ. No. 455), Vol.l: p709-714, 1-4 Sep 1998.
Brown, M.; Lewis, H.G.; Gunn, S.R.; Linear spectral mixture models and support vector machines for remote sensing. Geoscience and Remote Sensing, IEEE Transactions on, Vol.38 (5): p2346-2360, Sep 2000.
Bryson A.E., Ho Y.C.. Applied optimal control, Waltham, MA: Btaisdel 1969.
Burbidge R., B. Buxton, An Introduction to Support Vector Machines for Data minining. Keynote Papers, Young OR 12, University of Nottingham, 2001, 3-15. M. Sheppee (ed), Operational research Society.
Burges C. J. C. A tutorial on support vector machines for pattern recognition, Data Mining and Knowledge Discovery, Vol. 2 (2), 1998.
Caudill S. B., R. N. Acharya. Maximum-Likelihood estimation of a mixture of normal regressions: starting values and singularities. Comm. Statistics-Simulation, Vol.27 (3): p667-674, 1998.
Cauwenberghs G. and Poggio T. Incremental and decremental support vector machine. Machine Learning, Vol. 13,2001.
Chakrabartty S., Singh G., Cauwenberghs G. Hybrid support vector machine/hidden Markov model approach for continuous speech recognition. Circuits and Systems, 2000. Proceedings of the 43rd IEEE Midwest Symposium on, Vol.2: p828-831, 2000.
Challa S., Palaniswami M., Shilton A.. Distributed data fusion using support vector machines. Information Fusion, 2002. Proceedings of the Fifth International Conference on, Vol.2:p881-885, 2002.
Chang C. C., Hsu C. W., Lin C. J.. The analysis of decomposition methods for support vector machines. In Workshop on Support Vector Machines, IJCAI, 1999.
Chang C. C., Lin C. J. Some analysis on nu-support vector classification. Technical report, National Taiwan University, 1999.
Chen Datong, Hervd Bourlard, Jean-Philippe Thiran. Text Identification in Complex Background using SVM, IDIAP-RR 01-20, 2001.
Chen Datong and Jean-Marc Od0bez. Comparison of Support Vector Machine and Neural Network for Text Texture Verification, IDIAP-RR 02-19, 2002.
Chen S., S. R. Gunn, C. J. Harris. The relevance vector machine technique for channel equalization application. IEEE Transactions on Neural Networks, Vol.12(4): p1529-1532, Nov. 2001.
Chih W. H. ctal. A simple decomposition method for support vector machines. Technical report, National Taiwan University, 1999.
Colin W. Cryer. The solution of a quadratic programming problem using systematic overrelaxation. SIAM J. Control Vol. 9, No.3, August 1971.
Cortes C., V. Vapnik. Support vector networks. Machine Learning, Vol.20 (3): p273-297, 1995.
Crisp D. J., C. J. C. Burges, A geometric interpretation of nu-SVM classifiers. NIPS 12: p244-250, 2000.
Cristianini N., Campbell C., John Shawe-Taylor. Multiplicative updatings for support-vector learning. NeuroCOLT2 Technical Report Series, NC2-TR-1998-016.
David H. A.. Early sample measures of variability. Statistical Science, Vol.13 (4): pp.368-377, 1998.
De Veaux R. D.. Mixtures of linear regressions. Computational Statistical and Data Analysis, Vol. 8: pp227-245, 1989.
Dimitri P.Bertsekas, John N.Tsitsiklis. Parallel and distributed computation: numerical methods. Prentice Hall, Englewood Cliffs, p210—217, 1989.
Drezet P. M. L., Harrison R. F.. Support vector machines for system identification. Control'98, UKACC International Conference on (Conf. Publ. No. 455), Vol.1 (1-4): p688 -692, Sep. 1998.
Fabien Cardinaux and Sébastien Marcel. Face Verification using MLP and SVM. IDIAP-RR 02-21, 2002.
Fletcher R.. Practical methods of optimization. Chichester and New York: John
Wiley and Sons. 1987
Gestel T. V., Johan A. K. Suykens, Dirk-Emma Baestaens, A. Lambrechts, G. Lanckriet, B. Vandaele, B. D. Moor, J. Vandewalle. Financial time series prediction using least squares support vector machines within the evidence framework. IEEE Transactions on Neural Networks, Vol. 12(4) July 2001.
Gestel T. Van, J. A. K. Suykens, J. De Brabanter, B. De Moor, J. Vandewalle. Kernel canonical correlation analysis and least squares support vector machines. In Proceedings of the International Conference on Artificial Neural Networks (ICANN) 2001a.
Gordan M., Kotropoulos C., Pitas I.. Visual speech recognition using support vector machines. Digital Signal Processing, 2002 (DSP 2002). 2002 14th International Conference on, Vol.2:p1093 -1096, 2002.
Gretton A., Doucet A., Herbrich R., Rayner P. J. W., Schlkopf B.. Support vector regression for black-box system identification. Statistical Signal Processing, 2001. Proceedings of the 11th IEEE Signal Processing Workshop on, 2001. Page(s): 341-344.
Gutschoven B., P. Verlinde. Multi-modal Identity Verification Using Support Vector Machines. In Proceedings of the 3rd International Conference on Information Fusion. Paris, France, July 2000.
Hampel F. R., Ronchetti E. M., Rousseeuw P. J., Stahel W A.. Robust statistics: the approach based on inference functions, John Wiley &Sons, New York, 1986.
Hathaway R. J., Bezdek J. C.. Switching regression models and fuzzy clustering. Fuzzy Systems, IEEE Transactions on, Vol. 1 (3): pp195-204, Aug 1993.
Hoppner F., Klawonn F. Fuzzy clustering of sampled functions. The 19th International Conference of the North American on Fuzzy Information Processing Society, 2000 13-15 July 2000. Page(s): 251-255.
Hu Jun, Huang Houkuan. An algorithm for text categorization with SVM. TENCON'02. Proceedings. 2002 IEEE Region 10 Conference on Computers, Communications, Control and Power Engineering, Vol.1: pp47-50, 2002.
Inoue T., Abe S.. Fuzzy support vector machines for pattern classification. Neural
Networks, 2001. Proceedings. IJCNN'01. International Joint Conference on,Vol. 2:pp1449-1454, 2001.
Ivanov V. V.. On linear problems which are not well-posed. Soviet Math. Doct., 3(4): pp981-983, 1962
Joachims T.. Making large-scale support vector machine learning practical. In B. Schlkopf, C. J. C. Burges, A. J. Smola, editors, Advances in Kernel Methods-Support Vector Learning, pp170-184, MIT Press, 1998.
Joaquin Q. -C., Hansen L.K..Time series prediction based on the relevance vector machine with adaptive kernels. Acoustics, Speech, and Signal Processing, 2002 IEEE International Conference on, Vol.1: p985-988, 2002.
Keerthi S, S., S. K. Shevade, C. Bhattacharyya, K. R. K. Murthy. Improvements to Platt's SMO Algorithm for SVM Classifier Design, TR CD-99-14, Department of Mechanical and Production Engineering, National University of Singapore, 1999.
Keerthi S. S. et al. A fast iterative nearest point algorithm for support vector machine classifier design. TR-ISL-99-03 Dept. of CS and Auto. Indian Institute of Scince Bangalore, India, 1999a.
Keerthi S.S., E.G. Gilbert. Convergence of a generalized SMO algorithm for SVM classier design. Technical Report CD-00-01, Control Division, Dept. of Mechanical and Production Engineering, National University of Singapore,2000.
Knerr S. et al Single-layer learning revisited: a stepwise procedure for building and training a neural network. In Fogelman Soulie et al. (ed.) Neurocomputing: Algorithms, Architectures and Applications, NATO ASI, Springer, 1990.
Kordon A., Smits G, Jordaan E., Rightor E. Robust soft sensors based on integration of genetic programming, analytical neural networks, and support vector machines. Proceedings of the 2002 Congress on Evolutionary Computation, Vol. 1: pp896-901, 12-17 May 2002.
Kreβel U. H. -G.. Pairwise classification and support vector machines. In B. Schlkopf, C. J. C. Burges, A. J. Smola, editors, Advances in Kernel Methods
-Support Vector Learning, p255-268, MIT Press, 1998.
Krishnapuram B., Carin L. Support vector machines for improved multiaspect target recognition using the Fisher kernel scores of hidden Markov models. Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing, Vol. 3: pp2989-2992, 13-17 May 2002.
Kwok J. T. -Y.. The evidence framework applied to support vector machines. IEEE Transactions on Neural Networks, Vol. 11(5), September 2000.
Lee W. F., Sekhar C. C., Takeda K., Itakura F.. Multiple regression using support vector machines for recognition of speech in a moving car enviroment. Lipo Wang, Jagath C. Rahapakse, Kunihiko Fukushima, Soo-Young Lee, and Xin Yao, ed., Proceedings of the 9th International Conference on Neural Information Processing (ICONIP'02), Vol.2: pp904-908.
Li Yongmin, Shaogang Gong, Heather Liddell. Support vector regression and classification based multi-view face detection and recognition. In IEEE International Conference on Automatic Face & Gesture Recognition, pages 300-305, Grenoble, France, March 2000.
Lin Chun-Fu, Sheng-De Wang. Fuzzy support vector machines. IEEE Transaction on Neural Networks. Vol. 13 (2), March 2002.
Lin Y.. Support vector machines and the Bayes rule in classification. Technical Report 1014. Department of Statistics, University of Wisconsin, Madison, 1999.
Luo Zhi-Quan, Pual Tseng. On the convergence of a matrix splitting algorithm for the symmetric monotone linear. SIAM J. Control and Optimization, Vol. 29(5), pp. 1037-1060, September 1991.
Nello Cristianini, Colin Campbell, John Shawe-Taylor Multiplicative updatings for support-vector learning. Technical report, NC2-TR-1998-016, June, 1998.
Ma Changxue, Randolph M. A., Drish J.. A support vector machines-based rejection technique for speech recognition. Acoustics, Speech, and Signal Processing, 2001. Proceedings of 2001 IEEE International Conference on, Vol.1: p381-384, 2001.
Mackay D. J. C Bayesian non-linear modelling for the prediction competition.ASHRAE Transactions, 1994.
Mason J. C., D. A. Turner. Applications of Support Vector Machine Regression in Metrology and Data Fusion. http://helios.hud.ac.uk/scomdat/MasTur00.ps.
Masuyama T.. Nakagawa H.. Applying cascaded feature selection to SVM text categorization. Database and Expert Systems Applications, 2002. Proceedings.13th International Workshop on, 2002, Page(s): 203-207.
Mattera D., S. Haykin. Support vector machines for dynamic reconstruction of a chaotic system. In B. Schlkopf, C. J. C. Burges, A. J. Smola, editors, Advances in Kernel Methods-Support Vector Learning, p212-241, MIT Press, 1998.
McCormick G. P., R. A. Tapia. The gradient projection method under mild differentiability conditions. SIAM J. Control, Vol. 10(1), February 1972.
Mckay D., Fyfe C. Probability prediction using support vector machines. Proceedings of the fourth International Conference on Knowledge-based intelligent engineering systems & Allied Technologies, 30th Aug.-1st Sep. 2000, Brighton UK.
Melgani F., Bruzzone L.. Support vector machines for classification of hyperspectral remote-sensing images. Geoscience and Remote Sensing Symposium, 2002. IGARSS'02. 2002 IEEE International, Vol. 1 :p506-508, 2002.
Menard M., Dardignac P. A., Courboulay V.. Switching regression models using ambiguity and distance rejects: application to ionogram analysis. Proceedings of Pattern Recognition, 2000. 15th International Conference on, Vol. 2:pp688-691, 2000.
Mohamed N. N., Bhavik R. B., Prem K. Geol, Shen X. T. Bayesian principal component analysis. Journal of Chemometrics, Vol. 16 (11): pp576-595.2002.
Morris A. J., Montague G. A., Tham M. T. Soft-sensors in industrial process control. IEE Colloquium on Applied Developments in Process Control, 1 Mar 1989, Page(s): 1/1-1/3.
Mukherjee S., Osuna E., Girosi F.. Nonlinear prediction of chaotic time series using
support vector machines. Neural Networks for Signal Processing[1997], VII.Proceedings of the 1997 IEEE Workshop: p511-520, 24-26 Sep 1997.
Müller K. -R., A. J. Smola, G. Rtseh, B. Sehlkopf, J. Kohtmorgen, V. Vapnik.Using support vector machines for time series prediction. In B. Schlkopf, C. J.C. Burges, A. J. Smola, editors, Advances in Kernel Methods-Support Vector Learning, p243-253, MIT Press, 1998.
Nakayama H., Asada T. Support vector machines using multi objective programming and goal programming. Lipo Wang, Jagath C. Rahapakse, Kunihiko Fukushima, Soo-Young Lee, and Xin Yao, ed., Proceedings of the 9th International Conference on Neural Information Processing (ICONIP'02),Vol.2: pp1053-1057.
Olvi L. M., David R.. Successive overrelaxation for support vector machines. IEEE Transactions on Neural Networks, Vol. 10(5): p1032-1037, 1999.
Olvi L. Mangasarian, R. R. Meyer. Nonlinear perturbation of linear programs. SIAM J. Control and Optimization, Vol. 17(6), November 1979.
Olvi L. Mangasarian. Solution of symmetric linear complementarity problems by iterative methods. Journal of Optimization Theory and Applications, Vol.22(4), August 1977.
Olvi L. Mangasarian. Data discrimination via nonlinear generalized support vector machines. Computer Sciences Department, University of Wisconsin, Technical report, March 1999.
Olvi L. Mangasarian, David R. Musicant. Robust linear and support vector regression. IEEE Transactions on Pattern Analysis and Machine Intelligence:Vol.22(9), Sep. 2000.
Opper M. On the annealed VC entropy for margin classifiers: a statistical mechanies study. In B. Schlkopf, C. J. C. Burges, A. J. Smola, editors, Advances in Kernel Methods-Support Vector Learning, p117-126, MIT Press, 1998.
Osuna E., R. Freund, F. Girosi. Training support vector machines: An application to face detection. In Proceedings of CVPR'97. Puerto Rico, 1997a.
Osuna E., R. Freund, F. Girosi. An improved training algorithm for support vector
machines. In: Principe J., Gile L., Morgan N., Wilson E., editors, Proceedings of the 1997 IEEE Workshop on Neural Networks for Signal Processing, New York: IEEE, 1997: 276-285.
Pal N. R., Pal K., Bezdek J. C. A mixed c-means clustering model. Proceedings of the Sixth IEEE International Conference on Fuzzy Systems, Vol. 1: pp 11-21, 1-5 July 1997.
Panchapakesan S., N. Balakrishnan. Advances in Statistical Decision Theory and Applications. Birkhuser Boston, 1997.
Park S., Han C. A nonlinear soft sensor based on multivariat.e smoothing procedure for quality estimation in distillation columns. Computers and Chemical Engineering, Vol. 24: pp871-877, 2000.
Parzen E. On estimation of probability function and mode. Annals of Mathematical Statistics, 33(3) 1962
Pereira C., Dourado A. On the complexity and interpretability of support vector machines for process modeling. Proceedings of the 2002 International Joint Conference on Neural Networks, Vol. 3:pp2204-2209, 12-17 May 2002.
Philips D. Z. A technique for numerical solution of certain integral equation of the first kind. J. Assoc. Comput. Math., Vol. (9) :pp84-96. 1962.
Platt J. C.. Fast training of support vector machines using sequential minimal optimization. In B. Schlkopf, C. J. C. Burges, A. J. Smola, editors, Advances in Kernel Methods-Support Vector Learning, p185-208, MIT Press, 1998.
Platt J. C.. Using sparseness and analytic QP to speed training of support vector machines. In Kearns M. S., Solla S. A., Cohn D. A., editors, Advances in Neural Information Processing Systems, Vol.11, Cambridge, MA: MIT Press,1999.
Platt J. C. et al. Large Margin DAGs for multiclass classification. In Advances in Neural Information Processing Systems, Vol. 12, MIT Press, pp547-553, 2000.
Pontil M., Verri A.. Support vector machines for 3D object recognition. Pattern Analysis and Machine Intelligence, IEEE Transactions on, Vol.20 (6): p637-646,Jun 1998.
Quandt R E., J. B. Ramsey. Estimating mixtures of normal distributions and switching regressions. J. American Statist. Ass., Vol. 73: p730-752, 1978.
Robert V. H., Allen T. C. Introduction to mathematical statistics. Macmillan Publishing Co., Inc. New York, 1978.
Redner R. A., H. F. Walker. Mixture densities, maximum likelihood and the EM algorithm. SIAM Rev., Vol. 31 (2): p195-239, 1984.
Renan C. Samy B. Support vector machines for large-scale regression problem.IDAP-RR00-17, 2000.
Roobaert D., Van Hulle M. M.. View-based 3D object recognition with support vector machines. Neural Networks for Signal Processing Ⅸ, 1999. Proceedings of the 1999 IEEE Signal Processing Society Workshop, Aug 1999, Page(s): 77-84.
Rosenbtatt M. Remarks on some nonparametric estimation of density function. Annals of Mathematical Statistics, 27: 642～669, 1956.
Schouten T., Suykens J. A. K., De Moor B.. Multi resolution least squares SVM solver. Circuits and Systems, 2000. Proceedings of the 43rd IEEE Midwest Symposium on, Vol.3:p1187-1181, 2000.
Schlkopf B., C. Burges, V. Vapnik. Incorporating invariances in support vector learning machines. In C. vonder Malsburg, W. von Seelen, J. C. Vorbrüggen,and B. Sendhoff, editors, Artificial Neural Networks—ICANN'96:p47-52, Berlin, 1996a. Springer Lecture Notes in Computer Science, Vol. 1112.
Schlkopf B., Simard P., Alex. Smola, V. Vapnik. Prior knowledge in support vector kernels. In M. Jordan, M. Keams, and S. Solla, editors, Advances in Neural Information Processing Systems, Vol.10:p640—646, Cambridge, MA, 1998. MIT Press.
Schlkopf B., P. Bartlett, A. Smola, R. Williamson. Support vector regression with automatic accuracy control. In L. Niklasson, M. Bode, T.Ziemke, editors, Proceedings of ICANN'98, Perspectives in Neural Computing, pages 111-116, Berlin, 1998a. Springer Verlag.
Schlkopf B., A. Smola, K.-R.Müller. Nonlinear component analysis as a kernel
eigenvalue problem. Neural Component, 10: pp1299-1319, 1998.
Schlkopf B., Mika S., C. J. C. Burges, Knirsch P., Müller K. -R., G. Rtsch, Alex.Smola. Input space versus feature space in Kernel-based methods. IEEE Transaction on Neural networks, Vol.(5): pp1000-1017, Sep. 1999.
Schlkopf B. New support vector algorithms. Neural Computation, Vol. 12 (5): pp1083-1121,2000
Shigeo Abe, Takuya Inoue. Fuzzy support vector machines for multiclass problems.ESANN'2002 Proceedings-European Symposium on Artificial Neural Networks Bruges(Belgium), 24-26 April 2002, d-side public, p113-118.
Siolas G., d'Alche-Buc F.. Support Vector Machines based on a semantic kernel for text categorization. Neural Networks, 2000. IJCNN 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on, Vol.5:p205-209, 2000.
Smola Alex. J., Thilo T. Frieβ, B. Schlkopf Semiparametric Support Vector and Linear Programming Machines. In Advances in Neural Information Processing Systems, Vol. 11.1998. MIT Press.
Smola Alex. J.. Learning with kernel. PhD thesis, Technischen Universitt Berlin, Berlin, Germany, 1998.
Sollich P. Probabilistic interpretation and bayesian methods for support vector machines. In ICANN'99--Ninth International Conference on Artificial Neural Networks, pages 91-96, London, 1999.
Sollich P.. Probabilistic methods for support vector machines. In S. A. Solla, T. K. Leen, K. -R. Müller, editors, Advances in Neural Information Processing Systems 13: p349-355, 2000. MIT Press 2000.
Souheil Ben-Yacoub. Multi-Modal Data Fusion for Person Authentication using SVM. Proc. of Second International Conference on Audio-and Video-based Biometric Person Authentication (AVBPA'99), 1999.
Sun Zonghai, Youxian Sun. Using Support Vector Machines for Mining Regression Classes in Large Data Sets, IEEE Region 10 Conference on Computers, Communications, Control and Power Engineer Proceedings, 2002.
Sun Zonghai, Youxian Sun. Optimal Control by Weighted Least Squares Generalized
Support Vector Machines. Proceedings of the 2003 American Control Conference.
Sun Zonghai, Youxian Sun. Application to Microbiological Fermentation Fault Diagnosis for Relevance Vector Machine. 2003 IEEE Conference on Control Applications.
Sun Zonghai, Youxian Sun. Fuzzy Support Vector Machine for Regression Estimation. 2003 IEEE Conference on System, Man, Cybernetics.
Sun Zonghai, Youxian Sun. Microbiological fermentation fault diagnosis based on multi-layer support vector machine, 2003 IEEE International Conference on Machine Learning and Cybernetics.
Suykens J.A.K. Vandewalte J.. Chaos control using least-squares support vector machines. International Journal of circuit theory and applications, 27:p605-615,1999.
Suykens J.A.K. Vandewalle J., B. De Moor. Optimal control by squares support vector machines. Neural Networks, Vol.14(1): p23-35, January 2001.
Suykens J.A.K.. Nonlinear modelling and support vector machines. Instrumentation and Measurement Technology Conference, 2001(IMTC 2001), Proceedings of the 18th IEEE, Vol.l: p287-294, 21-23 May 2001a.
Suykens J.A.K., De Brabanter J., Lukas L., Vandewalle J.. Weighted least squares support vector machines: robustness and sparse approximation. Neurocomputing, Vol.48 (1-4): p85-105, October 2002.
Suykens J.A.K., Vandewalle J. Multiclass least squares support vector machines. In IJCNN'99 International Joint Conference on Neural Networks, Washington, DC, 1999.
Tao C. W.. Unsuperised fuzzy clustering with multi-center clusters. Fuzzy Sets and Systems, Vol. 128: pp305-322, 2002.
Tikhonov A. N.. On solving ill-posed problem and method of regularization. Doklady Akademii Nauk USSR, Vol. 153: pp501～504, 1963.
Tipping M. E. Sparse kernel principal component analysis. In Advances in Neural Information Processing Systems 13. MIT Press, 2001a.
Tipping M. E. Sparse bayesian learning and the relevance vector machine. Jounal of Machine Learning Research Vol. 1: p p211-244, 2001.
Tipping M. E. The relevance vector machine. In S. A. Solla, T. K. Leen, K.-R. Müller,editors, Advances in Neural Information Processing Systems, Vol. 12, p652-658,2000. MIT Press.
Vapnik V.. The Nature of Statistical Learning Theory. Springer Verlag, New York,1995.
Vapnik V.. Statistical learning theory. John Wiley and Sons, New York, 1998.
Vapnik V.. An overview of statistical learning theory. IEEE Transaction on Neural Networks, Vol. 10 (5): pp988-999, Sep. 1999.
Vijayakumar S., Wu S. Sequential support vector classifiers and regression. Proceedings of the International Conference on Soft Computing, 1999, pp610-619, 1999.
Wan V.; Campbell W. M.. Support vector machines for speaker verification and identification. Neural Networks for Signal Processing X, 2000. Proceedings of the 2000 IEEE Signal Processing Society Workshop, Vol.2: p775-784, 2000.
Weston J., Watkins C. Multiclass support vector machines. TR CSDTR-98-04, Department of Computer Science Egham, Surrey TW20 OEX, England, 1998.
Williamson R. C., A. Smola. B. Schlkopf. Generalization performance of regularization networks and support vector machines via entropy numbers of compact operators. IEEE Transactions on Information theory, Vol. 47 (6):pp2516-2532, Sep. 2001.
Wolfe E. A duality theory for nonlinear programming. Quarterly of Applied Mathematics, Vol. 19: pp239-244, 1961.
Xin D., Wu Z. Pan Y. A new multi-class support vector machines. Proceedings of 2001 IEEE International Conference on Systems, Man, and Cybernetics, Vol. 3: pp1673-1676, 7-10 Oct. 2001.
Yang M. H., Ahuja N. A geometric approach to train support vector machines. In proceedings of CVPR 2000, pp: 430-437, 2000.
Yee Leung, Ma J. H., Zhang W. X.. A new method for mining regression classes in
large data sets. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 23 (1): p5-21, January 2001.
Zhang X. -G.. Using class-center vectors to build support vector machines. Neural Networks for Signal Processing Ⅸ, 1999. Proceedings of the 1999 IEEE Signal Processing Society Workshop, Page(s): 3 -11, Aug 1999.
Zheng C. -H., Jiao L. -C.. Fuzzy pre-extractmg method for support vector machine. Proceedings of the First International Conference on Machine Learning and Cybernetics. Beiiing. 4-5. Nov. 2002.
边肇祺，张学工 模式识别．清华大学出版社，北京，2000。
蔡巍，王永成，尹中航 应用支持向量机进行网上信息自动分类．高技术通讯，Vol.11(11),2001。
陈光英，张千里，李星 基于SVM分类机的入侵检测系统．通信学报，Vol，23(5)，2002
崔伟东，周志华，李星 支持向量机研究．计算机工程与应用，2001。
候风雷，王炳锡 基于支持向量机的说话人辨认研究．通信学报，Vol．23 (6)，2003。
胡寿松 王源 基于支持向量机的非线性系统故障诊断，控制与决策，Vol.16(5)，2001。
黄厚宽，田盛丰 支持向量机多专家决策算法，模式识别与人工智能，Vol．13(2)，2000。
黄厚宽，田盛丰 基于支持向量机的数据库学习算法，计算机研究与发展，Vol．37(1)，2000。
刘江华，程君实，陈佳品 支持向量机训练算法综述．信息与控制，Vol．31 (1)，2002。
卢增祥，李衍达 交互支持向量机学习算法及其应用。清华大学学报(自然科学版)。Vol．39 (7)：p93-97，1999。
宋春雷，王龙，黄琳 学习理论与鲁棒控制．控制理论与应用，Vol．117 (5):pp633-636，Oct。2000．
孙宗海，孙优贤 关联向量机在微生物发酵传感器故障诊断中的应用，高校化学工程学报，2003。
孙宗海，孙优贤。用于回归估计的最小二乘广义支持向量机，系统工程理论与实践，2003。
肖健华，吴金培，杨叔子 基于SVM的综合评价方法研究．计算机工程，Vol．28 (8)，2002。
萧嵘，孙晨，王继成，张福炎 一种具有容噪性能的SVM多值分类器．计算机研究与发展，Vol．37 (9)，2000。
阎辉，张学工，马云潜，李衍达 基于变异函数的径向基核函数参数估计．自动化学报，Vol．28 (3)，2002。
阎辉，张学工，李衍达 基于核函数的最大间隔聚类算法．清华大学学报(自然科学版)，Vol．42 (1)，2002。
阎辉，张学工，李衍达 应用SVM方法进行沉积微相识别．物探化探计算技术，Vol．22 (2) 2000。
王国胜，钟一信 支持向量机的若干新进展．电子学报，Vol．29 (10)，2001。
王树青 生化反应过程模型化及计算机控制，浙江大学出版社，杭州，1998。
王树青，元英进 生化过程自动化技术，化学工业出版社，北京，1999。
张学工 关于统计学习理论与支持向量机，自动化学报，Vol．26 (1)：p32-42，2000。
张燕昆，杜平，刘重庆 基于主元分析与支持向量机的人脸识别方法．上海交通大学学报，Vol．36 (6)，2002。
张文生，丁辉，王珏 基于领域原理计算海量数据支持向量机研究．软件学报，Vol．12 (5) 2001。
周东华，叶银忠 现代故障诊断与容错控制，清华大学出版社，北京，2000。
诸静 模糊控制原理与应用 机械工业出版社，北京，P455-461 2001．6．
朱凌云，曹长修 基于支持向量机的缺陷识别方法．重庆大学学报(自然科学版)Vol．25 (6)，2002。
Vladimir N.Vapnik著，张学工译，统计学习理论的本质，清华大学出版社，2000，
R.S.Varga著，蒋尔雄等译 Matrix iterative analysis．上海科学技术出版社，1966．7，p52—81