基于统计学习理论的正则化最小二乘回归在时间序列建模和预测中的应用
|
摘要
时间序列模型经历了从线性模型到非线性模型的发展。非线性时间序列模型又分为参数模型和非参数模型。人工智能发展起来以后，神经网络、支持向量回归在时间序列建模中，发挥着越来越重要的作用。
本文借鉴神经网络和支持向量回归应用到时间序列预测的思想，将基于统计学习(Statistical Learning记为：SL)理论的正则化最小二乘回归(Regularized Least-Squares Regression记为：RLSR)应用到时间序列建模和预测中。利用RLS方法，对平稳序列和非平稳序列进行了随机模拟，并得到较好结果。之后，将RLS方法分别应用到太阳黑子、石油价格和英镑／美元的汇率的时间序列预测中，取得了比文献中已有研究更好的结果。
RLS方法充分利用了统计学习理论中再生核希尔伯特空间(Reproducing Kernel Hilbert Space记为：RKHS)的性质。在算法的求解过程中，最终转化为一个简单的线性方程。相对于文献中已有的模型，RLS方法的求解过程相对简单。
文章的贡献：
●尝试将基于统计学习理论的RLS方法应用于时间序列建模和预测中。
●通过随机模拟，用RLS方法对平稳序列、非平稳序列(含趋势项、周期项)进行了模拟，为RLS方法在时间序列分析中的应用提供了一定的支撑。
●尝试利用RLS和WRLS方法对太阳黑子数、原油价格和英镑／美元的汇率进行了预测，并取得了相对较好的预测效果。
文章的主体结构安排：
第一章：对时间序列模型的发展历程进行了简单回顾，介绍了时间序列模型预测效果评价和准确性度量的常用指标。
第二章：在统计学习理论的基础上，介绍了正则化最小二乘回归(RLSR)的基本框架。并阐述了如何将模型应用到时间序列建模和预测中。
第三章：通过随机模拟，模拟了RLS方法对平稳序列和非平稳序列(含趋势项、周期项)的预测。并在模拟的过程中，详细阐述了采用二维搜索和Holdout的方法选取参数的过程。
第四章：尝试利用RLS和WRLS方法，对太阳黑子个数进行了预测。预测效果与文献中已有的研究成果相当。同时，RLSR的算法求解相对简单。
第五章：尝试将RLS和WRLS方法应用于石油价格预测中，从RMSE指标来看，RLS方法的预测效果优于文献中已有的研究成果。
第六章：尝试将RLS和WRLS方法应用于英镑／美元汇率预测中，大部分情况下，RLS方法的预测效果优于文献已有的研究成果。
第七章：总结了文章的研究成果，与此同时提出了有待研究的问题。
In the development of time series model, two stages are included: linear model and nonlinear model. Nonlinear time series model can be classificed into parameter models and nonparemeter models. With the development of Artifical Intelligence (AI), Neural Network (NN) and Support Vector Regression (SVR) are adopted into the time series forecasting model.
Regularized Least-Squares Regression (RLSR) is a method of function estimation based on Statistical Learning (SL) theory. In this paper, we borrow the idea from using Neural Network (NN) and Support Vector Regession (SVR), and adopt the RLS method to the time series forecasting. After simuliating the RLS method with both stationary series and nonstationary series(with trend and seasonality), and get good performance. RLS method is applied to sunspot number, crude oil price, and GBP/USD currecy exchange rate times series forecasting. And the forecasting performance is better than the literature as far as we know. In addition, RLSR takes full use of the propertities of Reproducing Kernel Hilbert Space (RKHS), and the solution of RLSR is converted to solve a linear equation, the algorithm of RLS is comparably easier to be solved.The contributions of this paper are:
●Adopt the Regularized Least-Squares Regression (RLSR) to the time series forecasting model.
●Simulate the forecasting performance of RLS mothod with both stationary series and nonstationary series (with trend and seasonality).
●Apply the RLS method to the sunspot number prediction.
●Apply the RLS method to the crude oil price forecasting.
●Apply the RLS method to the GBP/USD exchange rate (daily, weekly, and biweekly) forecasting.
In the first chapter, this paper reviews the development of time series model, and introduces the forecasting evaluation and accuracy measures.
In the second chapter, this paper introduces the RLSR theory based on Statistical Learning (SL) theory and gives the frame work on how to apply RLS and WRLS methods to time series forecasting.
In the third chapter, this paper simulates the forecasting performance of RLS method with both stationary series and nonstationary series(with trend and seasonality). Parameters selection is detailed discussed.
In the fourth chapter, RLS and WRLS methods are applied to forecast the sunspot numer. The performance of RLS method is comparable to the model in the latest literature as far as sunspot number is concerned, in addition that the algorithm of RLS method is much ealier to be solved.
In the fifth chapter, RLS and WRLS methods are applied to forecast the crude oil price. The performance of RLS method is better than the models in the latest literature using the criteria of RMSE.
In the sixth chapter, RLS and WRLS methods are applied to forecast GBP/USD currency exchange rate. The performance of RLS method is better than the currency exchange rate forecasting model in the latest literature as far as daiy and weekly GBP/USD exchange rate is concered.
In the last chapter, this paper summarizes the result and gives some open problems needed to do further research.
本文借鉴神经网络和支持向量回归应用到时间序列预测的思想，将基于统计学习(Statistical Learning记为：SL)理论的正则化最小二乘回归(Regularized Least-Squares Regression记为：RLSR)应用到时间序列建模和预测中。利用RLS方法，对平稳序列和非平稳序列进行了随机模拟，并得到较好结果。之后，将RLS方法分别应用到太阳黑子、石油价格和英镑／美元的汇率的时间序列预测中，取得了比文献中已有研究更好的结果。
RLS方法充分利用了统计学习理论中再生核希尔伯特空间(Reproducing Kernel Hilbert Space记为：RKHS)的性质。在算法的求解过程中，最终转化为一个简单的线性方程。相对于文献中已有的模型，RLS方法的求解过程相对简单。
文章的贡献：
●尝试将基于统计学习理论的RLS方法应用于时间序列建模和预测中。
●通过随机模拟，用RLS方法对平稳序列、非平稳序列(含趋势项、周期项)进行了模拟，为RLS方法在时间序列分析中的应用提供了一定的支撑。
●尝试利用RLS和WRLS方法对太阳黑子数、原油价格和英镑／美元的汇率进行了预测，并取得了相对较好的预测效果。
文章的主体结构安排：
第一章：对时间序列模型的发展历程进行了简单回顾，介绍了时间序列模型预测效果评价和准确性度量的常用指标。
第二章：在统计学习理论的基础上，介绍了正则化最小二乘回归(RLSR)的基本框架。并阐述了如何将模型应用到时间序列建模和预测中。
第三章：通过随机模拟，模拟了RLS方法对平稳序列和非平稳序列(含趋势项、周期项)的预测。并在模拟的过程中，详细阐述了采用二维搜索和Holdout的方法选取参数的过程。
第四章：尝试利用RLS和WRLS方法，对太阳黑子个数进行了预测。预测效果与文献中已有的研究成果相当。同时，RLSR的算法求解相对简单。
第五章：尝试将RLS和WRLS方法应用于石油价格预测中，从RMSE指标来看，RLS方法的预测效果优于文献中已有的研究成果。
第六章：尝试将RLS和WRLS方法应用于英镑／美元汇率预测中，大部分情况下，RLS方法的预测效果优于文献已有的研究成果。
第七章：总结了文章的研究成果，与此同时提出了有待研究的问题。
In the development of time series model, two stages are included: linear model and nonlinear model. Nonlinear time series model can be classificed into parameter models and nonparemeter models. With the development of Artifical Intelligence (AI), Neural Network (NN) and Support Vector Regression (SVR) are adopted into the time series forecasting model.
Regularized Least-Squares Regression (RLSR) is a method of function estimation based on Statistical Learning (SL) theory. In this paper, we borrow the idea from using Neural Network (NN) and Support Vector Regession (SVR), and adopt the RLS method to the time series forecasting. After simuliating the RLS method with both stationary series and nonstationary series(with trend and seasonality), and get good performance. RLS method is applied to sunspot number, crude oil price, and GBP/USD currecy exchange rate times series forecasting. And the forecasting performance is better than the literature as far as we know. In addition, RLSR takes full use of the propertities of Reproducing Kernel Hilbert Space (RKHS), and the solution of RLSR is converted to solve a linear equation, the algorithm of RLS is comparably easier to be solved.The contributions of this paper are:
●Adopt the Regularized Least-Squares Regression (RLSR) to the time series forecasting model.
●Simulate the forecasting performance of RLS mothod with both stationary series and nonstationary series (with trend and seasonality).
●Apply the RLS method to the sunspot number prediction.
●Apply the RLS method to the crude oil price forecasting.
●Apply the RLS method to the GBP/USD exchange rate (daily, weekly, and biweekly) forecasting.
In the first chapter, this paper reviews the development of time series model, and introduces the forecasting evaluation and accuracy measures.
In the second chapter, this paper introduces the RLSR theory based on Statistical Learning (SL) theory and gives the frame work on how to apply RLS and WRLS methods to time series forecasting.
In the third chapter, this paper simulates the forecasting performance of RLS method with both stationary series and nonstationary series(with trend and seasonality). Parameters selection is detailed discussed.
In the fourth chapter, RLS and WRLS methods are applied to forecast the sunspot numer. The performance of RLS method is comparable to the model in the latest literature as far as sunspot number is concerned, in addition that the algorithm of RLS method is much ealier to be solved.
In the fifth chapter, RLS and WRLS methods are applied to forecast the crude oil price. The performance of RLS method is better than the models in the latest literature using the criteria of RMSE.
In the sixth chapter, RLS and WRLS methods are applied to forecast GBP/USD currency exchange rate. The performance of RLS method is better than the currency exchange rate forecasting model in the latest literature as far as daiy and weekly GBP/USD exchange rate is concered.
In the last chapter, this paper summarizes the result and gives some open problems needed to do further research.
引文
[1]. Abramson, B.and Finizza, A., Probablistic forecatings from probabilistic models: a case study in the oil market, International journal of forecasting, 11(1) (1995) 63-72.
[2]. Adesi, G., Bourgon, F. and Giannopoulos, K., Don't look back, Risk, (8) (1998) 100-103.
[3]. Andreou, A., Karytinos, A. and Pavlides, G., Non-linear time-series analysis of the Greek exchange-rate market, International Journal of Bifurcation and Chaos 7 (2001) 1729-1758.
[4]. Barnett, W., Bemdt, E. and White, H., Dynamic Econometric Modeling, Cambridge University Press, Cambridge, 1988.
[5]. Box, G., and Jenkins, G., Time Series Analysis: Forecasting and Control, Holden-Day, San Francisco, CA. 1976.
[6]. Cao, D., Pang, S. and Bai, Y., Forecasting exchange rate using support vector machines, Proceedings of the fourth international conference on machine learning and cybernetics, (2005) 3488-3452.
[7]. Cao, L. and Gub, Q., Dynamic support vector machines for non-stationary time series forecasting, Intelligent Data Analysis 6, IOS Press, (2002) 67-83.
[8]. Cao, L., support vector machines experts for time series forecasting, Neurocomputing 51, (2003) 321-339.
[9]. Chen, A. and Leung, M., Regression neural network for error correction in foreign exchange forecasting and trading, Computers and Operations Research 31 (2004) 1049-1068.
[10]. Chen, D. and Xiang, D., The consistency of multicategory support vector machines. Adv. Comput. Math., 24 (2006) 155-169.
[11]. Cottrell, M., Girard, Y., Mangeas, M., and Muller, C., Nerual modeling for time series: a statistical stepwise method for weight elimination, IEEE Trans. Neural Networks 6(6) (1995) 1355-1364.
[12]. Cristianini, N. and Taylor, J., An introduction to support vector machines and other kernel-based learning methods, Cambride Universiy Press (2000).
[13]. Cucker, F and Smale, S., Best choices for regnlarization parameters in learning theory. Found. Comput. Math., 2 (2002) 413-428.
[14]. Cucker, F. and Smale, S., On the mathematical foundations of learning. Bulletin Amer. Math. Soc., 39 (2002) 1-49.
[15]. Deco, G., Neuneier, R. and Schurmann, B., Non-parametric data selection for neural learning in non-stationary time series, Neural Networks 10(3), (1997) 401-407.
[16]. Dietterich, T., Machine learning for sequential data: A review. In T. Caelli, editor, Structural, Syntactic, and Statistical Pattem Recognition, volume 2396 of Lecture Notes in Computer Science, Springer-Verlag, (2002) 15-30.
[17]. Fan, J. and Yao, Q., Nonlinear time series-Nonparametric and Parametric Methods, Springer Verlag (2003).
[18]. Francis E., Tay, and Cao, L., Application of support vector machines in financial time series forecasting, The international journal of management science, Omega 29 (2001) 309-317.
[19]. Ginzberg, I, and Horn, D., Learning the rule of a time series, Int. J. Neural Systems 3(2) (1992) 167-177.
[20]. Gooijer, J. and Hyndman, R., 25 years of time series forecasting, international journal of forecasting 22 (2006) 443-473.
[21]. Groot, C., and Wurtz, D., Analysis of univariate time series with connectionist nets: a cast study of two classical examples, Neurocomputing 3 (1991) 177-192.
[22]. Hastie, T., Tibshirani, R. and Friedman, J., The elements of statistical learning: data Mining, inference, and prediction, Springer Verlag (2001). http://cbcl.mit.edu/projects/cbcl/res-area/abstracts/2002-abstracts/yeo-abstract-2.pdf
[23]. http://jpkc.xmu.edu.cn/gjjr/inter-finance/1/1.htm
[24]. Hutingtong, H., Oil price forecasting in the 1980s: What went wrong?, The energy journal 15 (2) (1994) 1-22.
[25]. Ivanov, V., On linear problems which are not well-posed, Soviet Math., Docl, 3 (4) 981-983.
[26]. Kehagias, A. and Petridis, V., Time-series segmentation using predictive modular neural networks, Neural Computation, 9, (1997) 1691-1709.
[27]. Kim, K., Finacial time series forecasting using support vector machines, Neurocomputing 55 (2003) 307-319.
[28]. Li, M., and Mehrotra, K., Chilukuri Mohan and Sanjay Ranka. Forecasting sunspot numbers using neural networks. Proceedings of IEEE symposium on intelligent control. (1990) 524-529.
[29]. Morana, C., A semiparametric approach to short-term oil price forecasting, Energy economics, 23(3) (2001) 325-338.
[30]. Nie, J., Nonlinear time series forecasting: a fuzzy-neural approach, Neurocomputing 16 (1997) 63-76.
[31]. Nowlan, S., and Hinton, G., simplifying neural networks by soft weight-sharing, neural compute 4 (1993) 473-493.
[32]. Parzen, E., AR-ARMA models for time series analysis and forecasting, Journal of Forecasting, 1, (1982) 67-82.
[33]. Pawelzik, K., Kohlmorgen, J. and Muller, K., Annealed competition of experts for a segmentation and classification of switching dynamics, Neural Computation, 8(2), (1996) 340-356.
[34]. Poggio, T. and Smale, S., The mathematics of Learning: Dealing with data. Amercian Mathematical Society, 50(5) (2003) 537-544.
[35]. Preda, C., Regression models for functional data by reproducing kernel Hilbert spaces methods, Journal of statistics Planning and inference 137 (2007) 829-840.
[36]. Qiang, W., Ying, Y. and Zhou, D., Learning theory: from regression to classification. In K. Jetter, M. Buhmann, W. Haussmann, R. Schaback, and J. Stoeckler, editors, Topics in Multivariate Ap-proximation and Interpolation, volume 12 of Studies in Computational Mathematics, Elsevier, (2006) 257-290.
[37]. Rifkin, R., Yeo, G. and Poggio, T., Regression and least-square classification, (2006) 1-12. http://www.mit.edu/～9.520/Classes/class04.pdf
[38]. Sfetsos, A. and Siropoulos, C., Time series forecasting of averaged data with efficient use of information, IEEE transactions on systems, man, and cybernetics-part a: systems and humans 35 (2005) 738-745.
[39]. Shewchuck, J., An introduction to the conjugate gradient method without the agonizing pain, Technical Report CMU-CS-94-125, School of Computer Science, Carnegie Mellon University, (1994) 2-48.
[40]. Taylor, J. and Cristianini, N., Kernel methods for pattern analysis. Cambridge University Press, (2004) 26-29.
[41]. Tikhonov, A., On solving ill-posed problem and method of regularization, Doklady Akademii Nauk USSR, 153 (1963) 510-504.
[42]. Thissen, U., brakel, R., Weijer, A., Melssen, W. and Buydens, L., Using support vector machines for time series prediction. Chemometrics and intelligent laboratory systems 69 (2003) 35-49.
[43]. Tong, H. and Lim, K., Threshold autoregessive, Limit cycles and cyclical data, d. Roy. Statistic. Soc. B 42 (3) (1980) 245-292.
[44]. Tseng, F., Yu, H. and Tzeng, G., Combining neural network model with seasonal time series ARIMA model. Technological Forecasting and Social Change 69 (2002) 71-87.
[45]. Vapnik, V., An overview of statistical learning theory. IEEE transactions on neural networks, 10 (5) (1999) 988-989.
[46]. Vapnik, V., Statistical learning theory. John Wiley & Sons (1998).
[47]. Wang, S,, An adaptive approach to market development forecasting, Neural Computing & Applications 8, (1999) 3-8.
[48]. Wang, S., Yu, L. and Lai, K. Crude oil price forecasting with TEI@I methodology, Journal of systems science and complexity 18 (2005) 145-166.
[49]. Wedding, D., and Cois, K., Time series forecasting by combining GAUSSIAN networks, certainty factors, and the box-Jenkins model, Neurocomputing 10 (1996) 149-168.
[50]. Weigend, A., Huberman, B., and Rumelhart, D., Predicting the futerue: a connectionist approach, Int. J. Neural Systems 1 (1990) 93-209.
[51]. Wu, Q. and Zhou, D., Analysis of support vector machine classification, J. Comput. Anal. Appl., 8 (2006) 99-119.
[52]. Wu, Q. and Zhou, D., SVM soft margin classifiers: linear programming versus quadratic programming. Neural Comp, 17 (2005) 1160-1187.
[53]. Wu, Q., Classification and Regularization in Learning Theory. PhD thesis, City University of Hong Kong (2005)
[54]. Xie, W., Yu, L., Xu, S. and Wang, S., A new method for crude oil price forecasting based on support vector machines, Lecture notes in computer science (2006) 3994: 441-451.
[55]. Yeo, G., and Rakhlin, A., Feature Selection for (Nonlinear) Regularized least-squaress Classification, (2002) 1-2. http://cbcl.mit.edu/projects/cbcl/res-area/abstracts/2002-abstracts/yeo-abstract-2.pdf
[56]. Yu, L., Wang, S., Lai, K., A novel nonlinear ensemble forecasting model incorporating GLAR and ANN for foreign exchange rates, Computers and Operations Research 32 (2005) 2523-2541.
[57]. Zhang, G., Berardi, V., Time series forecasting with neural network ensembles: an application for exchange rate prediction. Journal of the Operational Research Society 52 (2001) 652-664.
[58]. Zhang, P., Model selection via multifold cross-validation, Ann. Statist, 21 (1996) 299-311.
[59]. Zhang, R. and Rudnicky, A., Word level confidence annotation using combinations of features, European conference on speech communication and technology, (2001) 2105-2108.
[60]. Zhou, D., Capacity of reproducing kernel spaces in learning theory. IEEE Trans. Inform. Theory, 49 (2003) 1743-1752
[61]. Zhou, D., Density problem and approximation error in learning theory. Preprint, (2006)
[62]. Zhou, D., The covering number in learning theory. J. Complexity, 18 (2002) 739-767.
[2]. Adesi, G., Bourgon, F. and Giannopoulos, K., Don't look back, Risk, (8) (1998) 100-103.
[3]. Andreou, A., Karytinos, A. and Pavlides, G., Non-linear time-series analysis of the Greek exchange-rate market, International Journal of Bifurcation and Chaos 7 (2001) 1729-1758.
[4]. Barnett, W., Bemdt, E. and White, H., Dynamic Econometric Modeling, Cambridge University Press, Cambridge, 1988.
[5]. Box, G., and Jenkins, G., Time Series Analysis: Forecasting and Control, Holden-Day, San Francisco, CA. 1976.
[6]. Cao, D., Pang, S. and Bai, Y., Forecasting exchange rate using support vector machines, Proceedings of the fourth international conference on machine learning and cybernetics, (2005) 3488-3452.
[7]. Cao, L. and Gub, Q., Dynamic support vector machines for non-stationary time series forecasting, Intelligent Data Analysis 6, IOS Press, (2002) 67-83.
[8]. Cao, L., support vector machines experts for time series forecasting, Neurocomputing 51, (2003) 321-339.
[9]. Chen, A. and Leung, M., Regression neural network for error correction in foreign exchange forecasting and trading, Computers and Operations Research 31 (2004) 1049-1068.
[10]. Chen, D. and Xiang, D., The consistency of multicategory support vector machines. Adv. Comput. Math., 24 (2006) 155-169.
[11]. Cottrell, M., Girard, Y., Mangeas, M., and Muller, C., Nerual modeling for time series: a statistical stepwise method for weight elimination, IEEE Trans. Neural Networks 6(6) (1995) 1355-1364.
[12]. Cristianini, N. and Taylor, J., An introduction to support vector machines and other kernel-based learning methods, Cambride Universiy Press (2000).
[13]. Cucker, F and Smale, S., Best choices for regnlarization parameters in learning theory. Found. Comput. Math., 2 (2002) 413-428.
[14]. Cucker, F. and Smale, S., On the mathematical foundations of learning. Bulletin Amer. Math. Soc., 39 (2002) 1-49.
[15]. Deco, G., Neuneier, R. and Schurmann, B., Non-parametric data selection for neural learning in non-stationary time series, Neural Networks 10(3), (1997) 401-407.
[16]. Dietterich, T., Machine learning for sequential data: A review. In T. Caelli, editor, Structural, Syntactic, and Statistical Pattem Recognition, volume 2396 of Lecture Notes in Computer Science, Springer-Verlag, (2002) 15-30.
[17]. Fan, J. and Yao, Q., Nonlinear time series-Nonparametric and Parametric Methods, Springer Verlag (2003).
[18]. Francis E., Tay, and Cao, L., Application of support vector machines in financial time series forecasting, The international journal of management science, Omega 29 (2001) 309-317.
[19]. Ginzberg, I, and Horn, D., Learning the rule of a time series, Int. J. Neural Systems 3(2) (1992) 167-177.
[20]. Gooijer, J. and Hyndman, R., 25 years of time series forecasting, international journal of forecasting 22 (2006) 443-473.
[21]. Groot, C., and Wurtz, D., Analysis of univariate time series with connectionist nets: a cast study of two classical examples, Neurocomputing 3 (1991) 177-192.
[22]. Hastie, T., Tibshirani, R. and Friedman, J., The elements of statistical learning: data Mining, inference, and prediction, Springer Verlag (2001). http://cbcl.mit.edu/projects/cbcl/res-area/abstracts/2002-abstracts/yeo-abstract-2.pdf
[23]. http://jpkc.xmu.edu.cn/gjjr/inter-finance/1/1.htm
[24]. Hutingtong, H., Oil price forecasting in the 1980s: What went wrong?, The energy journal 15 (2) (1994) 1-22.
[25]. Ivanov, V., On linear problems which are not well-posed, Soviet Math., Docl, 3 (4) 981-983.
[26]. Kehagias, A. and Petridis, V., Time-series segmentation using predictive modular neural networks, Neural Computation, 9, (1997) 1691-1709.
[27]. Kim, K., Finacial time series forecasting using support vector machines, Neurocomputing 55 (2003) 307-319.
[28]. Li, M., and Mehrotra, K., Chilukuri Mohan and Sanjay Ranka. Forecasting sunspot numbers using neural networks. Proceedings of IEEE symposium on intelligent control. (1990) 524-529.
[29]. Morana, C., A semiparametric approach to short-term oil price forecasting, Energy economics, 23(3) (2001) 325-338.
[30]. Nie, J., Nonlinear time series forecasting: a fuzzy-neural approach, Neurocomputing 16 (1997) 63-76.
[31]. Nowlan, S., and Hinton, G., simplifying neural networks by soft weight-sharing, neural compute 4 (1993) 473-493.
[32]. Parzen, E., AR-ARMA models for time series analysis and forecasting, Journal of Forecasting, 1, (1982) 67-82.
[33]. Pawelzik, K., Kohlmorgen, J. and Muller, K., Annealed competition of experts for a segmentation and classification of switching dynamics, Neural Computation, 8(2), (1996) 340-356.
[34]. Poggio, T. and Smale, S., The mathematics of Learning: Dealing with data. Amercian Mathematical Society, 50(5) (2003) 537-544.
[35]. Preda, C., Regression models for functional data by reproducing kernel Hilbert spaces methods, Journal of statistics Planning and inference 137 (2007) 829-840.
[36]. Qiang, W., Ying, Y. and Zhou, D., Learning theory: from regression to classification. In K. Jetter, M. Buhmann, W. Haussmann, R. Schaback, and J. Stoeckler, editors, Topics in Multivariate Ap-proximation and Interpolation, volume 12 of Studies in Computational Mathematics, Elsevier, (2006) 257-290.
[37]. Rifkin, R., Yeo, G. and Poggio, T., Regression and least-square classification, (2006) 1-12. http://www.mit.edu/～9.520/Classes/class04.pdf
[38]. Sfetsos, A. and Siropoulos, C., Time series forecasting of averaged data with efficient use of information, IEEE transactions on systems, man, and cybernetics-part a: systems and humans 35 (2005) 738-745.
[39]. Shewchuck, J., An introduction to the conjugate gradient method without the agonizing pain, Technical Report CMU-CS-94-125, School of Computer Science, Carnegie Mellon University, (1994) 2-48.
[40]. Taylor, J. and Cristianini, N., Kernel methods for pattern analysis. Cambridge University Press, (2004) 26-29.
[41]. Tikhonov, A., On solving ill-posed problem and method of regularization, Doklady Akademii Nauk USSR, 153 (1963) 510-504.
[42]. Thissen, U., brakel, R., Weijer, A., Melssen, W. and Buydens, L., Using support vector machines for time series prediction. Chemometrics and intelligent laboratory systems 69 (2003) 35-49.
[43]. Tong, H. and Lim, K., Threshold autoregessive, Limit cycles and cyclical data, d. Roy. Statistic. Soc. B 42 (3) (1980) 245-292.
[44]. Tseng, F., Yu, H. and Tzeng, G., Combining neural network model with seasonal time series ARIMA model. Technological Forecasting and Social Change 69 (2002) 71-87.
[45]. Vapnik, V., An overview of statistical learning theory. IEEE transactions on neural networks, 10 (5) (1999) 988-989.
[46]. Vapnik, V., Statistical learning theory. John Wiley & Sons (1998).
[47]. Wang, S,, An adaptive approach to market development forecasting, Neural Computing & Applications 8, (1999) 3-8.
[48]. Wang, S., Yu, L. and Lai, K. Crude oil price forecasting with TEI@I methodology, Journal of systems science and complexity 18 (2005) 145-166.
[49]. Wedding, D., and Cois, K., Time series forecasting by combining GAUSSIAN networks, certainty factors, and the box-Jenkins model, Neurocomputing 10 (1996) 149-168.
[50]. Weigend, A., Huberman, B., and Rumelhart, D., Predicting the futerue: a connectionist approach, Int. J. Neural Systems 1 (1990) 93-209.
[51]. Wu, Q. and Zhou, D., Analysis of support vector machine classification, J. Comput. Anal. Appl., 8 (2006) 99-119.
[52]. Wu, Q. and Zhou, D., SVM soft margin classifiers: linear programming versus quadratic programming. Neural Comp, 17 (2005) 1160-1187.
[53]. Wu, Q., Classification and Regularization in Learning Theory. PhD thesis, City University of Hong Kong (2005)
[54]. Xie, W., Yu, L., Xu, S. and Wang, S., A new method for crude oil price forecasting based on support vector machines, Lecture notes in computer science (2006) 3994: 441-451.
[55]. Yeo, G., and Rakhlin, A., Feature Selection for (Nonlinear) Regularized least-squaress Classification, (2002) 1-2. http://cbcl.mit.edu/projects/cbcl/res-area/abstracts/2002-abstracts/yeo-abstract-2.pdf
[56]. Yu, L., Wang, S., Lai, K., A novel nonlinear ensemble forecasting model incorporating GLAR and ANN for foreign exchange rates, Computers and Operations Research 32 (2005) 2523-2541.
[57]. Zhang, G., Berardi, V., Time series forecasting with neural network ensembles: an application for exchange rate prediction. Journal of the Operational Research Society 52 (2001) 652-664.
[58]. Zhang, P., Model selection via multifold cross-validation, Ann. Statist, 21 (1996) 299-311.
[59]. Zhang, R. and Rudnicky, A., Word level confidence annotation using combinations of features, European conference on speech communication and technology, (2001) 2105-2108.
[60]. Zhou, D., Capacity of reproducing kernel spaces in learning theory. IEEE Trans. Inform. Theory, 49 (2003) 1743-1752
[61]. Zhou, D., Density problem and approximation error in learning theory. Preprint, (2006)
[62]. Zhou, D., The covering number in learning theory. J. Complexity, 18 (2002) 739-767.