基于支持向量机的非线性时间序列预测方法研究
|
摘要
基于时间序列模型的预测作为定量预测的主要方法,在实际中的应用几乎遍及预报活动的所有领域。传统的时间序列建模主要针对线性或弱非线性时间序列,但是面对现实生活中大量的复杂非线性时间序列(甚至混沌)时则显得力不从心。基于统计学习理论的支持向量机是一种新型的学习方法,它采用结构风险最小化原则,为解决小样本、非线性、高维数等学习问题提供了一个框架。近年来,支持向量机已开始应用于非线性时间序列预测中。本文以提高支持向量机预测模型的精度为主要目的,研究了非线性时间序列的降噪、非平稳化处理和模型参数优化等问题。
局部投影降噪算法已广泛应用于非线性时间序列的分析中,但其邻域选取具有主观性,严重影响到降噪的性能。本文研究了一种按照自适应方式选取邻域大小的局部投影降噪算法。首先用时间延迟方法将一维时间序列重构到高维相空间。然后逐步增大每个待分析相点的领域大小,在领域最大主方向变化过程中,自适应地确定该相点的最优领域。最后再用局部几何投影的方法去除噪声成分。实验结果表明,自适应邻域选取方法,提高了局部投影算法的降噪能力。
非平稳时间序列具有一定的周期性和随机性的,难以用单一的方法进行精确的预测。本文提出将经验模式分解(EMD)和最小二乘支持向量机(LS-SVM)相结合的预测方法。首先,运用EMD将趋势时间序列自适应地分解成一系列不同尺度的本征模式分量;其次,对每个本征模式分量,采用合适的核函数和超参数构造不同的LS-SVM进行预测;最后对各分量的预测值进行拟合得到最终的预测值。并将此方法成功应用于机械振动非平稳趋势序列的预测。
重构相空间和支持向量机的参数优化是提高预测性能的两个重要方面,传统上这两个问题是分开解决的。本文提出采用混合粒子群算法实现二者的联合优化。联合优化方法融合了离散粒子群和实数值粒子群算法,同时对空间重构的参数和支持向量机的参数设置进行优化。仿真试验表明,此方法可以提高预测精度。
As the main technique of the quantitative forecast, the method of time series prediction is used almost in all fields of forecast. The traditional methods of time series prediction mainly focus on linear time series and weak nonlinear time series, so they lack effectiveness when they face complex nonlinear time series (even chaos time series). Support Vector Machine (SVM) is a kind of novel machine learning methods, theoretically based on statistic learning theory. It employs the criteria of structural risk minimization and provides a framework for the small samples, nonlinearity and high dimension problems. SVM has been applied in nonlinear time series prediction in recent years. Focusing on raising accuracy of SVM prediction model, this paper studies noise-reduction, non-stationary processing and model parameters optimization of nonlinear time series.
Although widely applied in nonlinear time series analysis, the noise reduction method via local projection has the subjectivity of selecting the neighborhood, which greatly affects the performance. A new method by local projection using adaptive neighborhood selection is studied. First, one dimensional time series are embedded into a high dimensional phase space according to time-delay theory. The neighborhood size for each candidate phase point in phase space is increased by adding neighboring point one by one. The optimal neighborhood size for the phase point is determined during the direction’s variation of the most significant eigenvector of neighborhood as size increasing, and then the noise is eliminated through local geometric projection. Experiment results show that adaptive neighborhood selection can improve the noise reduction performance of local projection method.
Non-stationary time series has periodicity and randomness so that it is difficult to construct the model of accurate forecast with single method. A hybrid forecasting method based on Empirical Mode Decomposition (EMD) and Least Square Support Vector Machine (LS-SVM) is presented in this paper. Firstly, the non-stationary time series is adaptively decomposed into a series of stationary intrinsic mode functions (IMF) in different scale space using EMD. Then the right parameter and kernel functions are chosen to build different LS-SVM respectively to every IMF. Finally, these forecasting results of each IMF are combined to obtain final forecasting result. This method is successfully applied to non-stationary trend prediction of mechanical vibration.
Phase space reconstruction and SVM parameters optimization are two important aspects for improving prediction performance and are solved separately traditionally. This paper proposes a joint optimization algorithm based on Hybrid PSO. This method combines the discrete PSO with the continuous-valued PSO to simultaneously optimize the phase space reconstruction and the SVM parameters setting. The experimental results showed the proposed approach can raise prediction accuracy.
局部投影降噪算法已广泛应用于非线性时间序列的分析中,但其邻域选取具有主观性,严重影响到降噪的性能。本文研究了一种按照自适应方式选取邻域大小的局部投影降噪算法。首先用时间延迟方法将一维时间序列重构到高维相空间。然后逐步增大每个待分析相点的领域大小,在领域最大主方向变化过程中,自适应地确定该相点的最优领域。最后再用局部几何投影的方法去除噪声成分。实验结果表明,自适应邻域选取方法,提高了局部投影算法的降噪能力。
非平稳时间序列具有一定的周期性和随机性的,难以用单一的方法进行精确的预测。本文提出将经验模式分解(EMD)和最小二乘支持向量机(LS-SVM)相结合的预测方法。首先,运用EMD将趋势时间序列自适应地分解成一系列不同尺度的本征模式分量;其次,对每个本征模式分量,采用合适的核函数和超参数构造不同的LS-SVM进行预测;最后对各分量的预测值进行拟合得到最终的预测值。并将此方法成功应用于机械振动非平稳趋势序列的预测。
重构相空间和支持向量机的参数优化是提高预测性能的两个重要方面,传统上这两个问题是分开解决的。本文提出采用混合粒子群算法实现二者的联合优化。联合优化方法融合了离散粒子群和实数值粒子群算法,同时对空间重构的参数和支持向量机的参数设置进行优化。仿真试验表明,此方法可以提高预测精度。
As the main technique of the quantitative forecast, the method of time series prediction is used almost in all fields of forecast. The traditional methods of time series prediction mainly focus on linear time series and weak nonlinear time series, so they lack effectiveness when they face complex nonlinear time series (even chaos time series). Support Vector Machine (SVM) is a kind of novel machine learning methods, theoretically based on statistic learning theory. It employs the criteria of structural risk minimization and provides a framework for the small samples, nonlinearity and high dimension problems. SVM has been applied in nonlinear time series prediction in recent years. Focusing on raising accuracy of SVM prediction model, this paper studies noise-reduction, non-stationary processing and model parameters optimization of nonlinear time series.
Although widely applied in nonlinear time series analysis, the noise reduction method via local projection has the subjectivity of selecting the neighborhood, which greatly affects the performance. A new method by local projection using adaptive neighborhood selection is studied. First, one dimensional time series are embedded into a high dimensional phase space according to time-delay theory. The neighborhood size for each candidate phase point in phase space is increased by adding neighboring point one by one. The optimal neighborhood size for the phase point is determined during the direction’s variation of the most significant eigenvector of neighborhood as size increasing, and then the noise is eliminated through local geometric projection. Experiment results show that adaptive neighborhood selection can improve the noise reduction performance of local projection method.
Non-stationary time series has periodicity and randomness so that it is difficult to construct the model of accurate forecast with single method. A hybrid forecasting method based on Empirical Mode Decomposition (EMD) and Least Square Support Vector Machine (LS-SVM) is presented in this paper. Firstly, the non-stationary time series is adaptively decomposed into a series of stationary intrinsic mode functions (IMF) in different scale space using EMD. Then the right parameter and kernel functions are chosen to build different LS-SVM respectively to every IMF. Finally, these forecasting results of each IMF are combined to obtain final forecasting result. This method is successfully applied to non-stationary trend prediction of mechanical vibration.
Phase space reconstruction and SVM parameters optimization are two important aspects for improving prediction performance and are solved separately traditionally. This paper proposes a joint optimization algorithm based on Hybrid PSO. This method combines the discrete PSO with the continuous-valued PSO to simultaneously optimize the phase space reconstruction and the SVM parameters setting. The experimental results showed the proposed approach can raise prediction accuracy.
引文
1.吕金虎,陆均安,陈士华.混沌时间序列分析及其应用[M].武汉大学:武汉大学出版社,2002
2.陈关荣,吕金虎.Lorenz系统族的动力学分析、控制与同步[M].北京:科学出版社, 2003
3. Tong H. A note using threshold autoregressive models for multi-step-ahead prediction of cyclical data[J]. Journal of Time Series Analysis, 1982, 03: 137-140
4. Takens F. Determing strange attractors in turbulence[J]. Lecture notes in Math, 1981, 898: 361-381
5. Cowper M R, Mulgrew B, Unsworth C P. Nonlinear prediction of chaotic signals using a normalized radial basis function network [J]. Signal Processing, 2002, 82(5): 775-789
6.陈南祥,黄强,曹连海,徐建新.径流序列的相空间重构神经网络预测模型[J].河海大学学报(自然科学版), 2005, 05: 490-493
7.郁俊莉.基于混沌时间序列的非线性动态系统神经网络建模与预测[J].武汉大学学报(理学版), 2005, 03: 286-290
8.李军,刘君华.一种新型广义RBF神经网络在混沌时间序列预测中的研究[J].物理学报,2005, 10: 4569-4577
9. Zhu Jiayuan, Bo Ren, Heng Xi Zhang etal. Time Series Prediction via New Support Vector Machines[C]. In: Proceedings of ICMLC 2002, IEEE, China, Beijing, 2002: 364-366
10.李元诚,方廷健,于尔铿.短期负荷预测的支持向量机方法研究[J].中国电机工程学报, 2003, 23(6): 55-59
11.刘涵,刘丁,郑岗,等.基于最小二乘支持向量机的天然气负荷预测[J].化工学报, 2004, 55(5): 828-832.
12. Hammel S M. A noise reduction method for chaotic system[J]. Physics Letter A, 1990, 148 (8-9): 421-428
13. Schreiber T. Extremely simple nonlinear noise-reduction method[J]. Physical Review E, 1993, 47 (4): 2401-2404
14.何岱海,徐健学,陈永红.非线性动力学相空间重构中小波变换方法研究[J].振动工程学报,1999, 12(1): 27-32
15.游荣义,陈忠,徐慎初,吴伯僖.基于小波变换的混沌信号相空间重构研究[J].物理学报, 2004, 53(9): 2882-2888
16. Vautard R, Yiou P, Ghil M. Singular-spectrum analysis: a toolkit for short, noisy chaotic signals[J]. Physica D, 1992, 59: 95-126
17. Cawley R, Hsu G H. Local-geometric-projection method for noise reduction in chaotic maps and flows[J]. Physica Review A, 1992, 46(6): 3057-3082
18. Grassberger P, Hegger R, Kantz H, Schaffrath C, Schreiber T. On noise reduction methods for chaotic data[J]. Chaos, 1993, 3 (2) : 127-141
19. Schreiber T, Richter M. Fast nonlinear projective filtering in a data stream[J]. International Journal of Bifurcation and chaos, 1999, 9(10): 2039-2045
20.徐金梧,吕勇,王海峰.局部投影算法及其在非线性时间序列分析中的应用[J].机械工程学报, 2003, 39(9): 146-150
21.胡静,杨宗凯.基于局部投影方法的混沌信号去噪[J].华中科技大学学报(自然科学版), 2002, 04: 70-72
22.吴迪嘉,罗汉文,宋文涛,施聪.局部线性投影算法在混沌键控系统中进行噪声去除的应用[J].上海交通大学学报, 2002, 36(6): 757-760
23.李振平,闻邦椿.刚性转子轴承系统的复杂非线性动力学行为研究[J].振动与冲击, 2005, 24(3): 36-39
24. Grassberger P, Procaccia I. Estimation of the Kolmogorov entropy from a chaotic signal[J]. Psysica Review A, 1983, 28: 2591-2593
25. Zhang G P, Patuwo B E, Hu M Y. A simulation study of artificial neural networks for nonlinear time-series forecasting[J]. Computers & Operations Research, 2001, 28(3): 381-396 .
26. Cowper M R, Mulgrew B, Unsworth C P. Nonlinear prediction of chaotic signals using a normalized radial basis function network[J]. Signal Processing, 2002, 82(5): 775-789 .
27.胡寿松,张正道.基于神经网络的非线性时间序列故障预报[J].自动化学报, 2007, 07: 744-748
28.杨金芳,翟永杰,王东风,徐大平.基于支持向量回归的时间序列预测[J].中国电机工程学报, 2005, (17): 110-114
29. Niu Lin, Zhao Jian-guo, Du Zhi-gang, Jin Xiao-ling. Application of Time Series Forecasting Algorithm via Support Vector Machines to Power System Wide-area Stability Prediction[J]. IEEE/PES Transmission and Distribution Conference and Exhibition. Asia and Pacific. 2005: 1-6 .
30.董辉,傅鹤林,冷伍明.支持向量机的时间序列回归与预测[J].系统仿真学报, 2006, 07: 1785-1788
31. N. Vladimir, V. Vapnik. An overview of statistical learning theory[J]. IEEE Transaction on Neural Networks, 1999, 10(5): 988-999
32.张学工.关于统计学习理论与支持向量机[M].自动化学报, 2000, 26(1): 32-42
33.邓乃杨,田英杰.数据挖掘中的新方法—支持向量机[M].北京:科学出版社, 2004
34. Vapnik V. The nature of statistical learning theory[M]. 2nd edition. New York: Springer-Verlag, 1999
35. Shawe J, Cristianini N.赵玲玲,翁苏明,曾华军等译.模式识别的核方法[M].北京:机械工业出版社, 2006
36. Suykens A K, Gestel T Van, Brabanter J De, Moor B De, Vandewalle J. Least Squares Support Vector Machines[J]. World Scientific Pub. Co. , Singapore, 2002
37.张贤达,保铮.非平稳信号分析与处理[M].北京:国防工业出版社, 1998
38. Norden E. Huang, Zheng Shen, Steven R. Long, etal. The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis[J]. Proc. R. Soc. Lond. A, 1998, 454: 903-995
39.张西宁.大型回转机械运行状态检测和预报的研究[D].西安:西安交通大学机械工程学院, 1998.
40.裴树毅.基于神经网络的预测理论及其应用[D].西安:西安交通大学机械工程学院, 1994.
41. Thissen U, Brakel R V, Weijer A P D, et al. Using support vector machines for time series prediction[J]. Chemometrics and Intelligent Laboratory Systems, 2003, 69 (1-2): 35-49.
42.李凌均,张周锁,何正嘉.基于支持向量机的机械设备状态趋势预测研究[J].西安交通大学学报, 2004, 38(3): 230-234.
43. YANO M, HOMMA N, SAKAIM, ABE K. Phase space reconstruction from observed time series using Lyapunov spectrum analysis[C]. Proc of SICE Annual Conference 2002. Osaka (Japan): SICE, 2002: 721-726.
44.周佰成,翟延慧,李向前.Takens理论和小波分析在非线性预测中的应用[J].吉林大学自然科学学报, 2001, 7: 21-28.
45.盛昭瀚,姚洪兴,王海燕等.混沌动力系统的重构预测与控制[M].中国经济出版社, 2003
46.张胜,刘红星,高敦堂等.NN非线性时间序列预测模型输入延时的确定[J].东南大学学报(自然科学版), 2002, 32(6): 905-908
47. Kim H S, Eykholt R J, Salas D. Nonlinear dynamics, delay times, and embedding windows[J]. Physica: D, 1999, 127(1): 48-60.
48. Kirkpatrick S, Gelatt C D J, Vecchi M P. Optimization by simulated annealing[J]. Science, 2003, 220: 671-680.
49. De Castro L N, Von Zuben F J. Learning and mization Using Clonal Selection Principle[J]. IEEE Transaction on Evolutionary Computation, 2002, 6(3): 239-251
50. Shisanu T, Parallel genetic algorithm with parameter adaptation[J]. Information Processing Letters, 2002, 82: 47-54
51. Kennedy J, Eberhart R C. Particle swarm optimization[C]. IEEE International Conference on Neural Networks. Perth, Piscataway, NJ, Australia: IEEE Service Center, 1995, 4: 1942-1948
52.李爱国,覃征,鲍复民,等.粒子群优化算法[J].计算机工程与应用, 2002, 39(21): 1-3
53. Kennedy J, Eberhart R C. A discrete binary version of the particle swarm algorithm[C]. Proceedings of the World Multi-conference on Systemics, Cybernetics and Informatics 1997, Piscataway, 1997, NJ: 4104-4109
54. Shi Y, Eberhart R C. Parameter selection in particle swarm optimization[C]. Proc of the 7th Annual Conf on Evolutionary Programming. Washington DC, 1998: 591-600
55. Ito K, Nakano R. Optimizing Support Vector Regression Hyper-parameters Based On Cross-Validation[C]. Proceedings of the International Joint Conference on Neural Networks, 2003, 7(3): 2077-2082
2.陈关荣,吕金虎.Lorenz系统族的动力学分析、控制与同步[M].北京:科学出版社, 2003
3. Tong H. A note using threshold autoregressive models for multi-step-ahead prediction of cyclical data[J]. Journal of Time Series Analysis, 1982, 03: 137-140
4. Takens F. Determing strange attractors in turbulence[J]. Lecture notes in Math, 1981, 898: 361-381
5. Cowper M R, Mulgrew B, Unsworth C P. Nonlinear prediction of chaotic signals using a normalized radial basis function network [J]. Signal Processing, 2002, 82(5): 775-789
6.陈南祥,黄强,曹连海,徐建新.径流序列的相空间重构神经网络预测模型[J].河海大学学报(自然科学版), 2005, 05: 490-493
7.郁俊莉.基于混沌时间序列的非线性动态系统神经网络建模与预测[J].武汉大学学报(理学版), 2005, 03: 286-290
8.李军,刘君华.一种新型广义RBF神经网络在混沌时间序列预测中的研究[J].物理学报,2005, 10: 4569-4577
9. Zhu Jiayuan, Bo Ren, Heng Xi Zhang etal. Time Series Prediction via New Support Vector Machines[C]. In: Proceedings of ICMLC 2002, IEEE, China, Beijing, 2002: 364-366
10.李元诚,方廷健,于尔铿.短期负荷预测的支持向量机方法研究[J].中国电机工程学报, 2003, 23(6): 55-59
11.刘涵,刘丁,郑岗,等.基于最小二乘支持向量机的天然气负荷预测[J].化工学报, 2004, 55(5): 828-832.
12. Hammel S M. A noise reduction method for chaotic system[J]. Physics Letter A, 1990, 148 (8-9): 421-428
13. Schreiber T. Extremely simple nonlinear noise-reduction method[J]. Physical Review E, 1993, 47 (4): 2401-2404
14.何岱海,徐健学,陈永红.非线性动力学相空间重构中小波变换方法研究[J].振动工程学报,1999, 12(1): 27-32
15.游荣义,陈忠,徐慎初,吴伯僖.基于小波变换的混沌信号相空间重构研究[J].物理学报, 2004, 53(9): 2882-2888
16. Vautard R, Yiou P, Ghil M. Singular-spectrum analysis: a toolkit for short, noisy chaotic signals[J]. Physica D, 1992, 59: 95-126
17. Cawley R, Hsu G H. Local-geometric-projection method for noise reduction in chaotic maps and flows[J]. Physica Review A, 1992, 46(6): 3057-3082
18. Grassberger P, Hegger R, Kantz H, Schaffrath C, Schreiber T. On noise reduction methods for chaotic data[J]. Chaos, 1993, 3 (2) : 127-141
19. Schreiber T, Richter M. Fast nonlinear projective filtering in a data stream[J]. International Journal of Bifurcation and chaos, 1999, 9(10): 2039-2045
20.徐金梧,吕勇,王海峰.局部投影算法及其在非线性时间序列分析中的应用[J].机械工程学报, 2003, 39(9): 146-150
21.胡静,杨宗凯.基于局部投影方法的混沌信号去噪[J].华中科技大学学报(自然科学版), 2002, 04: 70-72
22.吴迪嘉,罗汉文,宋文涛,施聪.局部线性投影算法在混沌键控系统中进行噪声去除的应用[J].上海交通大学学报, 2002, 36(6): 757-760
23.李振平,闻邦椿.刚性转子轴承系统的复杂非线性动力学行为研究[J].振动与冲击, 2005, 24(3): 36-39
24. Grassberger P, Procaccia I. Estimation of the Kolmogorov entropy from a chaotic signal[J]. Psysica Review A, 1983, 28: 2591-2593
25. Zhang G P, Patuwo B E, Hu M Y. A simulation study of artificial neural networks for nonlinear time-series forecasting[J]. Computers & Operations Research, 2001, 28(3): 381-396 .
26. Cowper M R, Mulgrew B, Unsworth C P. Nonlinear prediction of chaotic signals using a normalized radial basis function network[J]. Signal Processing, 2002, 82(5): 775-789 .
27.胡寿松,张正道.基于神经网络的非线性时间序列故障预报[J].自动化学报, 2007, 07: 744-748
28.杨金芳,翟永杰,王东风,徐大平.基于支持向量回归的时间序列预测[J].中国电机工程学报, 2005, (17): 110-114
29. Niu Lin, Zhao Jian-guo, Du Zhi-gang, Jin Xiao-ling. Application of Time Series Forecasting Algorithm via Support Vector Machines to Power System Wide-area Stability Prediction[J]. IEEE/PES Transmission and Distribution Conference and Exhibition. Asia and Pacific. 2005: 1-6 .
30.董辉,傅鹤林,冷伍明.支持向量机的时间序列回归与预测[J].系统仿真学报, 2006, 07: 1785-1788
31. N. Vladimir, V. Vapnik. An overview of statistical learning theory[J]. IEEE Transaction on Neural Networks, 1999, 10(5): 988-999
32.张学工.关于统计学习理论与支持向量机[M].自动化学报, 2000, 26(1): 32-42
33.邓乃杨,田英杰.数据挖掘中的新方法—支持向量机[M].北京:科学出版社, 2004
34. Vapnik V. The nature of statistical learning theory[M]. 2nd edition. New York: Springer-Verlag, 1999
35. Shawe J, Cristianini N.赵玲玲,翁苏明,曾华军等译.模式识别的核方法[M].北京:机械工业出版社, 2006
36. Suykens A K, Gestel T Van, Brabanter J De, Moor B De, Vandewalle J. Least Squares Support Vector Machines[J]. World Scientific Pub. Co. , Singapore, 2002
37.张贤达,保铮.非平稳信号分析与处理[M].北京:国防工业出版社, 1998
38. Norden E. Huang, Zheng Shen, Steven R. Long, etal. The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis[J]. Proc. R. Soc. Lond. A, 1998, 454: 903-995
39.张西宁.大型回转机械运行状态检测和预报的研究[D].西安:西安交通大学机械工程学院, 1998.
40.裴树毅.基于神经网络的预测理论及其应用[D].西安:西安交通大学机械工程学院, 1994.
41. Thissen U, Brakel R V, Weijer A P D, et al. Using support vector machines for time series prediction[J]. Chemometrics and Intelligent Laboratory Systems, 2003, 69 (1-2): 35-49.
42.李凌均,张周锁,何正嘉.基于支持向量机的机械设备状态趋势预测研究[J].西安交通大学学报, 2004, 38(3): 230-234.
43. YANO M, HOMMA N, SAKAIM, ABE K. Phase space reconstruction from observed time series using Lyapunov spectrum analysis[C]. Proc of SICE Annual Conference 2002. Osaka (Japan): SICE, 2002: 721-726.
44.周佰成,翟延慧,李向前.Takens理论和小波分析在非线性预测中的应用[J].吉林大学自然科学学报, 2001, 7: 21-28.
45.盛昭瀚,姚洪兴,王海燕等.混沌动力系统的重构预测与控制[M].中国经济出版社, 2003
46.张胜,刘红星,高敦堂等.NN非线性时间序列预测模型输入延时的确定[J].东南大学学报(自然科学版), 2002, 32(6): 905-908
47. Kim H S, Eykholt R J, Salas D. Nonlinear dynamics, delay times, and embedding windows[J]. Physica: D, 1999, 127(1): 48-60.
48. Kirkpatrick S, Gelatt C D J, Vecchi M P. Optimization by simulated annealing[J]. Science, 2003, 220: 671-680.
49. De Castro L N, Von Zuben F J. Learning and mization Using Clonal Selection Principle[J]. IEEE Transaction on Evolutionary Computation, 2002, 6(3): 239-251
50. Shisanu T, Parallel genetic algorithm with parameter adaptation[J]. Information Processing Letters, 2002, 82: 47-54
51. Kennedy J, Eberhart R C. Particle swarm optimization[C]. IEEE International Conference on Neural Networks. Perth, Piscataway, NJ, Australia: IEEE Service Center, 1995, 4: 1942-1948
52.李爱国,覃征,鲍复民,等.粒子群优化算法[J].计算机工程与应用, 2002, 39(21): 1-3
53. Kennedy J, Eberhart R C. A discrete binary version of the particle swarm algorithm[C]. Proceedings of the World Multi-conference on Systemics, Cybernetics and Informatics 1997, Piscataway, 1997, NJ: 4104-4109
54. Shi Y, Eberhart R C. Parameter selection in particle swarm optimization[C]. Proc of the 7th Annual Conf on Evolutionary Programming. Washington DC, 1998: 591-600
55. Ito K, Nakano R. Optimizing Support Vector Regression Hyper-parameters Based On Cross-Validation[C]. Proceedings of the International Joint Conference on Neural Networks, 2003, 7(3): 2077-2082