基于独立成分分析的多元回归方法研究
|
摘要
本文以多变量回归模型问题为主要内容,利用现代信号处理技术,针对现有基于数据驱动的多变量统计过程预测方法的不足,引入独立元分析(Independent Component Analysis,简称ICA)应对实际工业过程中非高斯问题。
本文在分析ICA提取的独立元的性质的基础上,提出了基于修正独立元分析的多变量回归方法。回归分析中提取的特征成分包含目标信息的丰富程度对模型的精度影响很大,为了提取包含目标信息丰富信息的特征成分,本文首先利用修正的ICA方法提取独立元建立类似主元的回归模型,接着对建立的模型进行改进：对输入变量进行用于回归的特征独立元提取,使得提取的输入变量的独立元和目标变量的独立元互信息最大,从而建立回归模型。接着为了解决非线性问题,介绍了在输出空间利用核独立元分析提取非线性的独立元用于回归建模。
本文的基于ICA的回归方法对田纳西—库兹曼过程的部分过程测量变量和成分测量变量进行了预测研究,通过一些过程测量变量把本文的基于ICA的回归方法和偏最小二乘回归方法进行了比较,基于TE过程的仿真实验证实了其有效性。接着对部分成分测量变量进行了预测研究,证实核独立元提取的特征对非线性回归的有效性。
With the help of modern signal processing techniques, based on existing data-driven multivariate statistical forecasting methods of the process predictions methods. This paper studies process prediction methods within the application of data-driven multivariate statisti-cal process predictions methods in the flow industry. Flow industry is an essential part of our national economy. As the process involves high temperature, high pressure and high risk. the importance of its quality prediction is increasingly prominent.
Based on the analysis of ICs extracted from the nature of an independent element on the basis of the proposed amendments based on independent component analysis of multi-variable regression method. Regression analysis of the characteristics of components to extract target information included in the model of the abundance of great influence on the accuracy, In this paper, the first to use ICA to extract the amendment to establish a similar independent component of the PCA regression model, Followed by the establishment of the model to improve:The input variables for the characteristics of independent reunification Extraction, Makes extraction of the input variables and objectives of Independent Independent variables largest mutual information, regression model to build. In order to solve nonlinear problems and then introduced in the output space using the nuclear extraction independent component analysis for non-linear regression prediction.
The method is applied to the quality prediction of Tennessee—Eastman Process in this paper. The prediction performance of the proposed approach based ICA is compared to PLSR using some process variables examples. It is proved to be effective through the simulate application upon TE process. And use KICA to extract features of target variables to predict the quality of some process. The simulation result shows that the ICA method can capture the nonlinear dynamic features effectively, and predict the process quality successfully.
本文在分析ICA提取的独立元的性质的基础上,提出了基于修正独立元分析的多变量回归方法。回归分析中提取的特征成分包含目标信息的丰富程度对模型的精度影响很大,为了提取包含目标信息丰富信息的特征成分,本文首先利用修正的ICA方法提取独立元建立类似主元的回归模型,接着对建立的模型进行改进：对输入变量进行用于回归的特征独立元提取,使得提取的输入变量的独立元和目标变量的独立元互信息最大,从而建立回归模型。接着为了解决非线性问题,介绍了在输出空间利用核独立元分析提取非线性的独立元用于回归建模。
本文的基于ICA的回归方法对田纳西—库兹曼过程的部分过程测量变量和成分测量变量进行了预测研究,通过一些过程测量变量把本文的基于ICA的回归方法和偏最小二乘回归方法进行了比较,基于TE过程的仿真实验证实了其有效性。接着对部分成分测量变量进行了预测研究,证实核独立元提取的特征对非线性回归的有效性。
With the help of modern signal processing techniques, based on existing data-driven multivariate statistical forecasting methods of the process predictions methods. This paper studies process prediction methods within the application of data-driven multivariate statisti-cal process predictions methods in the flow industry. Flow industry is an essential part of our national economy. As the process involves high temperature, high pressure and high risk. the importance of its quality prediction is increasingly prominent.
Based on the analysis of ICs extracted from the nature of an independent element on the basis of the proposed amendments based on independent component analysis of multi-variable regression method. Regression analysis of the characteristics of components to extract target information included in the model of the abundance of great influence on the accuracy, In this paper, the first to use ICA to extract the amendment to establish a similar independent component of the PCA regression model, Followed by the establishment of the model to improve:The input variables for the characteristics of independent reunification Extraction, Makes extraction of the input variables and objectives of Independent Independent variables largest mutual information, regression model to build. In order to solve nonlinear problems and then introduced in the output space using the nuclear extraction independent component analysis for non-linear regression prediction.
The method is applied to the quality prediction of Tennessee—Eastman Process in this paper. The prediction performance of the proposed approach based ICA is compared to PLSR using some process variables examples. It is proved to be effective through the simulate application upon TE process. And use KICA to extract features of target variables to predict the quality of some process. The simulation result shows that the ICA method can capture the nonlinear dynamic features effectively, and predict the process quality successfully.
引文
1. Dunteman, GH. Principal component analysis [M], London:SAGE publication LTD, 1989.
2. Jackson, J.E. A user's guide to principal components [M], New York:Wiley,1991.
3. Geladi, P. and Kowalshi, B.R. partial least squares regression:A tutorial [J], Analytica Chimica Acta,1986,185 (1):1-17.
4. Hoskuldsson, A. PLS regression methods [J], J. Chemometrics,1988,2:211-228.
5.王惠文.偏最小二乘回归方法及其应用[M],北京：国防工业出版社,1999.
6. Wise, B.M., Richer, N.L., Veltkamp, D.F. and Kowalshi, B.R. A theoretical basis for the use of principal component models for modeling multivariate process [J], Process control and quality,1990,1:41-51.
7. Weil, S.A.V., Tucker, W.T., Faltin, R.W. and Doganksoy, N. Algorithmic statistical process control:concepts and an application [J], Technometrics,1992,34 (3):286-297.
8. Kourti, T., Lee, J. and MacGregor, J.F. Experience with industrial applications of projection methods for multivariate statistical process control [J], Computers & Chemical Engineering,1996,20 (1):745-750.
9. John F. MacGregor Using On-line process data to improve quality:Challenges for statisticians [J], International statistical review,1997,65(3):309-323.
10. Teppola, P., Mujunen, S.P., Minkkinen, P., Puijola, T., and Pursiheimo, P. Principal component analysis, contribution plots and feature weights in the monitoring of sequential process data from a paper machine's wet end [J], Chemometrics and Intelligent Laboratory systems,1998,44:307-317.
11.朱孔伟.基于主元分析的电厂锅炉故障检测与诊断[D],南京：东南大学,2006.
12.姚林,阳建宏,何飞,徐金梧.基于核偏最小二乘分析的锌层重量预测模型[J],控制工程,2008,4(1)：1671-1680.
13.张飞,宗胜悦,谢新亮.基于核偏最小二乘分析的GM-AGC厚度预测方法[J],冶金设备,2008,8(6)：171-174.
14. Gulandoust, M.T., Morris, A.J. and Tham, M.T. Adaptive estimation algorithm for inferential control [J]. Industrial and Engineering Chemistry Research.1988,2(7): 1658-1664.
15.杨英华,陆宁云,江云波,马立玲,王福利.一种基于非线性主成分回归的过程监测及故障诊断方法[J],信息与控制,2002,31(3)：272-276.
16. Tham M.T., Montague, G.A.; Morris, A.J.; Lant, P.A., Soft-sensors for process estimation and inferential control [J]. J. Process Control,1991,1(2):3-14.
17. Mejdell T. and Skogestad S. Estimation of distillation compositions from multiple temperature measurements using partial-least-squares regression [J], Industrial and Engineering Chemistry Research,1991,30(3):2543-2555.
18. Kresta J.V., Marlin T.E. and MacGregor J.F. Development of inferential process models using PLS [J]. Computers and Chemical Engineering,1994,18(5):597-611.
19.王惠文等.偏最小二乘回归线性与非线性方法[M],北京：国防工业出版社,2006.
20. Hartnett M.K., G Lightbody, GW. Irwin. Dynamic inferential estimation using principal components regression (PCR) [J], Chemometrics and intelligent laboratory systems,1998, 40(3):215-.224.
21. Zhang J. Inferential feedback control of distillation composition based on PCR and PLS models [C]. Proceedings of American Control Conference,2001.8(1):1196-1201.
22. Weil, S.A.V., Tucker, W.T., Faltin, R.W. and Doganksoy, N. Algorithmic statistical process control:concepts and an application [J], Technometrics,1992,34(3):286-297.
23. Cho J-H, Lee J-M, Choi S W, et al. Fault identification for process monitoring using kernel principal component analysis [J]. Chemical Engineering Science,2005,60(2): 279-283.
24. Holcomb, T.R. and Morari, M. PLS/Neural networks [J], Computers & Chemical Engineering,1992,16(4):393-411.
25. Kourti, T. and MacGregor, J.F. Process analysis, monitoring and diagnosis, using multivariate projection methods [J], Chemometrics and Intelligent Systems Laboratory, 1995,28(7):3-21.
26. Raich, A. and Cinar A. Statistical process monitoring and disturbance diagnosis in multivariable continuous process [J], AIChE J.,1996,42(1):995-1009.
27. Nelson, P. R.C., Taylor, P. A. and MacGregor, J.F. Missing data methods in PCA and PLS: score calculations with incomplete observations [J], Chemometrics and Intelligent laboratory systems,1996,35(8):45-65.
28.高翔.独立分量分析与盲源分离在流程工业过程监控中的应用[D],江南大学,2008
29.盛骤等.概率论与数理统计[M],北京：高等教育出社,2001。
30.刘磊,张宇明,钱积新.统计过程控制在连续过程中的应用[J],化工自动化及仪表, 1997,24(2):71-76.
31. Wold S. Cross-validatory estimation of the number of components in Factor and principal components models [J], Technometrics,1978,20(4):397-405.
32. Rannar, S. MacGregor, J.F. Wold, S. Adaptive batch monitoring using hierarchical PCA [J], Chemometrics and Intelligent Laboratory systems,1998,41(4):73-81.
33. Zhang J.2001. Inferential feedback control of distillation composition based on PCR and PLS models [C. Proceedings of American Control Conference,1196-1201.
34. Cover T. and Thomas J. Elements of information theory [D]. New York:Wiley,1991.
35.郭辉.基于ICA的工业过程监控研究[D],北京：北京化工大学,2006.
36. Roberts, S, J. and Everson, R. Independent component analysis:principles and practice [M]. Cambridge University Press,2001.
37. Girolami, M. An alternative perspective on adaptive independent component analysis algorithms [J]. Neural Computation,1998,10(3):2103-2114.
38. Lee, T. W. Girolami, M.and SejnwoskiT. Independent component analysis Using an extended infomax algorithm for mixed sub-Gaussian and super-Gaussian sources [J]. Neural Computation,1999,11(2):41-7441.
39. Lee, T-W Lewicki. M., S., T. J.ICA mixture models for unsupervised classification of Non-Gaussian classes automatic context switching in blind signal separation [J]. IEEE Transaction on Pattern Analysis and Machine Intelligence,2000,22(10):1-12.
40. Comom Pierre. Independent component analysis [J]. Signal Processing,1994,36(3): 287-314.
41. Bell A J, Sejnowski T J. An information-maximization approach to blind separation and blind deconvolution [J].Neural Computation,1995,7(6):1129-1159.
42. Hyvarinen A. Survey on Independent Component Analysis [J]. Neural Computing Surveys,1999,2(4):94-128.
43. A. Hyvarinen. Fast and robust fixed-point algorithms for independent component analysis [J], IEEE Transactions on,1999,10(3):1129-1159.
44. Kyungpil Kim, Jong-Min Lee, In-Beum Lee. A novel multivariate regression approach based on kernel partial least squares with orthogonal signal correction [J], Chemometrics and Intelligent Laboratory Systems,2005,7 (9) 22-30.
45. MATFORSK Norwegian Food Research institute. independent component analysis and regression applied on sensor data [J]. Journal Chemometrics,2005,1(9):171-179.
46. Xueguang Shao, Wei Wang, Zhenyu Hou, WenSheng Cai. A new regression method based on independent component analysis [J], Talanta,2006,6(9):676-680.
47. Nojun Kwak, Chunghoon Kim, Hwangnam Kim. Dimensionality reduction based on ICA for regression problems [J]. Neurocomputine,2008,7 (1):2596-2603.
48. Jong-Min Lee, S.Joe Qin, In-Beum Lee. Fault detection and diagnosis of multivariate process based on modified independent component analysis [J], AIChE Journal,2006, 52(6) 3501-3514.
49. Ji-Hoon Cho, Jong-Min Lee, et al. Fault identi.cation for process monitoring using kernel principal component analysis [J], ChemICAl Engineering Science,2005,6(4):279-288.
50.陈玉山,席斌.基于核独立成分分析和BP网络的人脸识别[J],计算机工程与应用,2007,43(26)：230-232.
51.蒋浩天等著,段建民译.工业系统的故障检测与诊断[M],北京：机械工业出版社,2001.
2. Jackson, J.E. A user's guide to principal components [M], New York:Wiley,1991.
3. Geladi, P. and Kowalshi, B.R. partial least squares regression:A tutorial [J], Analytica Chimica Acta,1986,185 (1):1-17.
4. Hoskuldsson, A. PLS regression methods [J], J. Chemometrics,1988,2:211-228.
5.王惠文.偏最小二乘回归方法及其应用[M],北京：国防工业出版社,1999.
6. Wise, B.M., Richer, N.L., Veltkamp, D.F. and Kowalshi, B.R. A theoretical basis for the use of principal component models for modeling multivariate process [J], Process control and quality,1990,1:41-51.
7. Weil, S.A.V., Tucker, W.T., Faltin, R.W. and Doganksoy, N. Algorithmic statistical process control:concepts and an application [J], Technometrics,1992,34 (3):286-297.
8. Kourti, T., Lee, J. and MacGregor, J.F. Experience with industrial applications of projection methods for multivariate statistical process control [J], Computers & Chemical Engineering,1996,20 (1):745-750.
9. John F. MacGregor Using On-line process data to improve quality:Challenges for statisticians [J], International statistical review,1997,65(3):309-323.
10. Teppola, P., Mujunen, S.P., Minkkinen, P., Puijola, T., and Pursiheimo, P. Principal component analysis, contribution plots and feature weights in the monitoring of sequential process data from a paper machine's wet end [J], Chemometrics and Intelligent Laboratory systems,1998,44:307-317.
11.朱孔伟.基于主元分析的电厂锅炉故障检测与诊断[D],南京：东南大学,2006.
12.姚林,阳建宏,何飞,徐金梧.基于核偏最小二乘分析的锌层重量预测模型[J],控制工程,2008,4(1)：1671-1680.
13.张飞,宗胜悦,谢新亮.基于核偏最小二乘分析的GM-AGC厚度预测方法[J],冶金设备,2008,8(6)：171-174.
14. Gulandoust, M.T., Morris, A.J. and Tham, M.T. Adaptive estimation algorithm for inferential control [J]. Industrial and Engineering Chemistry Research.1988,2(7): 1658-1664.
15.杨英华,陆宁云,江云波,马立玲,王福利.一种基于非线性主成分回归的过程监测及故障诊断方法[J],信息与控制,2002,31(3)：272-276.
16. Tham M.T., Montague, G.A.; Morris, A.J.; Lant, P.A., Soft-sensors for process estimation and inferential control [J]. J. Process Control,1991,1(2):3-14.
17. Mejdell T. and Skogestad S. Estimation of distillation compositions from multiple temperature measurements using partial-least-squares regression [J], Industrial and Engineering Chemistry Research,1991,30(3):2543-2555.
18. Kresta J.V., Marlin T.E. and MacGregor J.F. Development of inferential process models using PLS [J]. Computers and Chemical Engineering,1994,18(5):597-611.
19.王惠文等.偏最小二乘回归线性与非线性方法[M],北京：国防工业出版社,2006.
20. Hartnett M.K., G Lightbody, GW. Irwin. Dynamic inferential estimation using principal components regression (PCR) [J], Chemometrics and intelligent laboratory systems,1998, 40(3):215-.224.
21. Zhang J. Inferential feedback control of distillation composition based on PCR and PLS models [C]. Proceedings of American Control Conference,2001.8(1):1196-1201.
22. Weil, S.A.V., Tucker, W.T., Faltin, R.W. and Doganksoy, N. Algorithmic statistical process control:concepts and an application [J], Technometrics,1992,34(3):286-297.
23. Cho J-H, Lee J-M, Choi S W, et al. Fault identification for process monitoring using kernel principal component analysis [J]. Chemical Engineering Science,2005,60(2): 279-283.
24. Holcomb, T.R. and Morari, M. PLS/Neural networks [J], Computers & Chemical Engineering,1992,16(4):393-411.
25. Kourti, T. and MacGregor, J.F. Process analysis, monitoring and diagnosis, using multivariate projection methods [J], Chemometrics and Intelligent Systems Laboratory, 1995,28(7):3-21.
26. Raich, A. and Cinar A. Statistical process monitoring and disturbance diagnosis in multivariable continuous process [J], AIChE J.,1996,42(1):995-1009.
27. Nelson, P. R.C., Taylor, P. A. and MacGregor, J.F. Missing data methods in PCA and PLS: score calculations with incomplete observations [J], Chemometrics and Intelligent laboratory systems,1996,35(8):45-65.
28.高翔.独立分量分析与盲源分离在流程工业过程监控中的应用[D],江南大学,2008
29.盛骤等.概率论与数理统计[M],北京：高等教育出社,2001。
30.刘磊,张宇明,钱积新.统计过程控制在连续过程中的应用[J],化工自动化及仪表, 1997,24(2):71-76.
31. Wold S. Cross-validatory estimation of the number of components in Factor and principal components models [J], Technometrics,1978,20(4):397-405.
32. Rannar, S. MacGregor, J.F. Wold, S. Adaptive batch monitoring using hierarchical PCA [J], Chemometrics and Intelligent Laboratory systems,1998,41(4):73-81.
33. Zhang J.2001. Inferential feedback control of distillation composition based on PCR and PLS models [C. Proceedings of American Control Conference,1196-1201.
34. Cover T. and Thomas J. Elements of information theory [D]. New York:Wiley,1991.
35.郭辉.基于ICA的工业过程监控研究[D],北京：北京化工大学,2006.
36. Roberts, S, J. and Everson, R. Independent component analysis:principles and practice [M]. Cambridge University Press,2001.
37. Girolami, M. An alternative perspective on adaptive independent component analysis algorithms [J]. Neural Computation,1998,10(3):2103-2114.
38. Lee, T. W. Girolami, M.and SejnwoskiT. Independent component analysis Using an extended infomax algorithm for mixed sub-Gaussian and super-Gaussian sources [J]. Neural Computation,1999,11(2):41-7441.
39. Lee, T-W Lewicki. M., S., T. J.ICA mixture models for unsupervised classification of Non-Gaussian classes automatic context switching in blind signal separation [J]. IEEE Transaction on Pattern Analysis and Machine Intelligence,2000,22(10):1-12.
40. Comom Pierre. Independent component analysis [J]. Signal Processing,1994,36(3): 287-314.
41. Bell A J, Sejnowski T J. An information-maximization approach to blind separation and blind deconvolution [J].Neural Computation,1995,7(6):1129-1159.
42. Hyvarinen A. Survey on Independent Component Analysis [J]. Neural Computing Surveys,1999,2(4):94-128.
43. A. Hyvarinen. Fast and robust fixed-point algorithms for independent component analysis [J], IEEE Transactions on,1999,10(3):1129-1159.
44. Kyungpil Kim, Jong-Min Lee, In-Beum Lee. A novel multivariate regression approach based on kernel partial least squares with orthogonal signal correction [J], Chemometrics and Intelligent Laboratory Systems,2005,7 (9) 22-30.
45. MATFORSK Norwegian Food Research institute. independent component analysis and regression applied on sensor data [J]. Journal Chemometrics,2005,1(9):171-179.
46. Xueguang Shao, Wei Wang, Zhenyu Hou, WenSheng Cai. A new regression method based on independent component analysis [J], Talanta,2006,6(9):676-680.
47. Nojun Kwak, Chunghoon Kim, Hwangnam Kim. Dimensionality reduction based on ICA for regression problems [J]. Neurocomputine,2008,7 (1):2596-2603.
48. Jong-Min Lee, S.Joe Qin, In-Beum Lee. Fault detection and diagnosis of multivariate process based on modified independent component analysis [J], AIChE Journal,2006, 52(6) 3501-3514.
49. Ji-Hoon Cho, Jong-Min Lee, et al. Fault identi.cation for process monitoring using kernel principal component analysis [J], ChemICAl Engineering Science,2005,6(4):279-288.
50.陈玉山,席斌.基于核独立成分分析和BP网络的人脸识别[J],计算机工程与应用,2007,43(26)：230-232.
51.蒋浩天等著,段建民译.工业系统的故障检测与诊断[M],北京：机械工业出版社,2001.