基于非线性偏最小二乘的特征提取方法研究
|
摘要
特征提取是模式识别,尤其是图像识别中最为核心的问题之一,提取的结果直接影响最终的分类识别效果。作为“第二代多元统计分析技术”的偏最小二乘(PLS)因为综合了多元线性回归分析、主成分分析和典型相关分析的基本功能,在图像识别的特征提取方面有着绝对的优势。它的基本思想就是在自变量空间和因变量空间中分别找出各自具有特征代表性的主成分,且这对成分之间的相关性最大。本文以传统的偏最小二乘为基础,从外部样本变换和内部成分间非线性映射这两种思路出发,对其进行了非线性扩展,并将其应用于特征提取和图像识别方面,取得了较好的效果。
本文的主要工作和研究成果如下：
(1)对多种偏最小二乘实用算法进行了对比、分析和改进,并通过实验进行验证。分别从自变量的预处理、因变量的生成和自变量与因变量的约束关系角度出发对典型算法进行了改进,提出了基于正交约束条件的偏最小二乘算法。该方法以非迭代的奇异值分解方法为基础,首先使用正交约束去除自变量中与因变量正交的成分,然后使用共轭正交条件进行偏最小二乘建模,最后通过多组实验验证了算法的可行性和有效性。
(2)基于核函数思想,深入研究了基于外部样本变换的非线性偏最小二乘方法。通过核函数方法解决了样本映射时的维数灾难和非线性形式难于表达的难题,并从核函数的选择和构造两方面探讨了核函数的确定问题。通过核校准和将全局核与局部核组合的方法得到最佳的核函数,然后在偏最小二乘中使用核技巧,得到了可以更好地提取非线性特征的核偏最小二乘。基于非迭代的奇异值分解法提出了一种新的核偏最小二乘,并加入正交约束对其进行改进,提高了算法性能。
(3)以内部成分间非线性映射的思想为基础,深入研究了模糊偏最小二乘、神经网络偏最小二乘和模糊神经网络偏最小二乘等方法。分别通过级联和相互融合的方式对模糊模型和神经网络模型进行结合,得到了不同的模糊神经网络模型。将单独的模糊模型、神经网络模型和综合的模糊神经网络模型分别嵌入到迭代形式的偏最小二乘中对成对的成分进行非线性映射,得到了不同形式的非线性偏最小二乘方法,并通过对自变量和因变量成分进行相同的迭代更新对其进行改进,进一步提高了各种方法提取非线性特征的能力。多次实验表明这类算法是有效的,尤其是基于模糊神经网络的偏最小二乘算法的性能最好。
Feature extraction is one of the core problems in pattern recognition, especially in image recognition. Extracted result will directly affect the final classification and recognition result. Partial least squares(PLS) which is called "the second generation of multivariate statistical analysis" synthesizes basic functions of multiple linear regression analysis, principal component analysis and canonical correlation analysis, so it has an absolute advantage in the feature extraction of image recognition. Its basic idea is to find principal components which represent the characteristics of independent variables space and dependent variables space respectively, and meanwhile correlation between the components is the largest. In this paper, based on the traditional partial least squares, it will be expanded to nonlinear with methods of transforming samples externally and modeling nonlinear mapping between components internally. And apply it to feature extraction and image recognition, a better effect will be achieved.
The main work and research results of this paper can be summarized as follows:
(1)A variety of practical algorithms about Partial least squares are compared, analyzed, improved and verified by experiment. From the points of preprocessing independent variables, producing dependent variables and making restraint on them, we improved typical algorithms and proposed a new algorithm:partial least squares based on orthogonal constraint. The algorithm which is based on the non-iterative singular value decomposition partial least squares method, removes the components which are orthogonal to dependent variables from independent variables firstly, and then use the PLS method to model with condition of conjugate orthogonal. At last, through several experiments, we demonstrated the feasibility and effectiveness of the algorithm.
(2)Based on the idea of kernel function, we deep researched the Kernel partial least squares, the typical one of nonlinear PLS algorithms, which are based on the idea of transforming samples externally. With the method of kernel function, we solved the dimension disaster of sample mapping and the difficult problem of expressing the nonlinear form. Besides, we researched the problem of identifying kernel function from both the choice and the construction of kernel function in depth. With the method of kernel alignment, we get the best kernel function by combining the global kernel and the local kernel together. Then in process of partial least squares, by using the best kernel, we get a kernel partial least squares which can extract nonlinear features better. At last, based on the non-iterative singular value decomposition PLS method, we proposed a novel kernel partial least squares algorithm. And added the orthogonal constraint to it, performance of the algorithm was improved.
(3)Based on the idea of modeling nonlinear mapping between components internally, we deep researched several kinds of nonlinear PLS algorithms, such as fuzzy partial least squares(FPLS), neural network partial least squares (NNPLS) and fuzzy neural network partial least squares(FNNPLS). By combining fuzzy model and neural network model with method of cascade and mutual integration respectively, we obtained different fuzzy neural network models. When embed separate fuzzy model, neural network model and integrated fuzzy neural network model into the iterative partial least squares, we will find different types of nonlinear mapping and get different forms of nonlinear PLS method accordingly. Besides, we make the same iterative update on both independent variables and dependent variables to improve ability of extracting nonlinear features of the algorithms. Many experiments show that these methods are all effective. And especially the FNNPLS method can give the best result.
本文的主要工作和研究成果如下：
(1)对多种偏最小二乘实用算法进行了对比、分析和改进,并通过实验进行验证。分别从自变量的预处理、因变量的生成和自变量与因变量的约束关系角度出发对典型算法进行了改进,提出了基于正交约束条件的偏最小二乘算法。该方法以非迭代的奇异值分解方法为基础,首先使用正交约束去除自变量中与因变量正交的成分,然后使用共轭正交条件进行偏最小二乘建模,最后通过多组实验验证了算法的可行性和有效性。
(2)基于核函数思想,深入研究了基于外部样本变换的非线性偏最小二乘方法。通过核函数方法解决了样本映射时的维数灾难和非线性形式难于表达的难题,并从核函数的选择和构造两方面探讨了核函数的确定问题。通过核校准和将全局核与局部核组合的方法得到最佳的核函数,然后在偏最小二乘中使用核技巧,得到了可以更好地提取非线性特征的核偏最小二乘。基于非迭代的奇异值分解法提出了一种新的核偏最小二乘,并加入正交约束对其进行改进,提高了算法性能。
(3)以内部成分间非线性映射的思想为基础,深入研究了模糊偏最小二乘、神经网络偏最小二乘和模糊神经网络偏最小二乘等方法。分别通过级联和相互融合的方式对模糊模型和神经网络模型进行结合,得到了不同的模糊神经网络模型。将单独的模糊模型、神经网络模型和综合的模糊神经网络模型分别嵌入到迭代形式的偏最小二乘中对成对的成分进行非线性映射,得到了不同形式的非线性偏最小二乘方法,并通过对自变量和因变量成分进行相同的迭代更新对其进行改进,进一步提高了各种方法提取非线性特征的能力。多次实验表明这类算法是有效的,尤其是基于模糊神经网络的偏最小二乘算法的性能最好。
Feature extraction is one of the core problems in pattern recognition, especially in image recognition. Extracted result will directly affect the final classification and recognition result. Partial least squares(PLS) which is called "the second generation of multivariate statistical analysis" synthesizes basic functions of multiple linear regression analysis, principal component analysis and canonical correlation analysis, so it has an absolute advantage in the feature extraction of image recognition. Its basic idea is to find principal components which represent the characteristics of independent variables space and dependent variables space respectively, and meanwhile correlation between the components is the largest. In this paper, based on the traditional partial least squares, it will be expanded to nonlinear with methods of transforming samples externally and modeling nonlinear mapping between components internally. And apply it to feature extraction and image recognition, a better effect will be achieved.
The main work and research results of this paper can be summarized as follows:
(1)A variety of practical algorithms about Partial least squares are compared, analyzed, improved and verified by experiment. From the points of preprocessing independent variables, producing dependent variables and making restraint on them, we improved typical algorithms and proposed a new algorithm:partial least squares based on orthogonal constraint. The algorithm which is based on the non-iterative singular value decomposition partial least squares method, removes the components which are orthogonal to dependent variables from independent variables firstly, and then use the PLS method to model with condition of conjugate orthogonal. At last, through several experiments, we demonstrated the feasibility and effectiveness of the algorithm.
(2)Based on the idea of kernel function, we deep researched the Kernel partial least squares, the typical one of nonlinear PLS algorithms, which are based on the idea of transforming samples externally. With the method of kernel function, we solved the dimension disaster of sample mapping and the difficult problem of expressing the nonlinear form. Besides, we researched the problem of identifying kernel function from both the choice and the construction of kernel function in depth. With the method of kernel alignment, we get the best kernel function by combining the global kernel and the local kernel together. Then in process of partial least squares, by using the best kernel, we get a kernel partial least squares which can extract nonlinear features better. At last, based on the non-iterative singular value decomposition PLS method, we proposed a novel kernel partial least squares algorithm. And added the orthogonal constraint to it, performance of the algorithm was improved.
(3)Based on the idea of modeling nonlinear mapping between components internally, we deep researched several kinds of nonlinear PLS algorithms, such as fuzzy partial least squares(FPLS), neural network partial least squares (NNPLS) and fuzzy neural network partial least squares(FNNPLS). By combining fuzzy model and neural network model with method of cascade and mutual integration respectively, we obtained different fuzzy neural network models. When embed separate fuzzy model, neural network model and integrated fuzzy neural network model into the iterative partial least squares, we will find different types of nonlinear mapping and get different forms of nonlinear PLS method accordingly. Besides, we make the same iterative update on both independent variables and dependent variables to improve ability of extracting nonlinear features of the algorithms. Many experiments show that these methods are all effective. And especially the FNNPLS method can give the best result.
引文
[1]颜学峰.高维复杂模式识别的新方法.浙江大学博士学位论文,2002
[2]杨淑莹.图像模式识别.第1版.北京：清华大学出版社,北京交通大学出版社,2005
[3]边肇祺,张学工.模式识别.第2版.北京：清华大学出版社,2000
[4]吴介,裘正定.掌纹识别中的特征提取算法综述.北京电子科技学院学报.2005,13(2)：86-92
[5]刘青山,卢汉清,马颂德.综述人脸识别中的子空间方法.自动化学报.2003,29(6)：900-911
[6]Turk M., Pentland A.. Eigenfaces for recognition. J. of Cognitive Neuroscience.1991, 3(1):71-86
[7]Kibry M., Sirovich L.. Application of the Karhunen-loeve procedure for the characterization of human faces. IEEE Trans.on PAMI.1990,12(1):103-108
[8]Hotelling H. Relations between two sets of varieties. Biometrika,1936, (8):321-377
[9]Nguyen D V, Rocke D M. Tumor classification by partial least squares using microarray gene expression data. Bioinformatics.2002,18(1):39-50
[10]孙廷凯.增强型典型相关分析研究与应用.南京航空航天大学博士学位论文,2006
[11]Svante Wold. Personal memories of the early PLS development. Chemometrics and Intelligent Laboratory Systems.2001,58:83-84
[12]Svante Wold, Johan Trygg, Anders Berglund, et al. Some recent developments in PLS modeling. Chemometrics and Intelligent Laboratory Systems.2001,58:131-150
[13]王惠文.偏最小二乘回归方法及其应用.第1版.北京：国防工业出版社,1999
[14]王惠文.偏最小二乘回归的线性与非线性方法.第1版.北京：国防工业出版社,2005
[15]蒋红卫.偏最小二乘回归的扩展及其使用算法构建.第四军医大学硕士学位论文,2003
[16]Svante Wold, Henrik Antti, Fredrik Lindgren, et al. Orthogonal signal correction of near-infrared spectra. Chemometrics and Intelligent Laboratory Systems.1998,44:175-185
[17]M.H.Zhang, Q.S.Xu, D.L.Massart. Averaged and weighted averaged partial least squares. Analytica Chimica Acta,2004,504:279-289
[18]杨茂龙,孙权森,夏德深.二维共轭正交偏最小二乘分析及图像识别应用.计算机工程与应用.2008,44(29)：36-39
[19]Wold S., Kettaneh Wold N., SkagerbergB. Nonlinear PLS Modeling. Chemometrics IntLab System.1989, 11(7):53-65
[20]Wold S. Nonlinear PLS Modeling (Ⅱ) Spline Inner Function. Chemometrics IntLab System.1992,14(1):71-84
[21]Roman Rosipal, Leonard J.Trejo. Kernel Partial Least Squares Regression in Reproducing Kernel Hilbert Space. Journal of Machine Learning Research.2001,2:97-123
[22]Yoon Ho Bang, Chang Kyoo Yoo, In-Beum Lee. Nonlinear PLS modeling with fuzzy inference system. Chemometrics and Intelligent Laboratory Systems.2003,64:137-155
[23]Qin,S J, McAvoy,T J. Nonlinear PLS Modeling Using Neural Networks. Computers & Chemical Engineering.1992,16(4):379-391
[24]叶莺,陈崇帼,林熙.偏最小二乘回归的原理及应用.海峡预防医学杂志.2005,11(3)：3-6
[25]蒋红卫,夏结来,张春霞等.核偏最小二乘回归及其在医学中的应用.中国卫生统计.2007,24(3)：239-242
[26]蒲前超,夏继红,韩玉玲等.基于偏最小二乘回归法的河流病因诊断.河海大学学报(自然科学版).2009,37(1)：27-31
[27]李雪平,康辉明.PLS法在重庆巫山县邓家屋场滑坡稳定性影响因素分析中的应用.中国地质灾害与防治学报.2005,16(4)：1-4
[28]孙权森.基于相关投影分析的特征抽取与图像识别研究.南京理工大学博士学位论文,2006
[29]陈永国.偏最小二乘法在公共部门绩效多元评估中的应用.系统工程理论与实践.2009,29(1)：89-96
[30]楼安平,杨新.基于偏最小二乘法与支持向量机的人脸识别.计算机仿真.2005,12(22)：166-168
[31]Yegang Hu, Benyong Liu. Face Recognition Based on PLS and HMM. Pattern Recognition.2009,11:1-4
[32]郭金玉,苑玮琦.基于小波变换和部分最小二乘的掌纹识别.光电子.激光.2008,4(19)：554-557
[33]罗远胜.基于PLS的文本分类技术研究.江西师范大学硕士学位论文,2006
[34]岑芳明,王明文,王鹏鸣等.基于核偏最小二乘分类的垃圾邮件过滤.中文信息学报.2009,2(23)：48-53
[35]Elisaber Bostrom, Solveig Engen, Ingvar Eide. Mutagencicity testing of organic extracts of diesel exhaust particles after spiking with polycyclic aromatic hydrocarbons(PAH). Arch Toxicol,1998,72:645-649
[36]Avraham Lorber, Lawrence E.Wangen, Bruce R.Kowalski. A theoretical foundation for the PLS algorithm. Journal of Chemometrics.1987,1(1):19-31
[37]Giancarlo Diana, Chiara Tommasi. Cross-validation methods in principal component analysis:a comparison. Statistical Methods & Applications.2002,11:71-82
[38]J.A.Wegelin. A survey of Partial Least Squares(PLS) methods, with emphasis on the two-block case. Technical report, Department of Statistics, University of Washington, Seattle, 2000
[39]冉启文,谭立英.小波分析与分数傅里叶变换及应用.第1版.北京：国防工业出版社,2002
[40]Rayens W S. The art of maximizing covariance. University of Kentucky Technical Report,2000
[41]Johan Trygg, Svante Wold. Orthogonal projections to latent structures(O-PLS), Journal of chemometrics.2002,16:119-128
[42]Matthew Barker, William Rayens. Partial least squares for discrimination. Journal of Chemometrics.2003,17:166-173
[43]Momma M, BennettKp. (Eds.) Sparse kernel partial least squares regression, LNAI:COLT/Kernel.2003,2777:216-230
[44]曾三友,孙星明,夏利民等.基于Chebyshev多项式的自适应偏最小二乘回归建模.长沙铁道学院学报.2001,19(1)：95-99
[45]GeorgeR. Contributions to the problem of approximation of non-linear data with linear PLS in an absorption spectroscopic context. Chemometrics and Intelligent Laboratory Systems.1999,47:99-106
[46]李向阳,朱学峰,黄道平等.基于定性知识和线性PLS的间歇蒸煮过程Kappa值软测量方法.计算机工程与应用.2002,38(16)：55-57
[47]Rosipal R. Kernel Partial least squares regression in reproducing kernel Hilbert space. Journal of Machine Learning Research.2001,2:97-123
[48]Wold S, Kettaneh-Wold N, Skagerberg B. Nonlinear PLS modeling. Chemometrics and Intelligent Laboratory Systems.1989,7:53-65
[49]BaffiG, MartinEB, MorrisAJ. Non-linear projection to latent structures revisited:The quadrate PLS algorithm. Computers and Chemical Engineering.1999,23:395-411
[50]Durand J F. Local polynomial additive regression through PLS and splines:PLSS. Chemometrics and Intelligent Laboratory Systems.2001,58:235-246
[51]Holeomb T R, Morari M. PLS neural networks. Computers Chem.Engng..1992, 16(4):393-411
[52]Malthouse E C, Tamhane A C, Mah RSH. Nonlinear partial least squares. Computers Chem. Engng1997,21(8)：875-890
[53]班怀芸.基于模糊偏最小二乘的特征抽取方法研究.南京理工大学硕士学位论文,2009
[54]吴涛.核函数的性质、方法及其在障碍检测中的应用.国防科学技术大学博士学位论文,2003
[55]潘继斌.核函数的概念、性质及其应用.湖北师范学院学报(自然科学版).2007,27(1)：10-12
[56]罗林开,叶凌君,周绮凤.常用核函数的几何度量与几何性质.厦门大学学报(自然科学版).2009,48(6)：804-807
[57]Nello Cristianini, Jaz Kandola, Andre Elisseeff, et al. On kernel target alignment. StudFuzz.2006,194:205-256
[58]M.Seeger. Bayes method for support vector machine and Gaussian processes.1999
[59]O.Barzilay, V.L.Brailovaky. On domain knowledge and feature selection using a support vector machine. Pattern Recognition letters.1999,20:475-484
[60]V.L.Brailovaky. On global local mixed and neighborhood kernels for support vector machine. Pattern Recognition letters.1999,20:1183-1190
[61]Chen Bo, Liu Hongwei, Zheng Bao. A kernel optimization method based on the localized kernel Fisher criterion. Pattern Recognition.2008,41:1098-1109
[62]S.Amari, S.Wu. Improving support vector machine classifiers by modifying kernel functions. Neural Networks.1999,12:783-789
[63]S.Wu, S.Amari. Conformal Transformation of Kernel Functions:A Data—Dependent Way to Improve Support Vector Machine Classifiers. Neural Processing Letters.2001, 5:59-67
[64]Shen Ruimin, Fu Yonggang. Modifications of Kernels to improve support vector machine classifiers. Proceedings of the Third International Conference on Machine Learning and Cybernetics.2004:26-29
[65]Williams Peter, Li Sheng, Feng Jianfeng, et al. A Geometrical Method to Improve Performance of the Support Vector Machine. neural networks.2007,18(3):942-947
[66]李军,董海鹰.基于小波核偏最小二乘回归方法的混沌系统建模研究.物理学报.2008,57(8)：4756-4765
[67]赵玲玲,翁苏明,曾华军等译.模式分析的核方法.第1版.北京：机械工业出版社,2006
[68]李国勇.神经模糊控制理论及应用.第1版.北京：电子工业出版社,2009
[2]杨淑莹.图像模式识别.第1版.北京：清华大学出版社,北京交通大学出版社,2005
[3]边肇祺,张学工.模式识别.第2版.北京：清华大学出版社,2000
[4]吴介,裘正定.掌纹识别中的特征提取算法综述.北京电子科技学院学报.2005,13(2)：86-92
[5]刘青山,卢汉清,马颂德.综述人脸识别中的子空间方法.自动化学报.2003,29(6)：900-911
[6]Turk M., Pentland A.. Eigenfaces for recognition. J. of Cognitive Neuroscience.1991, 3(1):71-86
[7]Kibry M., Sirovich L.. Application of the Karhunen-loeve procedure for the characterization of human faces. IEEE Trans.on PAMI.1990,12(1):103-108
[8]Hotelling H. Relations between two sets of varieties. Biometrika,1936, (8):321-377
[9]Nguyen D V, Rocke D M. Tumor classification by partial least squares using microarray gene expression data. Bioinformatics.2002,18(1):39-50
[10]孙廷凯.增强型典型相关分析研究与应用.南京航空航天大学博士学位论文,2006
[11]Svante Wold. Personal memories of the early PLS development. Chemometrics and Intelligent Laboratory Systems.2001,58:83-84
[12]Svante Wold, Johan Trygg, Anders Berglund, et al. Some recent developments in PLS modeling. Chemometrics and Intelligent Laboratory Systems.2001,58:131-150
[13]王惠文.偏最小二乘回归方法及其应用.第1版.北京：国防工业出版社,1999
[14]王惠文.偏最小二乘回归的线性与非线性方法.第1版.北京：国防工业出版社,2005
[15]蒋红卫.偏最小二乘回归的扩展及其使用算法构建.第四军医大学硕士学位论文,2003
[16]Svante Wold, Henrik Antti, Fredrik Lindgren, et al. Orthogonal signal correction of near-infrared spectra. Chemometrics and Intelligent Laboratory Systems.1998,44:175-185
[17]M.H.Zhang, Q.S.Xu, D.L.Massart. Averaged and weighted averaged partial least squares. Analytica Chimica Acta,2004,504:279-289
[18]杨茂龙,孙权森,夏德深.二维共轭正交偏最小二乘分析及图像识别应用.计算机工程与应用.2008,44(29)：36-39
[19]Wold S., Kettaneh Wold N., SkagerbergB. Nonlinear PLS Modeling. Chemometrics IntLab System.1989, 11(7):53-65
[20]Wold S. Nonlinear PLS Modeling (Ⅱ) Spline Inner Function. Chemometrics IntLab System.1992,14(1):71-84
[21]Roman Rosipal, Leonard J.Trejo. Kernel Partial Least Squares Regression in Reproducing Kernel Hilbert Space. Journal of Machine Learning Research.2001,2:97-123
[22]Yoon Ho Bang, Chang Kyoo Yoo, In-Beum Lee. Nonlinear PLS modeling with fuzzy inference system. Chemometrics and Intelligent Laboratory Systems.2003,64:137-155
[23]Qin,S J, McAvoy,T J. Nonlinear PLS Modeling Using Neural Networks. Computers & Chemical Engineering.1992,16(4):379-391
[24]叶莺,陈崇帼,林熙.偏最小二乘回归的原理及应用.海峡预防医学杂志.2005,11(3)：3-6
[25]蒋红卫,夏结来,张春霞等.核偏最小二乘回归及其在医学中的应用.中国卫生统计.2007,24(3)：239-242
[26]蒲前超,夏继红,韩玉玲等.基于偏最小二乘回归法的河流病因诊断.河海大学学报(自然科学版).2009,37(1)：27-31
[27]李雪平,康辉明.PLS法在重庆巫山县邓家屋场滑坡稳定性影响因素分析中的应用.中国地质灾害与防治学报.2005,16(4)：1-4
[28]孙权森.基于相关投影分析的特征抽取与图像识别研究.南京理工大学博士学位论文,2006
[29]陈永国.偏最小二乘法在公共部门绩效多元评估中的应用.系统工程理论与实践.2009,29(1)：89-96
[30]楼安平,杨新.基于偏最小二乘法与支持向量机的人脸识别.计算机仿真.2005,12(22)：166-168
[31]Yegang Hu, Benyong Liu. Face Recognition Based on PLS and HMM. Pattern Recognition.2009,11:1-4
[32]郭金玉,苑玮琦.基于小波变换和部分最小二乘的掌纹识别.光电子.激光.2008,4(19)：554-557
[33]罗远胜.基于PLS的文本分类技术研究.江西师范大学硕士学位论文,2006
[34]岑芳明,王明文,王鹏鸣等.基于核偏最小二乘分类的垃圾邮件过滤.中文信息学报.2009,2(23)：48-53
[35]Elisaber Bostrom, Solveig Engen, Ingvar Eide. Mutagencicity testing of organic extracts of diesel exhaust particles after spiking with polycyclic aromatic hydrocarbons(PAH). Arch Toxicol,1998,72:645-649
[36]Avraham Lorber, Lawrence E.Wangen, Bruce R.Kowalski. A theoretical foundation for the PLS algorithm. Journal of Chemometrics.1987,1(1):19-31
[37]Giancarlo Diana, Chiara Tommasi. Cross-validation methods in principal component analysis:a comparison. Statistical Methods & Applications.2002,11:71-82
[38]J.A.Wegelin. A survey of Partial Least Squares(PLS) methods, with emphasis on the two-block case. Technical report, Department of Statistics, University of Washington, Seattle, 2000
[39]冉启文,谭立英.小波分析与分数傅里叶变换及应用.第1版.北京：国防工业出版社,2002
[40]Rayens W S. The art of maximizing covariance. University of Kentucky Technical Report,2000
[41]Johan Trygg, Svante Wold. Orthogonal projections to latent structures(O-PLS), Journal of chemometrics.2002,16:119-128
[42]Matthew Barker, William Rayens. Partial least squares for discrimination. Journal of Chemometrics.2003,17:166-173
[43]Momma M, BennettKp. (Eds.) Sparse kernel partial least squares regression, LNAI:COLT/Kernel.2003,2777:216-230
[44]曾三友,孙星明,夏利民等.基于Chebyshev多项式的自适应偏最小二乘回归建模.长沙铁道学院学报.2001,19(1)：95-99
[45]GeorgeR. Contributions to the problem of approximation of non-linear data with linear PLS in an absorption spectroscopic context. Chemometrics and Intelligent Laboratory Systems.1999,47:99-106
[46]李向阳,朱学峰,黄道平等.基于定性知识和线性PLS的间歇蒸煮过程Kappa值软测量方法.计算机工程与应用.2002,38(16)：55-57
[47]Rosipal R. Kernel Partial least squares regression in reproducing kernel Hilbert space. Journal of Machine Learning Research.2001,2:97-123
[48]Wold S, Kettaneh-Wold N, Skagerberg B. Nonlinear PLS modeling. Chemometrics and Intelligent Laboratory Systems.1989,7:53-65
[49]BaffiG, MartinEB, MorrisAJ. Non-linear projection to latent structures revisited:The quadrate PLS algorithm. Computers and Chemical Engineering.1999,23:395-411
[50]Durand J F. Local polynomial additive regression through PLS and splines:PLSS. Chemometrics and Intelligent Laboratory Systems.2001,58:235-246
[51]Holeomb T R, Morari M. PLS neural networks. Computers Chem.Engng..1992, 16(4):393-411
[52]Malthouse E C, Tamhane A C, Mah RSH. Nonlinear partial least squares. Computers Chem. Engng1997,21(8)：875-890
[53]班怀芸.基于模糊偏最小二乘的特征抽取方法研究.南京理工大学硕士学位论文,2009
[54]吴涛.核函数的性质、方法及其在障碍检测中的应用.国防科学技术大学博士学位论文,2003
[55]潘继斌.核函数的概念、性质及其应用.湖北师范学院学报(自然科学版).2007,27(1)：10-12
[56]罗林开,叶凌君,周绮凤.常用核函数的几何度量与几何性质.厦门大学学报(自然科学版).2009,48(6)：804-807
[57]Nello Cristianini, Jaz Kandola, Andre Elisseeff, et al. On kernel target alignment. StudFuzz.2006,194:205-256
[58]M.Seeger. Bayes method for support vector machine and Gaussian processes.1999
[59]O.Barzilay, V.L.Brailovaky. On domain knowledge and feature selection using a support vector machine. Pattern Recognition letters.1999,20:475-484
[60]V.L.Brailovaky. On global local mixed and neighborhood kernels for support vector machine. Pattern Recognition letters.1999,20:1183-1190
[61]Chen Bo, Liu Hongwei, Zheng Bao. A kernel optimization method based on the localized kernel Fisher criterion. Pattern Recognition.2008,41:1098-1109
[62]S.Amari, S.Wu. Improving support vector machine classifiers by modifying kernel functions. Neural Networks.1999,12:783-789
[63]S.Wu, S.Amari. Conformal Transformation of Kernel Functions:A Data—Dependent Way to Improve Support Vector Machine Classifiers. Neural Processing Letters.2001, 5:59-67
[64]Shen Ruimin, Fu Yonggang. Modifications of Kernels to improve support vector machine classifiers. Proceedings of the Third International Conference on Machine Learning and Cybernetics.2004:26-29
[65]Williams Peter, Li Sheng, Feng Jianfeng, et al. A Geometrical Method to Improve Performance of the Support Vector Machine. neural networks.2007,18(3):942-947
[66]李军,董海鹰.基于小波核偏最小二乘回归方法的混沌系统建模研究.物理学报.2008,57(8)：4756-4765
[67]赵玲玲,翁苏明,曾华军等译.模式分析的核方法.第1版.北京：机械工业出版社,2006
[68]李国勇.神经模糊控制理论及应用.第1版.北京：电子工业出版社,2009