基于核的降维和分类方法及其应用研究
|
摘要
特征降维和模式分类是模式识别研究的重要内容。目前，特征降维和模式分类方法受到广大学者的关注。特别是近年来核方法的快速发展，使传统方法的适用范围进一步扩大并形成了众多研究成果，广泛应用于数据挖掘、图像处理、语音识别、指纹识别、医疗诊断等领域。尽管如此，但上述方法在一定程度上仍面临鲁棒性不高、泛化能力不强等问题。针对上述问题，本课题进行了相关研究，具体研究内容如下：
1、针对线性判别分析算法面临的秩限制和小样本问题，提出几种改进算法：基于多阶矩阵组合的线性判别分析算法MLDA引入多阶矩阵组合的概念，重新定义了传统LDA中的类内离散度矩阵，使传统Fisher准则具有更好的健壮性和适应性；标量化的线性判别分析算法SLDA将类内离散度矩阵和类间离散度矩阵进行标量化处理，通过求解样本各维的权值达到特征降维的目的；基于矩阵指数的线性判别分析算法MELDA在矩阵指数的基础上，重新定义了类内离散度矩阵和类间离散度矩阵，可有效地同时提取类内离散度矩阵零空间和非零空间中的信息。此外，还从理论上对《核选择和非线性特征提取的双线性分析》一文提出的FKA算法的迭代收敛性进行了分析和探讨，并运用Radermacher复杂性分析法进行了证明。
2、当前主流特征提取方法大致有两种研究思路：（1）从高维数据的几何性质出发，根据某种寻优准则得到基于原始空间特征的一组特征数更少的新特征；（2）从降维误差角度出发，保证降维前后数据所呈现的某种偏差达到最小。本课题试图从降维过程中数据分布特征的变化入手，基于广泛使用的Parzen窗核密度估计方法，来审视和揭示Parzen窗估计与典型特征提取方法LPP、LDA和PCA之间的关系，从而说明这些特征提取方法可统一在Parzen窗框架下进行研究，为特征提取方法的研究提供了一个新的视角。
3、基于边界的分类方法中，超平面、超（椭）球等几何形状运用较为广泛。空间几何另一重要组成部分——点能否作为分类依据值得研究。受空间几何知识和光学领域光束角启发，提出基于光束角思想的最大间隔学习机BAMLM。从光学角度BAMLM可理解为在样本空间中寻找一个“光源”分别照射两类样本，根据照射区域的不同对样本进行分类；从空间几何角度BAMLM可理解为在样本空间内寻找一个分类点，通过计算样本与分类点间的夹角来判断样本类属。分析表明BAMLM的核化形式等价于核化CCMEB，通过引入核心向量机将BAMLM扩展为BACVM，有效地解决了大规模样本的分类问题。然而当训练样本中含有噪声点和孤立点时，上述方法的分类性能受到很大影响。鉴于此，提出基于空间点的最大间隔模糊分类器MFC。该方法引入模糊技术保证MFC分类时对样本区别对待，减小或消除奇异点的影响，有效提高了分类效率。
4、针对核SVM存在的信息泄露问题和大规模数据分类问题，提出面向大规模数据的隐私保护学习机PPLM和基于分类超平面的非线性集成学习机NALM。PPLM首先通过核心向量机对大规模样本进行采样，然后在核心集上选取两个样本点并将两点连线的法平面作为最优分类面。该方法有效解决大规模数据分类问题，并保证分类过程隐私安全。NALM首先将数据集分成若干数据子集，然后分别在各数据子集上运行分类超平面SH，最后将各子集上的分类结果进行集成得到最终的分类结果。该方法不仅继承了SH的优点，而且还将SH的适用范围从小规模数据扩展到中大规模数据，从线性空间推广到Hilbert核空间。
5、以SVM及其变种为代表的大间隔分类方法在实际应用中取得了较好的效果，但该方法易受到输入数据仿射或伸缩等变换的干扰，其原因在于这些方法只考虑数据类间的绝对间隔而忽视了类内数据的分布性状。针对大间隔分类方法的不足，提出基于核密度估计与熵理论的最大间隔学习机MEKLM。该方法用核密度估计表征样本的分布特征，用熵表征分类的不确定性。MEKLM可以真实反映类间数据的边界信息和类内数据的分布特征，同时解决二分类问题和单类问题，且分类性能优良。
Pattern classification and feature reduction are two of important tasks in pattern recognition and their related techniques attract more and more attentions of the researchers. With the development of kernel methods, the range of traditional pattern recognition techniques is widely broadened. Many research results are used in data mining, image processing, speech recognition, fingerprint classification and medical diagnosis. However, current feature reduction and pattern classification methods show drawbacks of low robustness and weak generalization ability to a certain extent. In order to solve the above problems, several issues are addressed in this dissertation:
Firstly, several improved Linear Discriminant Analysis (LDA) are proposed to deal with the rank limitation problem and small sample size problem in traditional LDA: Modified Linear Discriminant Analysis based on Linear Combination of K-order Matrices (MLDA) redefines within-class scatter matrix in order to make the traditional Fisher criterion get much more robust and adapt to practical applications. Scalarized Linear Discriminant Analysis (SLDA) introduces between-class scatter scalar and within-class scatter scalar and extracts features through computing the weight of each dimension in the sample space. Matrix Exponential Linear Discriminant Analysis (MELDA) also redefines the between-class scatter matrix and the within-class scatter matrix which can effectively extract the discriminative information included in the null subspace and non-null subspace of within-class scatter matrix. Besides, we provide an iterative convergence analysis of Fisher and Kernel Analysis algorithm (FKA) proposed by the paper “bilinear analysis for kernel selection and nonlinear feature extraction” using the concept of Radermacher complexity.
Secondly, researches on current feature extraction methods are mainly based on two ways. One originates from geometric properties of high-dimensional datasets and attempt to extract fewer features from the original data space according to a certain criterion. The other originates from dimension reduction deviation and try to make the deviation between data before and after dimension reduction be as small as possible. However, there exists almost no any study about them from the perspective of the scatter change of a dataset. Based on Parzen window density estimator, we thoroughly revisit the relevant feature extraction methods from a new perspective and the relationships between Parzen window and LPP, LDA and PCA are built.
Thirdly, hyperplane, hypersphere including ellipsoid are used in current boundary classification. Whether a spacial point can be used in classification is worthy to study. Inspired by space geometry and beam angle, a novel Maximum Margin Learning Machine based on Beam Angle (BAMLM) is proposed. In the view of optical, BAMLM is to find a light source to respectively irradiate two classes. In the view of space geometry, BAMLM is to find a classified point in the pattern space to separate two classes. Meanwhile, the kernel BAMLM is equivalent to the kernel Center-Constrained Minimum Enclosing Ball (CCMEB) and BAMLM can be extended to BACVM by introducing Core Vector Machine (CVM) which can work for large scale datasets. While the classification efficiency of BAMLM and BACVM are greatly influenced by the noise and isolated points, Maximum-margin Fuzzy Classifier based on Spacial Point (MFC) is proposed in which fuzzy techniques are introduced and the influences of the noise and isolated points are decreased.
Forthly, in order to solve the problems of private preserving and large scale data classification in kernel SVM, Privacy-Preserving Learning Machine for Large Scale Datasets (PPLM) and Nonlinearly Assembling Learning Machine based on Separating Hyperplane (NALM) are proposed. In PPLM, CVM is firstly introduced to sample the large scale datasets, and then two points from different classes are chosen in the core set and the hyperplane orthogonal to the line connecting these two points is treated as the optimal separating hyperplane. PPLM can work for large scale datasets and performs well. In NALM, the original datasets are firstly divided into several subsets. After running the SH algorithm on each subset, we can obtain the final classification results through assembling each result from each subset. NALM is not only privacy-preserving, but also extends the usage of SH from small scale datasets to medium and large scale datasets and from linear space to Hilbert kernel space.
In the last, the maximum-margin classification algorithms including SVM and its improved algorithms are widely used in practice, but these algorithms are greatly influenced by the affine or telescopic data. The main reason is that these algorithms only take the margin between classes into consideration while neglect the data distribution in each class. Therefore, Maximum-margin Learning Machine based on Entropy Theory and Kernel Density Estimation (MEKLM) is proposed to solve the drawback of the maximum-margin classification algorithms In MEKLM, data distributions in samples are represented by kernel density estimation and classification uncertainties are represented by entropy. MEKLM takes boundary data between classes and inner data in each class seriously, so it performs well. Meanwhile, it can work for two-class and one-class pattern classification.
1、针对线性判别分析算法面临的秩限制和小样本问题，提出几种改进算法：基于多阶矩阵组合的线性判别分析算法MLDA引入多阶矩阵组合的概念，重新定义了传统LDA中的类内离散度矩阵，使传统Fisher准则具有更好的健壮性和适应性；标量化的线性判别分析算法SLDA将类内离散度矩阵和类间离散度矩阵进行标量化处理，通过求解样本各维的权值达到特征降维的目的；基于矩阵指数的线性判别分析算法MELDA在矩阵指数的基础上，重新定义了类内离散度矩阵和类间离散度矩阵，可有效地同时提取类内离散度矩阵零空间和非零空间中的信息。此外，还从理论上对《核选择和非线性特征提取的双线性分析》一文提出的FKA算法的迭代收敛性进行了分析和探讨，并运用Radermacher复杂性分析法进行了证明。
2、当前主流特征提取方法大致有两种研究思路：（1）从高维数据的几何性质出发，根据某种寻优准则得到基于原始空间特征的一组特征数更少的新特征；（2）从降维误差角度出发，保证降维前后数据所呈现的某种偏差达到最小。本课题试图从降维过程中数据分布特征的变化入手，基于广泛使用的Parzen窗核密度估计方法，来审视和揭示Parzen窗估计与典型特征提取方法LPP、LDA和PCA之间的关系，从而说明这些特征提取方法可统一在Parzen窗框架下进行研究，为特征提取方法的研究提供了一个新的视角。
3、基于边界的分类方法中，超平面、超（椭）球等几何形状运用较为广泛。空间几何另一重要组成部分——点能否作为分类依据值得研究。受空间几何知识和光学领域光束角启发，提出基于光束角思想的最大间隔学习机BAMLM。从光学角度BAMLM可理解为在样本空间中寻找一个“光源”分别照射两类样本，根据照射区域的不同对样本进行分类；从空间几何角度BAMLM可理解为在样本空间内寻找一个分类点，通过计算样本与分类点间的夹角来判断样本类属。分析表明BAMLM的核化形式等价于核化CCMEB，通过引入核心向量机将BAMLM扩展为BACVM，有效地解决了大规模样本的分类问题。然而当训练样本中含有噪声点和孤立点时，上述方法的分类性能受到很大影响。鉴于此，提出基于空间点的最大间隔模糊分类器MFC。该方法引入模糊技术保证MFC分类时对样本区别对待，减小或消除奇异点的影响，有效提高了分类效率。
4、针对核SVM存在的信息泄露问题和大规模数据分类问题，提出面向大规模数据的隐私保护学习机PPLM和基于分类超平面的非线性集成学习机NALM。PPLM首先通过核心向量机对大规模样本进行采样，然后在核心集上选取两个样本点并将两点连线的法平面作为最优分类面。该方法有效解决大规模数据分类问题，并保证分类过程隐私安全。NALM首先将数据集分成若干数据子集，然后分别在各数据子集上运行分类超平面SH，最后将各子集上的分类结果进行集成得到最终的分类结果。该方法不仅继承了SH的优点，而且还将SH的适用范围从小规模数据扩展到中大规模数据，从线性空间推广到Hilbert核空间。
5、以SVM及其变种为代表的大间隔分类方法在实际应用中取得了较好的效果，但该方法易受到输入数据仿射或伸缩等变换的干扰，其原因在于这些方法只考虑数据类间的绝对间隔而忽视了类内数据的分布性状。针对大间隔分类方法的不足，提出基于核密度估计与熵理论的最大间隔学习机MEKLM。该方法用核密度估计表征样本的分布特征，用熵表征分类的不确定性。MEKLM可以真实反映类间数据的边界信息和类内数据的分布特征，同时解决二分类问题和单类问题，且分类性能优良。
Pattern classification and feature reduction are two of important tasks in pattern recognition and their related techniques attract more and more attentions of the researchers. With the development of kernel methods, the range of traditional pattern recognition techniques is widely broadened. Many research results are used in data mining, image processing, speech recognition, fingerprint classification and medical diagnosis. However, current feature reduction and pattern classification methods show drawbacks of low robustness and weak generalization ability to a certain extent. In order to solve the above problems, several issues are addressed in this dissertation:
Firstly, several improved Linear Discriminant Analysis (LDA) are proposed to deal with the rank limitation problem and small sample size problem in traditional LDA: Modified Linear Discriminant Analysis based on Linear Combination of K-order Matrices (MLDA) redefines within-class scatter matrix in order to make the traditional Fisher criterion get much more robust and adapt to practical applications. Scalarized Linear Discriminant Analysis (SLDA) introduces between-class scatter scalar and within-class scatter scalar and extracts features through computing the weight of each dimension in the sample space. Matrix Exponential Linear Discriminant Analysis (MELDA) also redefines the between-class scatter matrix and the within-class scatter matrix which can effectively extract the discriminative information included in the null subspace and non-null subspace of within-class scatter matrix. Besides, we provide an iterative convergence analysis of Fisher and Kernel Analysis algorithm (FKA) proposed by the paper “bilinear analysis for kernel selection and nonlinear feature extraction” using the concept of Radermacher complexity.
Secondly, researches on current feature extraction methods are mainly based on two ways. One originates from geometric properties of high-dimensional datasets and attempt to extract fewer features from the original data space according to a certain criterion. The other originates from dimension reduction deviation and try to make the deviation between data before and after dimension reduction be as small as possible. However, there exists almost no any study about them from the perspective of the scatter change of a dataset. Based on Parzen window density estimator, we thoroughly revisit the relevant feature extraction methods from a new perspective and the relationships between Parzen window and LPP, LDA and PCA are built.
Thirdly, hyperplane, hypersphere including ellipsoid are used in current boundary classification. Whether a spacial point can be used in classification is worthy to study. Inspired by space geometry and beam angle, a novel Maximum Margin Learning Machine based on Beam Angle (BAMLM) is proposed. In the view of optical, BAMLM is to find a light source to respectively irradiate two classes. In the view of space geometry, BAMLM is to find a classified point in the pattern space to separate two classes. Meanwhile, the kernel BAMLM is equivalent to the kernel Center-Constrained Minimum Enclosing Ball (CCMEB) and BAMLM can be extended to BACVM by introducing Core Vector Machine (CVM) which can work for large scale datasets. While the classification efficiency of BAMLM and BACVM are greatly influenced by the noise and isolated points, Maximum-margin Fuzzy Classifier based on Spacial Point (MFC) is proposed in which fuzzy techniques are introduced and the influences of the noise and isolated points are decreased.
Forthly, in order to solve the problems of private preserving and large scale data classification in kernel SVM, Privacy-Preserving Learning Machine for Large Scale Datasets (PPLM) and Nonlinearly Assembling Learning Machine based on Separating Hyperplane (NALM) are proposed. In PPLM, CVM is firstly introduced to sample the large scale datasets, and then two points from different classes are chosen in the core set and the hyperplane orthogonal to the line connecting these two points is treated as the optimal separating hyperplane. PPLM can work for large scale datasets and performs well. In NALM, the original datasets are firstly divided into several subsets. After running the SH algorithm on each subset, we can obtain the final classification results through assembling each result from each subset. NALM is not only privacy-preserving, but also extends the usage of SH from small scale datasets to medium and large scale datasets and from linear space to Hilbert kernel space.
In the last, the maximum-margin classification algorithms including SVM and its improved algorithms are widely used in practice, but these algorithms are greatly influenced by the affine or telescopic data. The main reason is that these algorithms only take the margin between classes into consideration while neglect the data distribution in each class. Therefore, Maximum-margin Learning Machine based on Entropy Theory and Kernel Density Estimation (MEKLM) is proposed to solve the drawback of the maximum-margin classification algorithms In MEKLM, data distributions in samples are represented by kernel density estimation and classification uncertainties are represented by entropy. MEKLM takes boundary data between classes and inner data in each class seriously, so it performs well. Meanwhile, it can work for two-class and one-class pattern classification.
引文
[1] Richard O D, Peter E H, David G S著.模式分类（第二版）[M].北京:机械工业出版社,2004
[2]边肇祺,张学工著.模式识别[M].第2版.北京:清华大学出版社,2004
[3] Guyon I, Elisseeff A. An introduction to variable and feature selection [J]. Journal ofMachine Learning Research,2003,3:1157-1182
[4]宋枫溪,高秀梅,刘树海,等.统计模式识别中的维数削减与低损降维[J].计算机学报,2005,28(11):1915-1922
[5] Sun Y J. Iterative RELIEF for feature weighting: algorithms, theories and applications [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2007,29(6):1035-1051
[6] Guoyou I, Weston J, Barnhill S, et al. Gene selection for cancer classification using supportvector machines [J]. Machine Learning,2002(46):389-422
[7] Chow T W S, Wang P Y, Ma E W M. A new feature selection scheme using a datadistribution factor for unsupervised nominal data [J]. IEEE Transaction on Systems, Man andCybernetics-Part B: Cybernetics,2008,38(2):499-509
[8] Rokach L, Chizi B, Maimon O. A methodology for improving the performance of non-rankerfeature selection filters [J]. International Journal of Pattern Recognition and ArtificialIntelligence,2007,21(5):809-830
[9] Yu L, Liu H. Feature selection for high-dimensional data: A fast correlation-based filtersolution [C]. In: Proc. of the20th International Conf. on Machine Learning (ICML),Washington DC: ICML,2003.856-863
[10] Kohavi R, John G H. Wrappers for feature subset selection [J]. Artificial Intelligence,2007,97(1-2):273-324
[11] Xu Z, King I, Lyu M R. Feature selection based on minimum error minimax probabilitymachine [J]. International Journal of Pattern Recognition,2007,21(8):1279-1292
[12] Kira K, Rendell L. The feature selection problem: Traditional methods and a new algorithm
[C]. In: Proc. of the9th Conf. on Artificial Intelligence, New Orleans: AAAI Press,1992.129-134
[13] Nakariyakui S, Casasent D P. Adaptive branch and bound algorithm for selecting optimalfeatures [J]. Pattern Recognition Letters,2007,28(12):1415-1427
[14] Dy J G, Brodley C E. Feature subset selection and order identification for unsupervisedlearning [C]. In: Proc. of the17th International Conf. on Machine Learning. San Francisco:Morgan Kaufmann Publishers,2000:88-97
[15] Whiteson S, Stone P, Stanley K O, et al. Automatic feature selection in neuroevolution [C].In: Proc. of Conf. on Genetic and Evolutionary Computation. New York: ACM,2005.1225-1232
[16] Hsing T, Liu L Y, Marcel B, et al. The coefficient of intrinsic dependence (feature selectionusing el CID)[J]. Pattern Recognition,2005,38(5):623-636
[17] Wang Q H, Zhang Y Y, Cai L, et al. Fault diagnosis for diesel value trains based onnon-negative matrix factorization and neural network ensemble [J]. Mechanical Systems andSignal Processing,2009,23(5):1683-1695
[18] Behrens T, Zhu A X, Schmidt K, et al. Multi-scale digital terrain analysis and featureselection for digital soil mapping [J]. Geoderma,2010,155(3-4):175-185
[19] Lipovetsky S. PCA and SVD with nonnegative loadings [J]. Pattern Recognition,2009,42(1):68-76
[20] Camacho J, Pic J, Ferrer A. Data understanding with PCA: structural and varianceinformation plots [J]. Chemometrics and Intelligent Laboratory Systems,2010,100(1):48-56
[21] Radulovic J, Rankovic V. Feedforward neural network and adaptive network-based fuzzyinference system in study of power lines [J]. Expert Systems with Applications,2010,37(1):165-170
[22] Peter N B, Joao P H, David J K, Eigenfaces vs. Fisherfaces: recognition Using Class SpecificLinear Projection [J]. IEEE Trans. on Pattern Analysis and Machine Intelligence,1997,19(7):711-720
[23] Lopez M M, Ramirez J, Alvarez I, et al. SVM-based CAD system for early detection of theAlzheimer’s disease using kernel PCA and LDA [J]. Neuroscience Letters,2009,464(3):233-238
[24] Mika S, Ratsch G, Weston J, et al. Constructing descriptive and discriminative nonlinearfeatures: rayleigh coefficients in kernel feature spaces [J]. IEEE Trans. on Pattern Analysisand Machine Intelligence,2003,25(3):623-628
[25] Yang M H. Kernel eigenfaces vs. kernel fisherfaces: face recognition using kernel methods
[C]. In: Proc. of the5th IEEE International Conf. on Automatic Face and GestureRecognition, Washington D C,2002.215-220
[26] Hu Y H, He S H. Integrated evaluation method [M]. Beijing: Scientific Press,2000
[27] Roweis S T, Saul L K. Nonlinear dimensionality reduction by locally linear embedding [J].Science,2000,290,2323-2326
[28] Laplacian eigenmaps and spectral techniques for embedding and clustering [C]. In: Proc. ofAdvances in Neural Information Processing Systems (NIPS). Cambridge: MIT Press,2001:585-591
[29] He X F, Niyogi P. Locality Preserving Projections [C]. In: Advances in Neural InformationProcessing Systems (NIPS), Vancouver, Canada,2003:153-160
[30]刘红岩,陈剑,陈国青.数据挖掘中的数据分类算法综述[J].清华大学学报（自然科学版）,2002,42(6):727-730
[31] Quinlan J R. Introduction of decision trees [J]. Machine Learning,1986,1(1):81-106
[32] Quinlan J R. C4.5: Programs for Machine Learning [M]. Morgan Kaufmann Publishers,1993
[33] Rastogi R, Shim K. Public: a decision tree classifier that integrates building and pruning [C].In: Proc. of the Very Large Database Conference (VLDB), New York,1998:404-415
[34] Mehta M, Agrawal R, Rissanen J. SLIQ: a fast scalable classifier for data mining [C]. In:Proc. of International Conf. Extending Database Technology(EDBT'96), France,1996:18-32
[35] Gehrke J, Ramakrishnan R, Ganti V. Rainforest: a framework for fast decision treeconstruction of large datasets [J]. Data Mining and Knowledge Discovery,2000,4(2-3):127-162
[36] Liu B, Hsu W, Ma Y. Integrating Classification and Association Rule [C]. In: Proc. of the4thInternational Conf. on Knowledge Discovery and Data Mining, New York, USA: AAAIPress,1998.80-86
[37] Li W M, Han J, Jian P. CMAR: accurate and efficient classification based on multiple classassociation rules [C]. In: Proc. of IEEE International Conf. on Data Mining,2001:369-376
[38] Yin X, Han J. Classification based on predictive association rules [J]. In: SIAM InternationalConf. on Data Mining, San Francisco,2003:331-335
[39] Vapnik V. The nature of statistical learning theory [M]. New York: Springer-Verlag,1995.
[40]邓乃扬,田英杰.支持向量机——理论、算法与拓展[M].科学出版社,2009.
[41] Pal M and Foody G M. Feature selection for classification of hyper spectral data by SVM [J].IEEE Trans. on Geoscience and Remote Sensing,2010,48(5):2297-2307
[42] Scholkopf B, Smola A, Bartlet P. New support vector algorithms [J]. Neural Computation,2000,12:1207-1245
[43] Scholkopf B, Platt J, Shawe-Taylor J, et a1. Estimating the support of high-dimensionaldistribution [J]. Neural Computation,2001,13:1443-1471
[44] Tax D M J and Duin R P W. Support vector data description [J]. Machine Learning,2004(54):45-66
[45] Tsang I W, Kwok J T and Cheung P M. Core vector machines: fast SVM training on verylarge data sets [J]. Journal of Machine Learning Research,2005,6:363-392
[46] Suykens J A, Vandewalle J. Least squares support vector machines classifiers [J]. NeuralProcessing Letters,1999,19(3):293-300
[47] Mangasarian O, Musicant D. Lagrange support vector machines [J]. Journal of MachineLearning Research,200l,1:161-177
[48] Lin K M, Lin C J. A study on reduced support vector machines [J]. IEEE Trans. on NeuralNetworks,2003,14(4):1449-1459
[49] Lee Y J, Mangasarian O. SSVM: A smooth support vector machines [J]. ComputationalOptimization and Applications,2001,20(1):5-22
[50] Kononenko I. Semi-naive Bayesian classifier [C]. In: Proc. of European Conf. on ArtificialIntelligence, Porto, Springer,1991:206-219
[51] Langley P, Sage S. Introduction of selective Bayesian classifier [C]. In: Proc. of the10thConf. on Uncertainty in Artificial Intelligence, Seattle, Morgan Kaufmann Publishers,1994:339-406
[52] Kohavi R. Scaling up the accuracy of naive-Bayes classifiers: a decision-tree hybrid [C]. In:Proc. of the2nd International Conf. on Knowledge Discovery and Data Mining. Menlo Park,AAAI Press,1996:202-207
[53] Zheng Z H, Webb G I. Lazy Bayesian rules [J]. Machine Learning,2000,32(1):53-84
[54] Friedman N, Geiger D, Goldszmidt M. Bayesian network classifiers [J]. Machine Learing,1997,29(2):131-163
[55]张丽娟,李舟军.分类方法的新研究:研究综述[J].计算机科学,2006,33(10):1-15
[56] Mercer J. Functions of positive and negative type and their connection with the theory ofintegral equations [C]. In: Philosophical Trans. of the Royal Society, London, Series A,Containing Papers of a Mathematical or Physical Character,1909,209:415-446
[57] Aizerman M A, Braverman E M, Rozonoer L I. Theoretical foundations of the potentialfunction method in pattern recognition learning [J]. Automation and Remote Control,1964,25:821-837
[58] Vapnik V N. An overview of statistical learning theory [J]. IEEE Trans. on Neural Networks,1999,10(5):988-999
[59] Scholkopf B, Smola A, Muller K R. Nonlinear component analysis as a kernel eigenvalueprobem [J]. Neural Computation,1998,10(5):1299-1319
[60] Mika S, Ratsch G, Weston J, et al. Fisher discriminant analysis with kernel [C]. In: Proc. ofthe IEEE Workshop on Neural Networks for Signal Processing, Madison,1999:41-48
[61] Robertson I, Lucy D, Baxter L, et al. A kernel-based Bayesian approach to climaticreconstruction [J]. The Holocene,1999,9(4):495
[62] Girolami M. Mercer kernel-based clustering in feature space [J]. IEEE Trans. on NeuralNetworks,2002,13(3):780-784
[63] Zhou X S, Gary A, Huang T S. Nonlinear variants of biased discriminants for interactiveimage retrieval [C]. In: IEEE Proc. of the3rd International Conf. on Image and VideoRetrieval, Dublin, Springer Verlag,2004.353-364
[64] Muller K R, Mika S, Ratsch G, et al. An Introduction to Kernel-based Learning algorithms[J]. IEEE Trans. on Neural Networks,2001,12(2):181-202
[65] Liang Z, Shi P. Kernel direct discriminant analysis and its theoretical foundation [J]. PatternRecognition,2005,38:445-447
[66] Bach F R, Joudan M I. Kernel independent component analysis [J]. Machine LearningResearch,2002,3(7):1-48
[67] Li J, Pan J, Chu S. Kernel class-wise locality preserving projection [J]. Information Sciences,2008,178(7):1825-1835
[68]严超,苏光大.人脸特征的定位与提取[J].中国图象图形学报,1998,3(5):375-379
[69]张宏林,蔡锐.Visual C++数字图像模式识别技术及工程实践[M].北京:人民邮电出版社,2003.2
[70]彭辉,张长水.基于K-L变换的人脸自动识别方法[J].清华大学学报（自然科学版）,1997,37(3):67-700
[71]杨键,杨静宇,叶晖.Fisher线性鉴别分析的理论研究及其应用[J].自动化学报,2003,29(4):482-493
[72]孙伯成,张文超.基于部件的级联线性判别分析人脸识别[J].计算机工程与应用,2006,42(16):67-69
[73]石跃祥,蔡自兴,王学武.基于改进的PCA算法和Fisher线性判别的人脸识别技术[J].小型微型计算机系统,2006,27(9):1731-1736
[74]郑字杰,於东军,杨静字.一种基于ICA和LDA组合的人脸识别新方法[J].计算机科学,2006,33(4):194-197
[75] Zhang X X, Jia Y D. A linear discriminant analysis framework based on random subspacefor face recognition [J]. Pattem Recognition,2007,40(9):2585-2591
[76] Belhumeur P N, Hespanha J P, Kriegman D J. Eiegnfaces vs. fisherfaces: recognition usingclass specific linear projection [J]. IEEE Transactions on Pattern Analysis and MachineIntelligence,1997,19(7):711-720
[77] Swets D, Weng J. Using discriminant eigenfeatures for image retrieval [J]. IEEETransactions on Pattern Analysis and Machine Intelligence,1996,18(8):831-836
[78] Hong Z Q, Yang J Y. Optimal discriminant plane for a small number of samples and designmethod of classifier on the plane [J]. Pattern Recognition,1991,24(4):317-324
[79] Chen L F, Liao H Y M, Ko M T, et al. A new LDA-based face recognition system which cansolve the small sample size problem [J]. Pattern Recognition,2000,32:317-324
[80] Yu H, Yang J. A direct LDA algorithm for high-dimensional data with application to facerecognition [J]. Pattern Recognition,2001,34(11):2067-2070
[81] Li M, Yuan B Z.2D-LDA: a statistical linear discriminant analysis for image matrix [J].Pattern Recognition Letters,2004(26):527-532
[82] Ye J. Characterization of a family of algorithm for generalized discriminant analysis onundersampled problems [J]. Journal of Machine Learning Research,2005,6:483-502
[83]宋枫溪,张大鹏,杨静宇,高秀梅.基于最大散度差鉴别准则的自适应分类算法[J].自动化学报,2006,32(4):541-549
[84]李道红.线性判别分析新方法研究及其应用[D]:[硕士论文].南京:南京航空航天大学,2005
[85]余冰,金连甫,陈平.利用标准化LDA进行人脸识别[J].计算机辅助设计与图形学,2003(3):302-306
[86] Bernstein D S and So W. Some explicit formulas for the matrix exponential [J]. IEEE Trans.on Automatic Control,38(8):1228-1232
[87] Liu Q S, Huang R. Face recognition using kernel based fisher discriminant analysis [C]. In:Proc. of the5th International Conf. on Automatic Face and Gesture Recognition, WashingtonDC, USA,2002.197-201
[88] Guo G, Li S Z, Kapluk C. Face recognition by support vector machines [C]. In: Proc. of the4th International Conf. on Automatic Face and Gesture Recognition, Grenoble, France,2000.196-201
[89] Liu W, Wang Y H. Null space based kernel fisher discriminant analysis for facerecognition[C]. In: Proc. of the6th International Conf. on Automatic Face and GestureRecognition, Seoul, Korea,2004.369-374
[90] Baudat G and Anouar F. Generalized discriminant analysis using a kernel approach [J].Neural Computation,2000,12(10):2385-2404
[91] Tristrom C. Two variations on fisher’ s linear discriminant for patter recognition [J]. IEEETrans. on Pattern Analysis and Machine Intelligence,2002,24(2):268-273
[92] Yang J, Frangi A F, Yang J Y. A new kernel fisher discriminant algorithm with applicationto face recognition [J]. Neural Computation,2004,56(4):415-421
[93] Liang Z H, Shi P F. An efficient and effective method to solve kernel fisher discriminantanalysis [J]. Neural Computation,2004,61(1):485-493
[94] Kranowski W J, Jonathan P, McCarthy W V, et al. Discriminant analysis with singularcovariance matrices: methods and application to spetroscopic data [J]. Applied Statistics,2004(44):887-894
[95] Cristianini N, Kandola J, Elisseeff A, et al. On kernel-target alighment [C]. In: Advances inNeural Information Processing Systems14,2002(14).367-373
[96] Yang H, Yan S C, Zhang C, et al. Bilinear analysis for kernel selection and nonlinear featureextraction [J]. IEEE Trans. on Neural Networks,2007,18(5):1442-1451
[97] Bartlett P L, Mendelson S. Rademacher and Gaussian complexities: risk bounds andstructural results [J]. Machine Learning Research,2002,3:463-482
[98] Shawe-Taylor J, Cristianini N. Kernel methods for pattern analysis [M]. Cambridge:Cambridge University Press,2004
[99]皋军,王士同,邓赵红.广义的势支撑特征选择方法GPSFM[J].计算机研究与发展,2009,46(1):41-51
[100]王晓明,王士同.广义的监督局部保留投影算法[J].电子与信息学报,2009,31(8):1840-1845
[101]王超,王士同.有局部保持的最大间距准则特征提取方法[J].模式识别与人工智能,2009,22(6):898-902
[102]吴葛铭,霍剑青,王晓蒲.一种基于加权Parzen窗的聚类算法[J].中国科学技术大学学报,2002,32(5):546-551
[103] Guo Q C, Chang X J, Chu H X. Mean-shift of variable window based on the epanechnikovkernel [C]. In: Proc. of IEEE International Conf. on Mechatronics and Automation,Piscataway,2007:2314-2319
[104] Scholkopf B, Platt J, Shawe-Taylor J, et a1. Estimating the support of high-dimensionaldistribution [J]. Neural Computation,2001,13:1443-1471
[105] Tax D M J, Duin R. P. W. Support vector data description [J]. Machine Learning,2004(54):45-66
[106]冯爱民,薛晖,刘学军,等.增强型单类支持向量机[J].计算机研究与发展,2008,45(11):1858-1864
[107] Lauckriet G R G, Ghaoui L E, Jordan M. Robust novelty detection with single-class MPM
[C]. Cambridge: MIT Press,2002
[108] Wei X K, Huang G B, Li Y H. Mahalanobis ellipsoidal learning machine for one classclassification [C]. In: Proc. of the6th International Conf. on Machine learning andcybernetics. Los Alamitos: IEEE Computer Society,2007:3528-3533
[109] Dolia A, Harris C, Shawe-Taylor J. Kernel ellipsoidal trimming [J]. Computational statisticsand data analysis,2007,52(1):309-324
[110] Juszczak P. Learning to recognize: A study on one-class classification and active learning
[D]. Delft: Delft University of Technology,2006
[111] Alsaadi F E, Elmirghani J M H. High-speed spot diffusing mobile optical wireless systememploying beam angle and power adaptation and imaging receivers [J]. Journal of lightwavetechnology,2010,28(16):2191-2206
[112] Lin C F, Wan S D. Fuzzy support vector machines [J]. IEEE Trans. on Neural Networks,2002,13(2):464-471
[113]孙名松,高庆国,王宣丹.基于双隶属度模糊支持向量机的邮件过滤[J].计算机工程与应用,2010,46(2):93-95
[114] Yu H, Jiang X Q, Vaidya J. Privacy-preserving SVM using nonlinear kernels on horizontallypartitioned data [C]. In: Proc. of2006ACM Symposium on Applied Computing, New York:ACM,2006.603-610
[115] Mangasarian O L, Wild E W. Privacy-preserving classification of horizontally partitioneddata via random kernels [C]. In: Proc. of2008International Conference on Data Mining, LasVegas,2008(2):473-479
[116] Yu H, Vaidya J, Jiang X Q. Privacy-preserving SVM classification on vertically partitioneddata [C]. In: Proc. of Knowledge Discovery and Data Mining,2006, Springer Verlag:647-656
[117] Mangasarian O L, Wild E W, Fung G. M. Privacy-preserving classification of verticallypartitioned data via random kernels [J]. ACM Transactions on Knowledge Discovery fromData,2008,3(2):1-16
[118] Lee Y J, Mangasarian O L. RSVM: reduced support vector machines [C]. In: Proc. of the1stSIAM International Conference on Data Mining, Chicago,2001.57-64
[119] Lin K M, Lin C J. A study on reduced support vector machines [J]. IEEE Trans. on NeuralNetwork,2003,45(2):199-204
[120] Tsang I W, Kocsor A, Kwok J T. Large-scale maximum margin discriminant analysis usingcore vector machines [J]. IEEE Trans. on Neural Networks,2008,19(4):610-624
[121] Deng Z H, Chung F L, Wang S T. FRSDE: fast reduced set density estimator using minimalenclosing ball approximation [J]. Pattern Recognition,2008,41(4):1363-1372
[122] Chung F L, Deng Z H, Wang S T. From minimum enclosing ball to fast fuzzy inferencesystem training on large datasets [J]. IEEE Trans. on Fuzzy Systems,2009,17(1):173-184
[123] Deng Z H, Choi K S, Chung F L, et al. Scalable TSK fuzzy modeling for very large datasetsusing minimal enclosing ball approximation [J]. IEEE. Trans. Fuzzy systems,2011,19(2):210-226
[124] Huang G. B, Zhu Q Y, Siew C K. Extreme learning machine: Theory and applications [J].Neurocomputing,2006,70(1-3):489-501
[125] Huang G. B, Chen L, Siew C K. Universal approximation using incremental constructivefeedforward networks with random hidden nodes [J]. IEEE Trans. on Neural Networks,2006,17(4):879-892
[126] Rong H J, Huang G. B, Sundararajan N, et al. Online sequential fuzzy extreme learningmachine for function approximation and classification problems [J]. IEEE Trans. on Systems,Man, Cybernetics-Part B: Cybernetics,2009,39(4):1067-1072
[127] Marcin O. New separating hyperplane method with application to the optimization of directmarketing campaigns [J]. Pattern Recognition Letters,2011,32:540-545
[128] Seep H and Klaus O. Support Vector Machines for Dyadic Data [J]. Neural Computation,2006,18(6):1472-1510
[129]奉国和.SVM分类核函数及参数选择比较[J].计算机工程与应用,2011,47(3):123-128
[130]吴景龙,杨淑霞,刘承水.基于遗传算法优化参数的支持向量机短期负荷预测方法[J].中南大学学报(自然科学版),2009,40(1):180-184
[131]杨旭,纪玉波,田雪.基于遗传算法的SVM参数选取[J].辽宁石油化工大学学报,2004,24(1):54-58
[132]庄严,白振林,许云峰.基于蚁群算法的支持向量机参数选择方法研究[J].计算机仿真,2011,28(5):216-219
[133]文传军,詹永照,陈长军.最大间隔最小体积球形支持向量机[J].控制与决策,2010,25(1):79-83
[134] Shivaswamy P K，Jebara T. Maximum relative margin and data-dependent regularization [J].Journal of Machine Learning Research,2010(11):747-788
[135] Odiowei P P, Cao Y. Nonlinear dynamic process monitoring using canonical variate analysisand kernel density estimations [J]. IEEE Trans. on Industrial Informatics,2010,6(1):36-45
[136] Penalver B A, Escolano R F, Saez J M. Learning Gaussian mixture models withentropy-based criteria [J]. IEEE Trans. on Neural Networks,2009,20(11):1756-1771
[137] Wand M P, Jones M C. Kernel Smoothing [M], Chapman&Hall,1995
[2]边肇祺,张学工著.模式识别[M].第2版.北京:清华大学出版社,2004
[3] Guyon I, Elisseeff A. An introduction to variable and feature selection [J]. Journal ofMachine Learning Research,2003,3:1157-1182
[4]宋枫溪,高秀梅,刘树海,等.统计模式识别中的维数削减与低损降维[J].计算机学报,2005,28(11):1915-1922
[5] Sun Y J. Iterative RELIEF for feature weighting: algorithms, theories and applications [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2007,29(6):1035-1051
[6] Guoyou I, Weston J, Barnhill S, et al. Gene selection for cancer classification using supportvector machines [J]. Machine Learning,2002(46):389-422
[7] Chow T W S, Wang P Y, Ma E W M. A new feature selection scheme using a datadistribution factor for unsupervised nominal data [J]. IEEE Transaction on Systems, Man andCybernetics-Part B: Cybernetics,2008,38(2):499-509
[8] Rokach L, Chizi B, Maimon O. A methodology for improving the performance of non-rankerfeature selection filters [J]. International Journal of Pattern Recognition and ArtificialIntelligence,2007,21(5):809-830
[9] Yu L, Liu H. Feature selection for high-dimensional data: A fast correlation-based filtersolution [C]. In: Proc. of the20th International Conf. on Machine Learning (ICML),Washington DC: ICML,2003.856-863
[10] Kohavi R, John G H. Wrappers for feature subset selection [J]. Artificial Intelligence,2007,97(1-2):273-324
[11] Xu Z, King I, Lyu M R. Feature selection based on minimum error minimax probabilitymachine [J]. International Journal of Pattern Recognition,2007,21(8):1279-1292
[12] Kira K, Rendell L. The feature selection problem: Traditional methods and a new algorithm
[C]. In: Proc. of the9th Conf. on Artificial Intelligence, New Orleans: AAAI Press,1992.129-134
[13] Nakariyakui S, Casasent D P. Adaptive branch and bound algorithm for selecting optimalfeatures [J]. Pattern Recognition Letters,2007,28(12):1415-1427
[14] Dy J G, Brodley C E. Feature subset selection and order identification for unsupervisedlearning [C]. In: Proc. of the17th International Conf. on Machine Learning. San Francisco:Morgan Kaufmann Publishers,2000:88-97
[15] Whiteson S, Stone P, Stanley K O, et al. Automatic feature selection in neuroevolution [C].In: Proc. of Conf. on Genetic and Evolutionary Computation. New York: ACM,2005.1225-1232
[16] Hsing T, Liu L Y, Marcel B, et al. The coefficient of intrinsic dependence (feature selectionusing el CID)[J]. Pattern Recognition,2005,38(5):623-636
[17] Wang Q H, Zhang Y Y, Cai L, et al. Fault diagnosis for diesel value trains based onnon-negative matrix factorization and neural network ensemble [J]. Mechanical Systems andSignal Processing,2009,23(5):1683-1695
[18] Behrens T, Zhu A X, Schmidt K, et al. Multi-scale digital terrain analysis and featureselection for digital soil mapping [J]. Geoderma,2010,155(3-4):175-185
[19] Lipovetsky S. PCA and SVD with nonnegative loadings [J]. Pattern Recognition,2009,42(1):68-76
[20] Camacho J, Pic J, Ferrer A. Data understanding with PCA: structural and varianceinformation plots [J]. Chemometrics and Intelligent Laboratory Systems,2010,100(1):48-56
[21] Radulovic J, Rankovic V. Feedforward neural network and adaptive network-based fuzzyinference system in study of power lines [J]. Expert Systems with Applications,2010,37(1):165-170
[22] Peter N B, Joao P H, David J K, Eigenfaces vs. Fisherfaces: recognition Using Class SpecificLinear Projection [J]. IEEE Trans. on Pattern Analysis and Machine Intelligence,1997,19(7):711-720
[23] Lopez M M, Ramirez J, Alvarez I, et al. SVM-based CAD system for early detection of theAlzheimer’s disease using kernel PCA and LDA [J]. Neuroscience Letters,2009,464(3):233-238
[24] Mika S, Ratsch G, Weston J, et al. Constructing descriptive and discriminative nonlinearfeatures: rayleigh coefficients in kernel feature spaces [J]. IEEE Trans. on Pattern Analysisand Machine Intelligence,2003,25(3):623-628
[25] Yang M H. Kernel eigenfaces vs. kernel fisherfaces: face recognition using kernel methods
[C]. In: Proc. of the5th IEEE International Conf. on Automatic Face and GestureRecognition, Washington D C,2002.215-220
[26] Hu Y H, He S H. Integrated evaluation method [M]. Beijing: Scientific Press,2000
[27] Roweis S T, Saul L K. Nonlinear dimensionality reduction by locally linear embedding [J].Science,2000,290,2323-2326
[28] Laplacian eigenmaps and spectral techniques for embedding and clustering [C]. In: Proc. ofAdvances in Neural Information Processing Systems (NIPS). Cambridge: MIT Press,2001:585-591
[29] He X F, Niyogi P. Locality Preserving Projections [C]. In: Advances in Neural InformationProcessing Systems (NIPS), Vancouver, Canada,2003:153-160
[30]刘红岩,陈剑,陈国青.数据挖掘中的数据分类算法综述[J].清华大学学报（自然科学版）,2002,42(6):727-730
[31] Quinlan J R. Introduction of decision trees [J]. Machine Learning,1986,1(1):81-106
[32] Quinlan J R. C4.5: Programs for Machine Learning [M]. Morgan Kaufmann Publishers,1993
[33] Rastogi R, Shim K. Public: a decision tree classifier that integrates building and pruning [C].In: Proc. of the Very Large Database Conference (VLDB), New York,1998:404-415
[34] Mehta M, Agrawal R, Rissanen J. SLIQ: a fast scalable classifier for data mining [C]. In:Proc. of International Conf. Extending Database Technology(EDBT'96), France,1996:18-32
[35] Gehrke J, Ramakrishnan R, Ganti V. Rainforest: a framework for fast decision treeconstruction of large datasets [J]. Data Mining and Knowledge Discovery,2000,4(2-3):127-162
[36] Liu B, Hsu W, Ma Y. Integrating Classification and Association Rule [C]. In: Proc. of the4thInternational Conf. on Knowledge Discovery and Data Mining, New York, USA: AAAIPress,1998.80-86
[37] Li W M, Han J, Jian P. CMAR: accurate and efficient classification based on multiple classassociation rules [C]. In: Proc. of IEEE International Conf. on Data Mining,2001:369-376
[38] Yin X, Han J. Classification based on predictive association rules [J]. In: SIAM InternationalConf. on Data Mining, San Francisco,2003:331-335
[39] Vapnik V. The nature of statistical learning theory [M]. New York: Springer-Verlag,1995.
[40]邓乃扬,田英杰.支持向量机——理论、算法与拓展[M].科学出版社,2009.
[41] Pal M and Foody G M. Feature selection for classification of hyper spectral data by SVM [J].IEEE Trans. on Geoscience and Remote Sensing,2010,48(5):2297-2307
[42] Scholkopf B, Smola A, Bartlet P. New support vector algorithms [J]. Neural Computation,2000,12:1207-1245
[43] Scholkopf B, Platt J, Shawe-Taylor J, et a1. Estimating the support of high-dimensionaldistribution [J]. Neural Computation,2001,13:1443-1471
[44] Tax D M J and Duin R P W. Support vector data description [J]. Machine Learning,2004(54):45-66
[45] Tsang I W, Kwok J T and Cheung P M. Core vector machines: fast SVM training on verylarge data sets [J]. Journal of Machine Learning Research,2005,6:363-392
[46] Suykens J A, Vandewalle J. Least squares support vector machines classifiers [J]. NeuralProcessing Letters,1999,19(3):293-300
[47] Mangasarian O, Musicant D. Lagrange support vector machines [J]. Journal of MachineLearning Research,200l,1:161-177
[48] Lin K M, Lin C J. A study on reduced support vector machines [J]. IEEE Trans. on NeuralNetworks,2003,14(4):1449-1459
[49] Lee Y J, Mangasarian O. SSVM: A smooth support vector machines [J]. ComputationalOptimization and Applications,2001,20(1):5-22
[50] Kononenko I. Semi-naive Bayesian classifier [C]. In: Proc. of European Conf. on ArtificialIntelligence, Porto, Springer,1991:206-219
[51] Langley P, Sage S. Introduction of selective Bayesian classifier [C]. In: Proc. of the10thConf. on Uncertainty in Artificial Intelligence, Seattle, Morgan Kaufmann Publishers,1994:339-406
[52] Kohavi R. Scaling up the accuracy of naive-Bayes classifiers: a decision-tree hybrid [C]. In:Proc. of the2nd International Conf. on Knowledge Discovery and Data Mining. Menlo Park,AAAI Press,1996:202-207
[53] Zheng Z H, Webb G I. Lazy Bayesian rules [J]. Machine Learning,2000,32(1):53-84
[54] Friedman N, Geiger D, Goldszmidt M. Bayesian network classifiers [J]. Machine Learing,1997,29(2):131-163
[55]张丽娟,李舟军.分类方法的新研究:研究综述[J].计算机科学,2006,33(10):1-15
[56] Mercer J. Functions of positive and negative type and their connection with the theory ofintegral equations [C]. In: Philosophical Trans. of the Royal Society, London, Series A,Containing Papers of a Mathematical or Physical Character,1909,209:415-446
[57] Aizerman M A, Braverman E M, Rozonoer L I. Theoretical foundations of the potentialfunction method in pattern recognition learning [J]. Automation and Remote Control,1964,25:821-837
[58] Vapnik V N. An overview of statistical learning theory [J]. IEEE Trans. on Neural Networks,1999,10(5):988-999
[59] Scholkopf B, Smola A, Muller K R. Nonlinear component analysis as a kernel eigenvalueprobem [J]. Neural Computation,1998,10(5):1299-1319
[60] Mika S, Ratsch G, Weston J, et al. Fisher discriminant analysis with kernel [C]. In: Proc. ofthe IEEE Workshop on Neural Networks for Signal Processing, Madison,1999:41-48
[61] Robertson I, Lucy D, Baxter L, et al. A kernel-based Bayesian approach to climaticreconstruction [J]. The Holocene,1999,9(4):495
[62] Girolami M. Mercer kernel-based clustering in feature space [J]. IEEE Trans. on NeuralNetworks,2002,13(3):780-784
[63] Zhou X S, Gary A, Huang T S. Nonlinear variants of biased discriminants for interactiveimage retrieval [C]. In: IEEE Proc. of the3rd International Conf. on Image and VideoRetrieval, Dublin, Springer Verlag,2004.353-364
[64] Muller K R, Mika S, Ratsch G, et al. An Introduction to Kernel-based Learning algorithms[J]. IEEE Trans. on Neural Networks,2001,12(2):181-202
[65] Liang Z, Shi P. Kernel direct discriminant analysis and its theoretical foundation [J]. PatternRecognition,2005,38:445-447
[66] Bach F R, Joudan M I. Kernel independent component analysis [J]. Machine LearningResearch,2002,3(7):1-48
[67] Li J, Pan J, Chu S. Kernel class-wise locality preserving projection [J]. Information Sciences,2008,178(7):1825-1835
[68]严超,苏光大.人脸特征的定位与提取[J].中国图象图形学报,1998,3(5):375-379
[69]张宏林,蔡锐.Visual C++数字图像模式识别技术及工程实践[M].北京:人民邮电出版社,2003.2
[70]彭辉,张长水.基于K-L变换的人脸自动识别方法[J].清华大学学报（自然科学版）,1997,37(3):67-700
[71]杨键,杨静宇,叶晖.Fisher线性鉴别分析的理论研究及其应用[J].自动化学报,2003,29(4):482-493
[72]孙伯成,张文超.基于部件的级联线性判别分析人脸识别[J].计算机工程与应用,2006,42(16):67-69
[73]石跃祥,蔡自兴,王学武.基于改进的PCA算法和Fisher线性判别的人脸识别技术[J].小型微型计算机系统,2006,27(9):1731-1736
[74]郑字杰,於东军,杨静字.一种基于ICA和LDA组合的人脸识别新方法[J].计算机科学,2006,33(4):194-197
[75] Zhang X X, Jia Y D. A linear discriminant analysis framework based on random subspacefor face recognition [J]. Pattem Recognition,2007,40(9):2585-2591
[76] Belhumeur P N, Hespanha J P, Kriegman D J. Eiegnfaces vs. fisherfaces: recognition usingclass specific linear projection [J]. IEEE Transactions on Pattern Analysis and MachineIntelligence,1997,19(7):711-720
[77] Swets D, Weng J. Using discriminant eigenfeatures for image retrieval [J]. IEEETransactions on Pattern Analysis and Machine Intelligence,1996,18(8):831-836
[78] Hong Z Q, Yang J Y. Optimal discriminant plane for a small number of samples and designmethod of classifier on the plane [J]. Pattern Recognition,1991,24(4):317-324
[79] Chen L F, Liao H Y M, Ko M T, et al. A new LDA-based face recognition system which cansolve the small sample size problem [J]. Pattern Recognition,2000,32:317-324
[80] Yu H, Yang J. A direct LDA algorithm for high-dimensional data with application to facerecognition [J]. Pattern Recognition,2001,34(11):2067-2070
[81] Li M, Yuan B Z.2D-LDA: a statistical linear discriminant analysis for image matrix [J].Pattern Recognition Letters,2004(26):527-532
[82] Ye J. Characterization of a family of algorithm for generalized discriminant analysis onundersampled problems [J]. Journal of Machine Learning Research,2005,6:483-502
[83]宋枫溪,张大鹏,杨静宇,高秀梅.基于最大散度差鉴别准则的自适应分类算法[J].自动化学报,2006,32(4):541-549
[84]李道红.线性判别分析新方法研究及其应用[D]:[硕士论文].南京:南京航空航天大学,2005
[85]余冰,金连甫,陈平.利用标准化LDA进行人脸识别[J].计算机辅助设计与图形学,2003(3):302-306
[86] Bernstein D S and So W. Some explicit formulas for the matrix exponential [J]. IEEE Trans.on Automatic Control,38(8):1228-1232
[87] Liu Q S, Huang R. Face recognition using kernel based fisher discriminant analysis [C]. In:Proc. of the5th International Conf. on Automatic Face and Gesture Recognition, WashingtonDC, USA,2002.197-201
[88] Guo G, Li S Z, Kapluk C. Face recognition by support vector machines [C]. In: Proc. of the4th International Conf. on Automatic Face and Gesture Recognition, Grenoble, France,2000.196-201
[89] Liu W, Wang Y H. Null space based kernel fisher discriminant analysis for facerecognition[C]. In: Proc. of the6th International Conf. on Automatic Face and GestureRecognition, Seoul, Korea,2004.369-374
[90] Baudat G and Anouar F. Generalized discriminant analysis using a kernel approach [J].Neural Computation,2000,12(10):2385-2404
[91] Tristrom C. Two variations on fisher’ s linear discriminant for patter recognition [J]. IEEETrans. on Pattern Analysis and Machine Intelligence,2002,24(2):268-273
[92] Yang J, Frangi A F, Yang J Y. A new kernel fisher discriminant algorithm with applicationto face recognition [J]. Neural Computation,2004,56(4):415-421
[93] Liang Z H, Shi P F. An efficient and effective method to solve kernel fisher discriminantanalysis [J]. Neural Computation,2004,61(1):485-493
[94] Kranowski W J, Jonathan P, McCarthy W V, et al. Discriminant analysis with singularcovariance matrices: methods and application to spetroscopic data [J]. Applied Statistics,2004(44):887-894
[95] Cristianini N, Kandola J, Elisseeff A, et al. On kernel-target alighment [C]. In: Advances inNeural Information Processing Systems14,2002(14).367-373
[96] Yang H, Yan S C, Zhang C, et al. Bilinear analysis for kernel selection and nonlinear featureextraction [J]. IEEE Trans. on Neural Networks,2007,18(5):1442-1451
[97] Bartlett P L, Mendelson S. Rademacher and Gaussian complexities: risk bounds andstructural results [J]. Machine Learning Research,2002,3:463-482
[98] Shawe-Taylor J, Cristianini N. Kernel methods for pattern analysis [M]. Cambridge:Cambridge University Press,2004
[99]皋军,王士同,邓赵红.广义的势支撑特征选择方法GPSFM[J].计算机研究与发展,2009,46(1):41-51
[100]王晓明,王士同.广义的监督局部保留投影算法[J].电子与信息学报,2009,31(8):1840-1845
[101]王超,王士同.有局部保持的最大间距准则特征提取方法[J].模式识别与人工智能,2009,22(6):898-902
[102]吴葛铭,霍剑青,王晓蒲.一种基于加权Parzen窗的聚类算法[J].中国科学技术大学学报,2002,32(5):546-551
[103] Guo Q C, Chang X J, Chu H X. Mean-shift of variable window based on the epanechnikovkernel [C]. In: Proc. of IEEE International Conf. on Mechatronics and Automation,Piscataway,2007:2314-2319
[104] Scholkopf B, Platt J, Shawe-Taylor J, et a1. Estimating the support of high-dimensionaldistribution [J]. Neural Computation,2001,13:1443-1471
[105] Tax D M J, Duin R. P. W. Support vector data description [J]. Machine Learning,2004(54):45-66
[106]冯爱民,薛晖,刘学军,等.增强型单类支持向量机[J].计算机研究与发展,2008,45(11):1858-1864
[107] Lauckriet G R G, Ghaoui L E, Jordan M. Robust novelty detection with single-class MPM
[C]. Cambridge: MIT Press,2002
[108] Wei X K, Huang G B, Li Y H. Mahalanobis ellipsoidal learning machine for one classclassification [C]. In: Proc. of the6th International Conf. on Machine learning andcybernetics. Los Alamitos: IEEE Computer Society,2007:3528-3533
[109] Dolia A, Harris C, Shawe-Taylor J. Kernel ellipsoidal trimming [J]. Computational statisticsand data analysis,2007,52(1):309-324
[110] Juszczak P. Learning to recognize: A study on one-class classification and active learning
[D]. Delft: Delft University of Technology,2006
[111] Alsaadi F E, Elmirghani J M H. High-speed spot diffusing mobile optical wireless systememploying beam angle and power adaptation and imaging receivers [J]. Journal of lightwavetechnology,2010,28(16):2191-2206
[112] Lin C F, Wan S D. Fuzzy support vector machines [J]. IEEE Trans. on Neural Networks,2002,13(2):464-471
[113]孙名松,高庆国,王宣丹.基于双隶属度模糊支持向量机的邮件过滤[J].计算机工程与应用,2010,46(2):93-95
[114] Yu H, Jiang X Q, Vaidya J. Privacy-preserving SVM using nonlinear kernels on horizontallypartitioned data [C]. In: Proc. of2006ACM Symposium on Applied Computing, New York:ACM,2006.603-610
[115] Mangasarian O L, Wild E W. Privacy-preserving classification of horizontally partitioneddata via random kernels [C]. In: Proc. of2008International Conference on Data Mining, LasVegas,2008(2):473-479
[116] Yu H, Vaidya J, Jiang X Q. Privacy-preserving SVM classification on vertically partitioneddata [C]. In: Proc. of Knowledge Discovery and Data Mining,2006, Springer Verlag:647-656
[117] Mangasarian O L, Wild E W, Fung G. M. Privacy-preserving classification of verticallypartitioned data via random kernels [J]. ACM Transactions on Knowledge Discovery fromData,2008,3(2):1-16
[118] Lee Y J, Mangasarian O L. RSVM: reduced support vector machines [C]. In: Proc. of the1stSIAM International Conference on Data Mining, Chicago,2001.57-64
[119] Lin K M, Lin C J. A study on reduced support vector machines [J]. IEEE Trans. on NeuralNetwork,2003,45(2):199-204
[120] Tsang I W, Kocsor A, Kwok J T. Large-scale maximum margin discriminant analysis usingcore vector machines [J]. IEEE Trans. on Neural Networks,2008,19(4):610-624
[121] Deng Z H, Chung F L, Wang S T. FRSDE: fast reduced set density estimator using minimalenclosing ball approximation [J]. Pattern Recognition,2008,41(4):1363-1372
[122] Chung F L, Deng Z H, Wang S T. From minimum enclosing ball to fast fuzzy inferencesystem training on large datasets [J]. IEEE Trans. on Fuzzy Systems,2009,17(1):173-184
[123] Deng Z H, Choi K S, Chung F L, et al. Scalable TSK fuzzy modeling for very large datasetsusing minimal enclosing ball approximation [J]. IEEE. Trans. Fuzzy systems,2011,19(2):210-226
[124] Huang G. B, Zhu Q Y, Siew C K. Extreme learning machine: Theory and applications [J].Neurocomputing,2006,70(1-3):489-501
[125] Huang G. B, Chen L, Siew C K. Universal approximation using incremental constructivefeedforward networks with random hidden nodes [J]. IEEE Trans. on Neural Networks,2006,17(4):879-892
[126] Rong H J, Huang G. B, Sundararajan N, et al. Online sequential fuzzy extreme learningmachine for function approximation and classification problems [J]. IEEE Trans. on Systems,Man, Cybernetics-Part B: Cybernetics,2009,39(4):1067-1072
[127] Marcin O. New separating hyperplane method with application to the optimization of directmarketing campaigns [J]. Pattern Recognition Letters,2011,32:540-545
[128] Seep H and Klaus O. Support Vector Machines for Dyadic Data [J]. Neural Computation,2006,18(6):1472-1510
[129]奉国和.SVM分类核函数及参数选择比较[J].计算机工程与应用,2011,47(3):123-128
[130]吴景龙,杨淑霞,刘承水.基于遗传算法优化参数的支持向量机短期负荷预测方法[J].中南大学学报(自然科学版),2009,40(1):180-184
[131]杨旭,纪玉波,田雪.基于遗传算法的SVM参数选取[J].辽宁石油化工大学学报,2004,24(1):54-58
[132]庄严,白振林,许云峰.基于蚁群算法的支持向量机参数选择方法研究[J].计算机仿真,2011,28(5):216-219
[133]文传军,詹永照,陈长军.最大间隔最小体积球形支持向量机[J].控制与决策,2010,25(1):79-83
[134] Shivaswamy P K，Jebara T. Maximum relative margin and data-dependent regularization [J].Journal of Machine Learning Research,2010(11):747-788
[135] Odiowei P P, Cao Y. Nonlinear dynamic process monitoring using canonical variate analysisand kernel density estimations [J]. IEEE Trans. on Industrial Informatics,2010,6(1):36-45
[136] Penalver B A, Escolano R F, Saez J M. Learning Gaussian mixture models withentropy-based criteria [J]. IEEE Trans. on Neural Networks,2009,20(11):1756-1771
[137] Wand M P, Jones M C. Kernel Smoothing [M], Chapman&Hall,1995
© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号 地址：北京市海淀区学院路29号 邮编：100083 电话：办公室：(+86 10)66554848；文献借阅、咨询服务、科技查新：66554700 |