一种改进的主成分分析特征抽取算法:YJ-MICPCA
|
摘要
针对主成分分析(PCA)假设数据服从高斯分布的条件以及只能处理特征之间线性关系的不足,提出一种基于Yeo-Johnson变换和最大信息系数(MIC)的PCA特征抽取算法,命名为YJ-MICPCA。通过YeoJohnson变换改善原始数据分布,使其近似服从高斯分布,并将PCA中计算协方差矩阵转化为计算MIC矩阵的平方,使其也能处理特征间存在的非线性关系。以UCI机器学习数据库中的11个数据集为实验对象,采用支持向量机、朴素贝叶斯模型、k近邻算法这3种分类器,比较了YJ-MICPCA与PCA及其他常用非线性降维方法LLE、Isomap、MSD、KPCA的降维效果和分类精度,结果表明YJ-MICPCA总体上优于其他几种算法。
Principal component analysis(PCA)method assumes that the data obey Gaussian distribution,and it can only deal with the linear relationship between features.To address the problem,an improved PCA algorithm for feature extraction(named as YJ-MICPCA)was presented based on YeoJohnson transformation and maximal information coefficient(MIC).The original data distribution was changed into approximate Gaussian distribution by Yeo-Johnson transformation.Then,instead of covariance matrix in PCA,the square of MIC matrix was calculated so that YJ-MICPCA can also handle the non-linear relationship between features.Experiments on eleven datasets from UCI Machine Learning Repository were conducted,and three classifiers,i.e.support vector machine(SVM),naive Bayes model(NB)and k-nearest neighbor algorithm(k-NN),were used to compare the effect of dimensionality reduction and classification accuracy of YJ-MICPCA with PCA and such common nonlinear dimensionality reduction methods as LLE,Isomap,MSD and KPCA.The results show that YJMICPCA is superior to other algorithms as a whole.
Principal component analysis(PCA)method assumes that the data obey Gaussian distribution,and it can only deal with the linear relationship between features.To address the problem,an improved PCA algorithm for feature extraction(named as YJ-MICPCA)was presented based on YeoJohnson transformation and maximal information coefficient(MIC).The original data distribution was changed into approximate Gaussian distribution by Yeo-Johnson transformation.Then,instead of covariance matrix in PCA,the square of MIC matrix was calculated so that YJ-MICPCA can also handle the non-linear relationship between features.Experiments on eleven datasets from UCI Machine Learning Repository were conducted,and three classifiers,i.e.support vector machine(SVM),naive Bayes model(NB)and k-nearest neighbor algorithm(k-NN),were used to compare the effect of dimensionality reduction and classification accuracy of YJ-MICPCA with PCA and such common nonlinear dimensionality reduction methods as LLE,Isomap,MSD and KPCA.The results show that YJMICPCA is superior to other algorithms as a whole.
引文
[1] Wei H L,Billings S A.Feature subset selection and ranking for data dimensionality reduction[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2007,29(1):162-166.
[2] Jolliffe I T.Principal component analysis[M].New York:Springer,2002.
[3] 周志华.机器学习[M].北京:清华大学出版社,2016:60-63.
[4] Roweis S T,Sual L K.Nonlinear dimensionality reduction by locally linear embedding[J].Science,2000,290(5500):2323-2326.
[5] Belkin M,Niyogi P.Laplacian eigenmaps for dimensionality reduction and data representation[J].Neural Computation,2003,15(6):1373-1396.
[6] Tenenbaum J B,de Silva V,Langford J C.A global geometric framework for nonlinear dimensionality reduction[J].Science,2000,290(5500):2319-2323.
[7] Cox T F,Cox M A A.Multidimensional scaling[M].London:Chapman and Hall/CRC,2001.
[8] Hyvarinen A,Oja E.Independent component analysis:algorithms and applications[J].Neural Networks,2000,13(4-5):411-430.
[9] Ge Z Q,Song Z H.Process monitoring based on independent component analysis-principal component analysis(ICA-PCA)and similarity factors[J].Industrial and Engineering Chemistry Research,2007,46(7):2054-2063.
[10]Tang L C,Than S E.Computing process capability indices for non-normal data:a review and comparative study[J].Quality and Reliability Engineering International,1999,15(5):339-353.
[11]Liu X Q,Xie L.Statistical-based monitoring of multivariate non-Gaussian systems[J].AIChE Journal,2008,54(9):2379-2391.
[12]任智伟,吴玲达.基于信息量改进主成分分析的高光谱图像特征提取方法[J].兵器装备工程学报,2018,39(7):151-154.
[13]王中伟,宋宏,李帅,等.基于对数变换和最大信息系数PCA的过程监测[J].科学技术与工程,2017,17(16):259-265.
[14]Scholkopf B,Smola A, Müller K-R.Nonlinear component analysis as a kernel eigenvalue problem[J].Neural Computation,1998,10(5):1299-1319.
[15] Yeo I-K,Johnson R A.A new family of power transformations to improve normality or symmetry[J].Biometrika,2000,87(4):954-959.
[16]Reshef D N,Reshef Y A,Finucane H K,et al.Detecting novel associations in large data sets[J].Science,2011,334(6062):1518-1524.
[17]Reshef D N,Reshef Y A,Mitzenmacher M M,et al.Equitability analysis of the maximal information coefficient,with comparisons[J/OL].Computer Science,2013.[2018-12-25].https://arxiv.org/pdf/1301.6314.pdf.
[18]Hsu C-W,Liu C-J.A comparison of methods for multiclass support vector machines[J].IEEE Transactions on Neural Networks,2002,13(2):415-425.
[19]Domingos P,Pazzani M.On the optimality of the simple Bayesian classifier under zero-one loss[J].Machine Learing,1997,29(2-3):103-130.
[20]李航.统计学习方法[M].北京:清华大学出版社,2012:37-45.
[2] Jolliffe I T.Principal component analysis[M].New York:Springer,2002.
[3] 周志华.机器学习[M].北京:清华大学出版社,2016:60-63.
[4] Roweis S T,Sual L K.Nonlinear dimensionality reduction by locally linear embedding[J].Science,2000,290(5500):2323-2326.
[5] Belkin M,Niyogi P.Laplacian eigenmaps for dimensionality reduction and data representation[J].Neural Computation,2003,15(6):1373-1396.
[6] Tenenbaum J B,de Silva V,Langford J C.A global geometric framework for nonlinear dimensionality reduction[J].Science,2000,290(5500):2319-2323.
[7] Cox T F,Cox M A A.Multidimensional scaling[M].London:Chapman and Hall/CRC,2001.
[8] Hyvarinen A,Oja E.Independent component analysis:algorithms and applications[J].Neural Networks,2000,13(4-5):411-430.
[9] Ge Z Q,Song Z H.Process monitoring based on independent component analysis-principal component analysis(ICA-PCA)and similarity factors[J].Industrial and Engineering Chemistry Research,2007,46(7):2054-2063.
[10]Tang L C,Than S E.Computing process capability indices for non-normal data:a review and comparative study[J].Quality and Reliability Engineering International,1999,15(5):339-353.
[11]Liu X Q,Xie L.Statistical-based monitoring of multivariate non-Gaussian systems[J].AIChE Journal,2008,54(9):2379-2391.
[12]任智伟,吴玲达.基于信息量改进主成分分析的高光谱图像特征提取方法[J].兵器装备工程学报,2018,39(7):151-154.
[13]王中伟,宋宏,李帅,等.基于对数变换和最大信息系数PCA的过程监测[J].科学技术与工程,2017,17(16):259-265.
[14]Scholkopf B,Smola A, Müller K-R.Nonlinear component analysis as a kernel eigenvalue problem[J].Neural Computation,1998,10(5):1299-1319.
[15] Yeo I-K,Johnson R A.A new family of power transformations to improve normality or symmetry[J].Biometrika,2000,87(4):954-959.
[16]Reshef D N,Reshef Y A,Finucane H K,et al.Detecting novel associations in large data sets[J].Science,2011,334(6062):1518-1524.
[17]Reshef D N,Reshef Y A,Mitzenmacher M M,et al.Equitability analysis of the maximal information coefficient,with comparisons[J/OL].Computer Science,2013.[2018-12-25].https://arxiv.org/pdf/1301.6314.pdf.
[18]Hsu C-W,Liu C-J.A comparison of methods for multiclass support vector machines[J].IEEE Transactions on Neural Networks,2002,13(2):415-425.
[19]Domingos P,Pazzani M.On the optimality of the simple Bayesian classifier under zero-one loss[J].Machine Learing,1997,29(2-3):103-130.
[20]李航.统计学习方法[M].北京:清华大学出版社,2012:37-45.