属性自表达的低秩无监督属性选择算法
|
摘要
针对现有无监督属性约简方法只单一使用子空间学习或属性选择的方法,并且忽略数据之间的内在相关性,本文提出一种新的属性选择方法。首先提出一个属性自表达损失函数加上一个稀疏正则化(l2,1-范数)实现无监督学习与属性选择。然后嵌入子空间学习方法,并使用低秩约束和图正则化项考虑数据的全局结构和局部结构。经聚类实验验证,该算法较对比算法能取得更好的效果。
A new method of feature selection is proposed to remedy several drawbacks with the existing unsupervised feature reduction method which only utilize the single subspace approach or feature selection method and ignores the inherent correlation within data.Specifically,a novel feature selfrepresentation loss function is proposed to conduct unsupervised learning and feature selection by combining a sparse regularization(l2,1-norm).And then,subspace learning method and low rank constraint with graph regularization are embeded into the model,to take into account of the global structure and local structure of the data,respectively.Experimental results show that the presented algorithm can achieve better results than some selected comparison algorithms.
A new method of feature selection is proposed to remedy several drawbacks with the existing unsupervised feature reduction method which only utilize the single subspace approach or feature selection method and ignores the inherent correlation within data.Specifically,a novel feature selfrepresentation loss function is proposed to conduct unsupervised learning and feature selection by combining a sparse regularization(l2,1-norm).And then,subspace learning method and low rank constraint with graph regularization are embeded into the model,to take into account of the global structure and local structure of the data,respectively.Experimental results show that the presented algorithm can achieve better results than some selected comparison algorithms.
引文
[1]ZHU Xiaofeng,HUANG Zi,SHEN Hengtao,et al.Dimensionality reduction by mixed kernel canonical correlation analysis[J].Pattern Recognition,2012,45(8):3003-3016.DOI:10.1016/j.patcog.2012.02.007.
[2]ZHU Xiaofeng,ZHANG Shichao,JIN Zhi,et al.Missing value estimation for mixed-attribute data sets[J].IEEE Transactions on Knowledge and Data Engineering,2011,23(1):110-121.DOI:10.1109/TKDE.2010.99.
[3]ZHANG Shichao,LI Xuelong,ZONG Ming,et al.Learning k for KNN classification[J].ACM Transactions on Intelligent Systems and Technology,2017,8(3):43.DOI:10.1145/2990508.
[4]ZHU Xiaofeng,LI Xuelong,ZHANG Shichao.Block-row sparse multiview multilabel learning for image classification[J].IEEE Transactions on Cybernetics,2016,46(2):450-461.DOI:10.1109/TCYB.2015.2403356.
[5]ZHU Xiaofeng,LI Xuelong,ZHANG Shichao,et al.Robust joint graph sparse coding for unsupervised spectral feature selection[J].IEEE Transactions on Neural Networks and Learning Systems,2017,28(6):1263-1275.DOI:10.1109/TNNLS.2016.2521602.
[6]ZHU Xiaofeng,HUANG Zi,CHENG Hong,et al.Sparse hashing for fast multimedia search[J].ACM Transactions on Information Systems,2013,31(2):9.DOI:10.1145/2457465.2457469.
[7]ZHU Xiaofeng,HUANG Zi,YANG Yang,et al.Self-taught dimensionality reduction on the high-dimensional smallsized data[J].Pattern Recognition,2013,46(1):215-229.DOI:10.1016/j.patcog.2012.07.018.
[8]PYATYKH S,HESSER J,ZHENG Lei.Image noise level estimation by principal component analysis[J].IEEE Transactions on Image Processing,2013,22(2):687-699.DOI:10.1109/TIP.2012.2221728.
[9]FAN Zizhu,XU Yong,ZHANG D.Local linear discriminant analysis framework using sample neighbors[J].IEEE Transactions on Neural Networks,2011,22(7):1119-1132.DOI:10.1109/TNN.2011.2152852.
[10]HE Xiaofei,NIYOQI P.Locality preserving projections[M]//THRUN S,SAUL L K,SCHOLKOPF B.Advances in Neural Information Processing Systems 16.Cambridge,MA:MIT Press,2004:153-160.
[11]KONIETSCHKE F,PAULY M.Bootstrapping and permuting paired t-test type statistics[J].Statistics and Computing,2014,24(3):283-296.DOI:10.1007/s11222-012-9370-4.
[12]LIIMATAINEN K,HEIKKILAR,YLI-HARJA O,et al.Sparse logistic regression and polynomial modelling for detection of artificial drainage networks[J].Remote Sensing Letters,2015,6(4):311-320.DOI:10.1080/2150704X.2015.1031919.
[13]GU Quanquan,LI Zhenhui,HAN Jiawei.Joint feature selection and subspace learning[C]//Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence.Menlo Park,CA:AAAI Press,2011:1294-1299.DOI:10.5591/978-1-57735-516-8/IJCAI11-219.
[14]ZHU Pengfei,ZUO Wangmeng,ZHANG Lei,et al.Unsupervised feature selection by regularized self-representation[J].Pattern Recognition,2015,48(2):438-446.DOI:10.1016/j.patcog.2014.08.006.
[15]CAI Xiao,DING C,NIE Feiping,et al.On the equivalent of low-rank linear regressions and linear discriminant analysis based regressions[C]//Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.New York:ACM Press,2013:1124-1132.DOI:10.1145/2487575.2487701.
[16]MERRIS R.Laplacian matrices of graphs:a survey[J].Linear Algebra and Its Applications,1994,197/198:143-176.DOI:10.1016/0024-3795(94)90486-3.
[17]ZHU Xiaofeng,SUK H,WANG Li,et al.A novel relational regularization feature selection method for joint regression and classification in AD diagnosis[J].Medical Image Analysis,2017,38:205-214.DOI:10.1016/j.media.2015.10.008.
[18]ZHU Xiaofeng,ZHANG Shichao,ZHANG Jilian,et al.Cost-sensitive imputing missing values with ordering[C]//Proceedings of the 22nd National Conference on Artificial Intelligence:Volume 2.Menlo Park,CA:AAAI Press,2007:1922-1923.
[19]ZHU Xiaofeng,ZHANG Lei,HUANG Zi.A sparse embedding and least variance encoding approach to hashing[J].IEEE Transactions on Image Processing,2014,23(9):3737-3750.DOI:10.1109/TIP.2014.2332764.
[20]GRAHAM D B,ALLINSON N M.Characterizing virtual eigensignatures for general-purpose face recognition[M]//WECHSLER H,PHILLIPS P J,BRUCE V,et al.Face Recognition:From Theory to Applications.Berlin:Springer,1998:446-456.DOI:10.1007/978-3-642-72201-1_25.
[21]WOLD S,ESBENSEN K,GELADI P.Principal component analysis[J].Chemometrics and Intelligent Laboratory Systems,1987,2(1/2/3):37-52.DOI:10.1016/0169-7439(87)80084-9.
[22]NIE Feiping,ZHU Wei,LI Xuelong.Unsupervised feature selection with structured graph optimization[C]//Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence.Menlo Park,CA:AAAI Press,2016:1302-1308.
[2]ZHU Xiaofeng,ZHANG Shichao,JIN Zhi,et al.Missing value estimation for mixed-attribute data sets[J].IEEE Transactions on Knowledge and Data Engineering,2011,23(1):110-121.DOI:10.1109/TKDE.2010.99.
[3]ZHANG Shichao,LI Xuelong,ZONG Ming,et al.Learning k for KNN classification[J].ACM Transactions on Intelligent Systems and Technology,2017,8(3):43.DOI:10.1145/2990508.
[4]ZHU Xiaofeng,LI Xuelong,ZHANG Shichao.Block-row sparse multiview multilabel learning for image classification[J].IEEE Transactions on Cybernetics,2016,46(2):450-461.DOI:10.1109/TCYB.2015.2403356.
[5]ZHU Xiaofeng,LI Xuelong,ZHANG Shichao,et al.Robust joint graph sparse coding for unsupervised spectral feature selection[J].IEEE Transactions on Neural Networks and Learning Systems,2017,28(6):1263-1275.DOI:10.1109/TNNLS.2016.2521602.
[6]ZHU Xiaofeng,HUANG Zi,CHENG Hong,et al.Sparse hashing for fast multimedia search[J].ACM Transactions on Information Systems,2013,31(2):9.DOI:10.1145/2457465.2457469.
[7]ZHU Xiaofeng,HUANG Zi,YANG Yang,et al.Self-taught dimensionality reduction on the high-dimensional smallsized data[J].Pattern Recognition,2013,46(1):215-229.DOI:10.1016/j.patcog.2012.07.018.
[8]PYATYKH S,HESSER J,ZHENG Lei.Image noise level estimation by principal component analysis[J].IEEE Transactions on Image Processing,2013,22(2):687-699.DOI:10.1109/TIP.2012.2221728.
[9]FAN Zizhu,XU Yong,ZHANG D.Local linear discriminant analysis framework using sample neighbors[J].IEEE Transactions on Neural Networks,2011,22(7):1119-1132.DOI:10.1109/TNN.2011.2152852.
[10]HE Xiaofei,NIYOQI P.Locality preserving projections[M]//THRUN S,SAUL L K,SCHOLKOPF B.Advances in Neural Information Processing Systems 16.Cambridge,MA:MIT Press,2004:153-160.
[11]KONIETSCHKE F,PAULY M.Bootstrapping and permuting paired t-test type statistics[J].Statistics and Computing,2014,24(3):283-296.DOI:10.1007/s11222-012-9370-4.
[12]LIIMATAINEN K,HEIKKILAR,YLI-HARJA O,et al.Sparse logistic regression and polynomial modelling for detection of artificial drainage networks[J].Remote Sensing Letters,2015,6(4):311-320.DOI:10.1080/2150704X.2015.1031919.
[13]GU Quanquan,LI Zhenhui,HAN Jiawei.Joint feature selection and subspace learning[C]//Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence.Menlo Park,CA:AAAI Press,2011:1294-1299.DOI:10.5591/978-1-57735-516-8/IJCAI11-219.
[14]ZHU Pengfei,ZUO Wangmeng,ZHANG Lei,et al.Unsupervised feature selection by regularized self-representation[J].Pattern Recognition,2015,48(2):438-446.DOI:10.1016/j.patcog.2014.08.006.
[15]CAI Xiao,DING C,NIE Feiping,et al.On the equivalent of low-rank linear regressions and linear discriminant analysis based regressions[C]//Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.New York:ACM Press,2013:1124-1132.DOI:10.1145/2487575.2487701.
[16]MERRIS R.Laplacian matrices of graphs:a survey[J].Linear Algebra and Its Applications,1994,197/198:143-176.DOI:10.1016/0024-3795(94)90486-3.
[17]ZHU Xiaofeng,SUK H,WANG Li,et al.A novel relational regularization feature selection method for joint regression and classification in AD diagnosis[J].Medical Image Analysis,2017,38:205-214.DOI:10.1016/j.media.2015.10.008.
[18]ZHU Xiaofeng,ZHANG Shichao,ZHANG Jilian,et al.Cost-sensitive imputing missing values with ordering[C]//Proceedings of the 22nd National Conference on Artificial Intelligence:Volume 2.Menlo Park,CA:AAAI Press,2007:1922-1923.
[19]ZHU Xiaofeng,ZHANG Lei,HUANG Zi.A sparse embedding and least variance encoding approach to hashing[J].IEEE Transactions on Image Processing,2014,23(9):3737-3750.DOI:10.1109/TIP.2014.2332764.
[20]GRAHAM D B,ALLINSON N M.Characterizing virtual eigensignatures for general-purpose face recognition[M]//WECHSLER H,PHILLIPS P J,BRUCE V,et al.Face Recognition:From Theory to Applications.Berlin:Springer,1998:446-456.DOI:10.1007/978-3-642-72201-1_25.
[21]WOLD S,ESBENSEN K,GELADI P.Principal component analysis[J].Chemometrics and Intelligent Laboratory Systems,1987,2(1/2/3):37-52.DOI:10.1016/0169-7439(87)80084-9.
[22]NIE Feiping,ZHU Wei,LI Xuelong.Unsupervised feature selection with structured graph optimization[C]//Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence.Menlo Park,CA:AAAI Press,2016:1302-1308.