分类稀疏低秩表示的子空间聚类方法
|
摘要
近年来低秩表示和稀疏表示用于子空间聚类的研究得到了广泛关注,文献中已有许多相关的子空间聚类方法.文章结合弹性网正则化低秩表示和分类稀疏表示,提出一种分类稀疏低秩表示的子空间聚类方法.方法旨在更充分地捕获数据集的局部线性结构和全局结构信息,提高聚类性能.首先采用并行分裂的自适应惩罚的线性交替方向法求解模型,然后利用求得的系数矩阵构造相似度矩阵,最后应用谱聚类方法进行聚类.另外,取代现有方法手动调节正则化参数,文章采用自适应调节正则化参数确定目标函数中各项的权重.在人工数据集、Extended Yale B数据库和CMU PIE数据库上的实验结果表明,文章方法有更明显的聚类效果和更高的准确率.
Low-rank representation and sparse representation for subspace clustering have attracted much attention, and many related subspace clustering methods have been developed in recent years. In this paper, combining elastic net regularized low-rank representation and class-wise sparse representation, we propose a new subspace clustering method. The method aims to improve the clustering performance by fully capturing the local linear structure and global information of data sets. First, we solve the model by the linearized alternating direction method with parallel splitting and adaptive penalty, and then use the obtained coefficient matrix to build the affinity matrix. After that, the spectral clustering method is employed to cluster. In addition, we adopt an adaptive rule for the estimation of the regularization parameters instead of manually adjusting them in existing methods. Experimental results on the artificial data set, Extended Yale B database and CMU PIE database demonstrate the presented model brings more obvious clustering effect and higher accuracy than some existing models.
Low-rank representation and sparse representation for subspace clustering have attracted much attention, and many related subspace clustering methods have been developed in recent years. In this paper, combining elastic net regularized low-rank representation and class-wise sparse representation, we propose a new subspace clustering method. The method aims to improve the clustering performance by fully capturing the local linear structure and global information of data sets. First, we solve the model by the linearized alternating direction method with parallel splitting and adaptive penalty, and then use the obtained coefficient matrix to build the affinity matrix. After that, the spectral clustering method is employed to cluster. In addition, we adopt an adaptive rule for the estimation of the regularization parameters instead of manually adjusting them in existing methods. Experimental results on the artificial data set, Extended Yale B database and CMU PIE database demonstrate the presented model brings more obvious clustering effect and higher accuracy than some existing models.
引文
[1] Bradleg P S, Mangasarian O L. K-plane clustering. Journal of Global Optimization, 2000, 16(1):23-32.
[2] Gear C W. Multibody grouping from motion images. International Journal of Computer Vision,1998, 29(2):133-150.
[3] Yang A Y, Rao S R, Ma Y. Robust statistical estimation and segmentation of multiple subspaces.Conference on Computer Vision and Pattern Recognition Workshop, 2006, 99.
[4] Elhamifar E, Vidal R. Sparse subspace clustering. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2009, 2790-2797.
[5] Liu G C, Lin Z C, Yu Y. Robust subspace segmentation by low-rank representation. International Conference on Machine Learning, 2010, 663-670.
[6] Luo D J, Nie F P, Ding C, et al. Multi-subspace representation and discovery. Joint European Conference on Machine Learning and Knowledge Discovery in Databases, 2011, 405-420.
[7] Zhuang L S, Gao H Y, Lin Z C,et al. Non-negative low rank and sparse graph for semi-supervised learning. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2012,2328-2335.
[8] Wright J, Yang A Y, Ganesh A, et al. Robust face recognition via sparse representation. IEEE Transactions on Pattern and Machine Intelligence, 2009, 31(2):210-227.
[9]刘芳,武娇,杨淑媛,等.结构化压缩感知研究进展.自动化学报,2013, 39(12):1980-1995.(Liu F,Wu J,Yang S Y,et al. Research advances on structured compressive sensing. Acta Automatica Sinica, 2013, 39(12):1980-1995.)
[10] Li T, Wang W W, Feng X C, et al. Image denoising using low-rank dictionary and sparse representation. International Conference on Computational Intelligence and Security, 2014, 228-232.
[11]马坚伟,徐杰,鲍跃全,等.压缩感知及其应用:从稀疏约束到低秩约束优化.信号处理,2012, 28(5):609-623.(Ma J W, Xu J, Bao Y Q, et al. Compressive sensing and its application:From sparse to low-rank regularized optimization. Journal of Signal Processing, 2012, 28(5):609-623.)
[12]彭义刚,索津莉,戴琼海,等.从压缩传感到低秩矩阵恢复:理论与应用.自动化学报,2013, 39(7):981-994.(Peng Y G, Suo J L, Dai Q H, et al. From compressed sensing to low-rank matrix recovery:Theory and applications. Acta Automatica Sinica, 2013, 39(7):981-994.)
[13] Zhang X, Sun F C, Liu G C, et al. Fast low-rank subspace segmentation. IEEE Transactions on Knowledge and Data Engineering, 2014, 26(5):1293-1297.
[14] Talwalkar A, Mackey L, Mu Y D, et al. Distributed low-rank subspace segmentation. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2013, 3543-3550.
[15] Chen J H, Yang J. Robust subspace segmentation via low-rank representation. IEEE Transactions on Cybernetics, 2014, 44(8):1432-1445.
[16] Liu G C, Yan S C. Latent low-rank representation for subspace segmentation and feature extraction. IEEE International Conference on Computer Vision, 2011, 1615-1622.
[17] Yin M, Gao J B, Lin Z C. Laplacian regularized low-rank representation and its applications.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(3):504-517.
[18] Zhang Z Y, Zhao K K. Low-rank matrix approximation with manifold regularization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(7):1717-1729.
[19]张涛,唐振民.一种基于低秩表示的子空间聚类改进算法.电子与信息学报,2016, 38(11):2811-2818.(Zhang T, Tang Z M. Improved algorithm based on low rank representation for subspace clustering. Journal of Electronics and Information Technology, 2016, 38(11):2811-2818.)
[20] Lai J, Jiang Y. Classwise sparse and collaborative patch representation for face recognition. IEEE Transactions on Image Processing, 2016, 25(7):3261-3272.
[21] Lin Z C, Liu R S, Su Z X. Linearized alternating direction method with adaptive penalty for low-rank representation. Advances in Neural Information Processing Systems, 2011, 612-620.
[22] Lin Z C, Liu R S, Li H. Linearized alternating direction method with parallel splitting and adaptive penalty for separable convex programs in machine learning. Machine Learning, 2015,99(2):287-325.
[23] Montefusco L B, Lazzaro D. An iterative l_1-based image restoration algorithm with an adaptive parameter estimation. IEEE Transactions on Image Processing, 2012, 21(4):1676-1686.
[24] Hu H, Lin Z C, Feng J J, et al. Smooth representation clustering. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2014, 3834-3841.
[25] Liu G C, Lin Z C, Yan S C, et al. Robust recovery of subspace structures by low-rank representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(1):171-184.
[26] Pham D S, Budhaditya S, Phung D, et al. Improved subspace clustering via exploitation of spatial constraints. IEEE Computer Society Conference on Computer Vision and Pattern Recognition,2012, 550-557.
[27] Lu C Y, Tang J H, Lin M, et al. Correntropy induced l_2 graph for robust subspace clustering.IEEE International Conference on Computer Vision, 2013, 1801-1808.
[28] Zhang Y Y, Sun Z N, He R, et al. Robust subspace clustering via half-quadratic minimization.IEEE International Conference on Computer Vision, 2013, 3096-3103.
[29] Lu C Y, Min H, Zhao Z Q, et al. Robust and efficient subspace segmentation via least squares regression. European Conference on Computer Vision, 2012, 347-360.
[30] Cai J F, Candés E J, Shen Z W. A singular value thresholding algorithm for matrix completion.SIAM Journal on Optimization, 2010, 20(4):1956-1982.
[31] Lin Z C, Ganesh A, Wright J, et al. Fast convex optimization algorithms for exact recovery of a corrupted low-rank matrix. Journal of the Marine Biological Association of the UK, 2009, 56(3):707-722.
[32] Lin Z C, Chen M M, Ma Y. The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices. arXiv:1009.5055, 2010.
[33] Yin M, Gao J B, Lin Z C, et al. Dual graph regularized latent low-rank representation for subspace clustering. IEEE Transactions on Image Processing, 2015, 24(12):4918-4933.
[34] Liu J M, Chen Y J, Zhang J S. Enhancing low-rank subspace clustering by manifold regularization. IEEE Transactions on Image Processing, 2014, 23(9):4022-4030.
[35]王卫卫,李小平,冯象初,等.稀疏子空间聚类综述.自动化学报,2015, 41(8):1374-1384.(Wang W W, Li X P, Feng X C, et al. A survey on sparse subspace clustering. Acta Automatica Sinica, 2015, 41(8):1374-1384.)
[36]张涛,唐振民.一种基于非负低秩稀疏图的半监督学习改进方法.电子与信息学报,2017, 39(4):915-921.(Zhang T, Tang Z M. Improved algorithm based on non-negative low rank and sparse graph for semi-supervised learning. Journal of Electronics and Information Technology, 2017, 39(4):915-921.)
[2] Gear C W. Multibody grouping from motion images. International Journal of Computer Vision,1998, 29(2):133-150.
[3] Yang A Y, Rao S R, Ma Y. Robust statistical estimation and segmentation of multiple subspaces.Conference on Computer Vision and Pattern Recognition Workshop, 2006, 99.
[4] Elhamifar E, Vidal R. Sparse subspace clustering. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2009, 2790-2797.
[5] Liu G C, Lin Z C, Yu Y. Robust subspace segmentation by low-rank representation. International Conference on Machine Learning, 2010, 663-670.
[6] Luo D J, Nie F P, Ding C, et al. Multi-subspace representation and discovery. Joint European Conference on Machine Learning and Knowledge Discovery in Databases, 2011, 405-420.
[7] Zhuang L S, Gao H Y, Lin Z C,et al. Non-negative low rank and sparse graph for semi-supervised learning. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2012,2328-2335.
[8] Wright J, Yang A Y, Ganesh A, et al. Robust face recognition via sparse representation. IEEE Transactions on Pattern and Machine Intelligence, 2009, 31(2):210-227.
[9]刘芳,武娇,杨淑媛,等.结构化压缩感知研究进展.自动化学报,2013, 39(12):1980-1995.(Liu F,Wu J,Yang S Y,et al. Research advances on structured compressive sensing. Acta Automatica Sinica, 2013, 39(12):1980-1995.)
[10] Li T, Wang W W, Feng X C, et al. Image denoising using low-rank dictionary and sparse representation. International Conference on Computational Intelligence and Security, 2014, 228-232.
[11]马坚伟,徐杰,鲍跃全,等.压缩感知及其应用:从稀疏约束到低秩约束优化.信号处理,2012, 28(5):609-623.(Ma J W, Xu J, Bao Y Q, et al. Compressive sensing and its application:From sparse to low-rank regularized optimization. Journal of Signal Processing, 2012, 28(5):609-623.)
[12]彭义刚,索津莉,戴琼海,等.从压缩传感到低秩矩阵恢复:理论与应用.自动化学报,2013, 39(7):981-994.(Peng Y G, Suo J L, Dai Q H, et al. From compressed sensing to low-rank matrix recovery:Theory and applications. Acta Automatica Sinica, 2013, 39(7):981-994.)
[13] Zhang X, Sun F C, Liu G C, et al. Fast low-rank subspace segmentation. IEEE Transactions on Knowledge and Data Engineering, 2014, 26(5):1293-1297.
[14] Talwalkar A, Mackey L, Mu Y D, et al. Distributed low-rank subspace segmentation. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2013, 3543-3550.
[15] Chen J H, Yang J. Robust subspace segmentation via low-rank representation. IEEE Transactions on Cybernetics, 2014, 44(8):1432-1445.
[16] Liu G C, Yan S C. Latent low-rank representation for subspace segmentation and feature extraction. IEEE International Conference on Computer Vision, 2011, 1615-1622.
[17] Yin M, Gao J B, Lin Z C. Laplacian regularized low-rank representation and its applications.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(3):504-517.
[18] Zhang Z Y, Zhao K K. Low-rank matrix approximation with manifold regularization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(7):1717-1729.
[19]张涛,唐振民.一种基于低秩表示的子空间聚类改进算法.电子与信息学报,2016, 38(11):2811-2818.(Zhang T, Tang Z M. Improved algorithm based on low rank representation for subspace clustering. Journal of Electronics and Information Technology, 2016, 38(11):2811-2818.)
[20] Lai J, Jiang Y. Classwise sparse and collaborative patch representation for face recognition. IEEE Transactions on Image Processing, 2016, 25(7):3261-3272.
[21] Lin Z C, Liu R S, Su Z X. Linearized alternating direction method with adaptive penalty for low-rank representation. Advances in Neural Information Processing Systems, 2011, 612-620.
[22] Lin Z C, Liu R S, Li H. Linearized alternating direction method with parallel splitting and adaptive penalty for separable convex programs in machine learning. Machine Learning, 2015,99(2):287-325.
[23] Montefusco L B, Lazzaro D. An iterative l_1-based image restoration algorithm with an adaptive parameter estimation. IEEE Transactions on Image Processing, 2012, 21(4):1676-1686.
[24] Hu H, Lin Z C, Feng J J, et al. Smooth representation clustering. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2014, 3834-3841.
[25] Liu G C, Lin Z C, Yan S C, et al. Robust recovery of subspace structures by low-rank representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(1):171-184.
[26] Pham D S, Budhaditya S, Phung D, et al. Improved subspace clustering via exploitation of spatial constraints. IEEE Computer Society Conference on Computer Vision and Pattern Recognition,2012, 550-557.
[27] Lu C Y, Tang J H, Lin M, et al. Correntropy induced l_2 graph for robust subspace clustering.IEEE International Conference on Computer Vision, 2013, 1801-1808.
[28] Zhang Y Y, Sun Z N, He R, et al. Robust subspace clustering via half-quadratic minimization.IEEE International Conference on Computer Vision, 2013, 3096-3103.
[29] Lu C Y, Min H, Zhao Z Q, et al. Robust and efficient subspace segmentation via least squares regression. European Conference on Computer Vision, 2012, 347-360.
[30] Cai J F, Candés E J, Shen Z W. A singular value thresholding algorithm for matrix completion.SIAM Journal on Optimization, 2010, 20(4):1956-1982.
[31] Lin Z C, Ganesh A, Wright J, et al. Fast convex optimization algorithms for exact recovery of a corrupted low-rank matrix. Journal of the Marine Biological Association of the UK, 2009, 56(3):707-722.
[32] Lin Z C, Chen M M, Ma Y. The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices. arXiv:1009.5055, 2010.
[33] Yin M, Gao J B, Lin Z C, et al. Dual graph regularized latent low-rank representation for subspace clustering. IEEE Transactions on Image Processing, 2015, 24(12):4918-4933.
[34] Liu J M, Chen Y J, Zhang J S. Enhancing low-rank subspace clustering by manifold regularization. IEEE Transactions on Image Processing, 2014, 23(9):4022-4030.
[35]王卫卫,李小平,冯象初,等.稀疏子空间聚类综述.自动化学报,2015, 41(8):1374-1384.(Wang W W, Li X P, Feng X C, et al. A survey on sparse subspace clustering. Acta Automatica Sinica, 2015, 41(8):1374-1384.)
[36]张涛,唐振民.一种基于非负低秩稀疏图的半监督学习改进方法.电子与信息学报,2017, 39(4):915-921.(Zhang T, Tang Z M. Improved algorithm based on non-negative low rank and sparse graph for semi-supervised learning. Journal of Electronics and Information Technology, 2017, 39(4):915-921.)