稀疏流形建模及其在人脸识别中的应用
|
摘要
近年来,流形学习用于人脸识别引起了广泛关注,有研究表明,人脸很有可能是位于一个非线性的流形上,这提示我们可以将原始数据集对应的高维空间的流形映射至低维空间的流形,通过寻找人脸训练集内在的代表性变量,根据这些变量即可将不同人的图像区别开来。很多基于流形学习方法已经被提出来,本文的主要工作包括以下几个部分:
1.简要综述了当前流形学习的发展概况,介绍了几种经典的流形算法,包括等距流形映射(ISOMAP)、局部线性嵌入(LLE)以及它的变换形式:海赛局部线性嵌入(HLLE)和局部切空间排列(LTSA)等;总结概括了当前流形算法的框架:流形学习方法可以分为全局(如ISOMAP)和局部方法(如LLE,HLLE,LTSA等),从另一方面也可分为线性和非线性;分析了现在主要的流形学习方法,主要从时间复杂度、对合适的数据集的要求以及对噪声的鲁棒性等方面总结和比较了它们的不足和优点。
2.在深入研究LLE的基础上提出了稀疏局部线性嵌入算法(SLLE),同时我们提出了SLLE相应的学习高维的新样本的算法。LLE是流形学习方面的经典算法,参数少、计算快、易求全局最优解,但是LLE遇到的第一个问题便是邻域的求取,这一步关系到最后嵌入结果的质量,LLE对于噪声亦非常敏感,缺乏对新样本的学习能力,在人脸识别方面的识别率很低,SLLE便是针对上述缺点提出来的。该算法能够自适应选取样本的邻域,在野值存在的情况下有很强的鲁棒性,通过在两个标准人脸库AR和ORL上的识别实验表明SLLE在人脸识别方面远远优于LLE。
Nowadays manifold learning for face recognition has aroused extensive attention. Studies have shown that human faces are most likely located in a non-linear manifold, which prompted us to map the original data sets of high-dimensional space to the low-dimensional manifold. Through finding the representation inherent variables, according to these variables of the training set, images of different people can be distinguished. A lot of learning-based non-linear manifold dimensionality reduction methods have already been proposed. The main contributions in this paper are as follows:
1. We give a overview of the current manifold learning methods, and introduce several classical manifold algorithm, including isometric mapping Manifold (ISOMAP), local linear embedding (LLE), as well as its transformation forms: Hessian Locally Linear Embedding (HLLE) and Local Tangent Space alignment (LTSA), etc; Sum up the framework of current algorithm manifold: manifold learning methods can be divided into the global (such as ISOMAP) and local methods (such as LLE, HLLE, LTSA, etc.). On the one hand, they can also be divided into linear and nonlinear methods; Analyze the current main manifold learning methods, and summarize and compare their deficiencies and advantages mainly from the time complexity, the desire character of the data sets and robustness to noise, etc.
2. We present the sparse local linear embedding algorithm (SLLE) after investigating LLE deeply; at the same time we have put forward the corresponding algorithm of SLLE which maps the new high-dimensional samples to the manifold of training sets. LLE is one of the classical manifold learning methods and has many advantage, for example, its less parameters to set, fast computing speed, effortless of finding the global optimal solution, but the first problem LLE encountered is the search of neighborhood for data, which can be related to the embedded results. LLE is also very sensitive to noise, lacks of the learning ability for new samples, and works not very well in the face recognition. SLLE is proposed against the above-mentioned shortcomings of LLE. The algorithm can compute the neighborhood of samples adaptively and robustly. Experiments on the AR face database and the ORL face database show that the proposed SLLE method outperfoms the LLE method in terms of both semantic-relationship preservation and robust embedding against outliers.
1.简要综述了当前流形学习的发展概况,介绍了几种经典的流形算法,包括等距流形映射(ISOMAP)、局部线性嵌入(LLE)以及它的变换形式:海赛局部线性嵌入(HLLE)和局部切空间排列(LTSA)等;总结概括了当前流形算法的框架:流形学习方法可以分为全局(如ISOMAP)和局部方法(如LLE,HLLE,LTSA等),从另一方面也可分为线性和非线性;分析了现在主要的流形学习方法,主要从时间复杂度、对合适的数据集的要求以及对噪声的鲁棒性等方面总结和比较了它们的不足和优点。
2.在深入研究LLE的基础上提出了稀疏局部线性嵌入算法(SLLE),同时我们提出了SLLE相应的学习高维的新样本的算法。LLE是流形学习方面的经典算法,参数少、计算快、易求全局最优解,但是LLE遇到的第一个问题便是邻域的求取,这一步关系到最后嵌入结果的质量,LLE对于噪声亦非常敏感,缺乏对新样本的学习能力,在人脸识别方面的识别率很低,SLLE便是针对上述缺点提出来的。该算法能够自适应选取样本的邻域,在野值存在的情况下有很强的鲁棒性,通过在两个标准人脸库AR和ORL上的识别实验表明SLLE在人脸识别方面远远优于LLE。
Nowadays manifold learning for face recognition has aroused extensive attention. Studies have shown that human faces are most likely located in a non-linear manifold, which prompted us to map the original data sets of high-dimensional space to the low-dimensional manifold. Through finding the representation inherent variables, according to these variables of the training set, images of different people can be distinguished. A lot of learning-based non-linear manifold dimensionality reduction methods have already been proposed. The main contributions in this paper are as follows:
1. We give a overview of the current manifold learning methods, and introduce several classical manifold algorithm, including isometric mapping Manifold (ISOMAP), local linear embedding (LLE), as well as its transformation forms: Hessian Locally Linear Embedding (HLLE) and Local Tangent Space alignment (LTSA), etc; Sum up the framework of current algorithm manifold: manifold learning methods can be divided into the global (such as ISOMAP) and local methods (such as LLE, HLLE, LTSA, etc.). On the one hand, they can also be divided into linear and nonlinear methods; Analyze the current main manifold learning methods, and summarize and compare their deficiencies and advantages mainly from the time complexity, the desire character of the data sets and robustness to noise, etc.
2. We present the sparse local linear embedding algorithm (SLLE) after investigating LLE deeply; at the same time we have put forward the corresponding algorithm of SLLE which maps the new high-dimensional samples to the manifold of training sets. LLE is one of the classical manifold learning methods and has many advantage, for example, its less parameters to set, fast computing speed, effortless of finding the global optimal solution, but the first problem LLE encountered is the search of neighborhood for data, which can be related to the embedded results. LLE is also very sensitive to noise, lacks of the learning ability for new samples, and works not very well in the face recognition. SLLE is proposed against the above-mentioned shortcomings of LLE. The algorithm can compute the neighborhood of samples adaptively and robustly. Experiments on the AR face database and the ORL face database show that the proposed SLLE method outperfoms the LLE method in terms of both semantic-relationship preservation and robust embedding against outliers.
引文
[1] R Chellappa, C L Wilson, S Sirhey. Human and machine recognition of faces, A survey. In processing of IEEE, vol.83, pp.705~740, 1995.
[2] Shashua A, Levin, Avidan S. Manifold pursuit: anew approach to appearance based recognition.16th International Conference on Pattern Recognition.pp:590~594. 2002.
[3] Jolliffe I T. Principal component analysis. Springer-Verlag, New Yok, 1986.
[4] Martinez, Kak A. PCA versus LDA. IEEE Trans. On Pattern Analysis and Machine Intelligence, 2001, 23(2):228~233.
[5] Balakrishnama S, Ganapathirraju A. Linear discriminate analysis. Institute for Signal and Information Processing, Mississippi State University,1998.Available at http://www.isip.msstate.edu/publications/reports/isip_internal/1998/linear_dicrim_analysis/lda_theory.pdf.
[6] Yu H, Yang J. A direct LDA algorithm for high dimensional data with applications to face recognition. Pattern recognition, 2001, 34:2001, 34:2067~2070.
[7] Cox T, Cox M. Multidimensional scaling. Chapman & Hall, London, 1994.
[8] He X F, Niyogi P. Locally preserving projections. Techinical Report TR-2002-09, University of Chicago Computer Science, Oct.2002.
[9] Tenenbaum J B, Silva V de, Langford J C. A global geometric framework for nonlinear dimensionality reduction. Science, 2000, 290(5500):2319~2323.
[10]Roweis S, Lawrence K. Nonlinear dimensionality reduction by locally linear embedding. Science, 2000, 290(5500):2323~2326.
[11]Kohonen T. Self-organizing maps (EDs.2). Springer, 1995.
[12]David L. Donoho, Carrie Grimes. Hessian eigenmaps :Locally linear embedding techniques for high-dimensional data. Proceedings of the National Academy of Sciences, 2003, 105(48):5591~5596.
[13]Belkin M, Niyogi P. Laplacian eigenmaps for dimensinality reduction and data representation. Neural Computation, 2003, 15(6):1373~1396.
[14]Lawrence K, Roweis S. An introduction to locally linear embedding. Technical Report, Gatsby Computational Neuscience Unit, UCL, 2001.
[15]Brans M, MERL. Charting a manifold. Neural Information Processing Systems: Natural and Synthetic, Vancouver, Canada, 2002.
[16]Hinton G, Roweis S. Stochastic neighbor embedding. Neural Information Proceeding Systems: Natural and Synthetic, Vancouver, Canada, 2002.
[17]Z Zhang, H Zha. Principal Manifolds and Nonlinear Dimensionality Reduction via Tangent Space Alignment. SIAM J. Scientific Computing, 26(1):313~338, 2004.
[18]Kouropteva O, Okun O, Pietikainen M. Supervised locally linear embedding algorithm for pattern recognition. Pattern Recognition and Image Analysis, IbPRIA2003, LNCS2652, 2003. 384~394.
[19]Alessandro L K, Robert S, Ching Y S. Recognition and verification of unconstrained handwritten words. IEEE Trans. On Pattern Analysis and Machine Intelligence. 2005, 27(10):1509~1522.
[20]Jackson J K. A user’s Guide to principal component analysis. Wiley Series in Probability and Mathematical Statistics, John Wiley and Sons, New York, London, Sydney, 1991.
[21]Fodor I K. A survey of dimension reduction techniques. Techinical report UCRL-ID-148494, LLNL, 2002.
[22]Kambhatla N, and Leen T K. Dimension reduction by local principal component analysis. Neural Component, Oct. 1997, 9(7):1493~1516.
[23]Yeung K Y, Ruzzo W L. An empirical study on Principal Component Analysis for clustering gene expression data. Technical Report UW-CSE-2000-11-03,2000.
[24]Tipping M E, Bishop C M. Mixture of probabilistic principal component analysis. Neural Comput, 1999, 11:443~482.
[25]Xu L. Multisets modeling learning: An unified theory for supervised and unsupervised learning. Proc. IEEE Internationa, Conf. on Neural Neworks, 1994, I: 315~320.
[26]Liu Z Y, Chiu K C, Xu L. Strip line detection and thinning by RPCL-based local PCA. Pattern Recognition Letters, ISSN: 0167-8655, Oct. 2003, 24(14): 2335~2344.
[27]Jordan M I, Xu L. Convergence results for the EM approach to mixtures of experts architectures, Neural Networks, 1995, 8(9): 1409~1431.
[28]Mardia K V, Kent J T, Bibby J M. Multivariate analysis. Probability and Mathematical Statics, Academic Press, 1995.
[29]Shepard R N. Multidimensional scaling, tree fitting, and clustering. Science, 1980, 210:390~398.
[30]Faloutsos C, Lin K I. FastMap: A fast algorithm for indexing, data-mining and visualization of traditional and multimedia datasets. Proc. 1995 ACM SIGMOD International Conf. on Management of Data, California, 1995. 163~174.
[31]Balasubramanian M, Schwatz E L, Tenenbaum J B, et al. The isomap algorithm and topological stability. Science, 2002, 295:7a.
[32]Silva V D, Tenenbaum J B. Global versus local methods in nonlinear dimensionality reduction. Neural Information Processing Systems: Natural and Synthetic, Vancouver, Canada, 2002.
[33]Jenkins O dest, Mataric Maja. A spatio-temporal extension to isomap nonlinear dimension reduction. The 21st International Conf. on Machine Learning, Banff, Canada, 2004.
[34]Lawrence K, Roweis S. An Introduction to Locally Linear Embedding. Technical Report, Gatsby Computational Neuroscience Unit, UCL, 2001.
[35]Ridder D de, Kouropteva O, Okun O, et al. Supervised locally linear embedding. In Proc. ICANN/ICONIP 2003, LNCS 2714, Springer-Verlag, 2003.333~341.
[36]Ridder D de, Loog M, Reinders M J. Local fisher embedding. 17th ICPR, Cambridge, UK, 2004.
[37]A. Hdid, O. Kouropteva, M. Pietikainen. Unsupervised learning using locally linear embedding: experiments in face pose analysis. Proc. 16th International Conference on Pattern Recognition, August 11-15, Quebec City, Canada, 1:111~114.
[38]O Jenkins, M Mataric. A Spaatio-temporal Extension to Isomap Nonlinear Dimansion Reduction. Proceedings of the Twenty-First International Conference on Machine Learning (ICML-2004),July 4-8, 2004, Banff, Alberta, Canada.
[39]D Kulpinski. LLE and Isomap Analysis of Spaectra and Color Images. Master Thesis, School of Computer Science, Simon Fraser University.
[40]De Silva V, Tenenbaum. Global versus local mathods in nonlinear dimensionality reduction. Advances in Neural Information Processing System 15. 2003. 705~712.
[41]J Zhang, S Z Li, J Wang. Manifold Learning ang application in Recognition. Intelligent Multimedia Processing with Soft Computing. Springer-Verlag, Heidelberg. 2004.
[42]J Rissanen. Modleing by shortest data description. Automatica, vol.14, pp.465~471, 1978.
[43]M Hansen, B Yu. Model selection and the minimum description length principle. Journal of the American Statistical Association, vol.96, pp. 746~774, 2001.
[44]A d’Aspremont, L E Ghaoui, M Jordan, G Lanckriet. A direct formulation of sparse PCA using semidefinite programming. SIAM Review, vol. 49, pp.434~448, 2007.
[45]K Huang, S Aviyente. Sparse representation for signal classification. In Advances in Neural Information Processing Systems (NIPS), 2006.
[46]V Vapnik. The Nature of Statistical Learning Theory. Springer,2000.
[47]T Cover. Geometrical and ststistical properties of systems of linear inequalities with applications in pattern recognition. IEEE Transactions on Electronic Computers, vol. 14, no.3, pp.326~334,1965.
[48]B Olshause and D Field. Sparse coding with an overcomplete basis set: Astrategy employed by V1? Vision Research, vol. 37,pp.3311~3325, 1997.
[49]T Serre. Learning a dictionary of shape-components in visual cortex: Comparison with neurons, human and machines. Ph.D. dissertation, MIT,2006.
[50]D Donoho. For most large underdetermined systems of linear equations the minimal L1-norm solution is also the sparsest slution. Comm. On Pure and Applied Math, vol.59,no.6, pp.797~829, 2006.
[51]E Candes, J Romberg, T Tao. Stable signal recovery from incomplete and inaccurate measurements. Comm. On Pure ang Applied Math, vol.59, no.8, pp.1207~1223, 2006
[52]E Candes, T Tao. Near0optimal signal recovery from random projections: Universal encoding strategies. IEEE Trans. Information Theory, vol.52, no.12, pp.5406~5425, 2006.
[53]P Zhao, B Yu. On model selection consistency of lasso. Journal of Machine Learning Research, no.7, pp. 2541~2567,2006.
[54]E Amaldi, V Kann. On the approximability of minimizing nonzero variables or unsatisfied relations in linear systems. Theoretical Computer Science, vol.209, pp. 237~260,1998.
[55]R Tibshirani. Regression shrinkage and selection via the LASSO. Journal of the Royal Statistical Society B, vol. 58, no.1, pp.267~288,1996.
[56]E Candes. Compressive sampling. In Proceedings of the Internatinal Cogress ofMathematicians, 2006.
[57]John Wright, Allen Y. Yang, Arvind Ganesh. Robust face recognition via sparse representation. Manuscript revised for IEEE Trans. Pattern Anal. Mach. Intell., 2008.
[58]RG Baraniuk. Compressive Sensing. Signal Processing Magazine, IEEE, 2007.
[59]S Chen, D Donoho, M Saunders. Atomic decomposition by basis pursuit. SIAM Review, vol. 43, no. 1, pp. 129~159.
[60]D Donoho, Y Tsaig. Fast solution of L1-norm minimization problems when the solution may be sparse. Preprint, http://www.stanford.edu/tsaig/research.html, 2006.
[61]D Donoho. Neighborly polytopes and sparse solution of underdetermined linear equations. Preprint, 2005.
[62]Y Sharon, J Wright, Y Ma. Computation and relaxation of conditions for equivalence between L1 and L0 minimization. Submitted to IEEE Transactions on Information Theory, 2007.
[63]A Martinez. Recognizing imprecisely localized, partially occluded, and expression variant faces from a single sample per class. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 6, pp.748~763, 2002.
[64]A Leonardis and H Bischof. Robust recognition using eigenimages. Computer Vision and Image Understanding, vol.78, no. 1, pp. 99~118,2000.
[65]F Sanja, D Skocaj, A Leonardis. Combining reconstructive and discriminative subspace methods for robust classification and regression by subsampling. IEEE Transactins on Pattern Analysis and Machine Intelligence. Vol. 28, no. 3, 3006.
[66]J Kim, J Choi, J Yi, M Turk. Effective representation using ICA for face recognition robust to local distortion and partial occlusion. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2005.
[67]S Li, X Hou, H Zhang, Q Cheng. Learning spatially localized, parts-based representation. In proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2001, pp.1~6.
[68]T Ahonen, A Halid, M Pietikainen. Face description with local binary patterns: Application to face recognition. IEEE Transactions on PAMI, vol.28, no.12, pp2037~2041, 2006.
[69]M Lades, J Vorbruggen, J Buhmann, J Lange. Distortion invariant object recognitionin dynamic link architecture. IEEE Tranctions on Computers, vol.42, pp.300~311, 1993.
[70]A Pentland, B Moghaddam, T Starner. View-based and modular eigenspaces for face recognition. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 1994.
[71]Jun Wang, Zhang Changshui, Kou Zhongbao. An Analytical Mapping for LLE and Its Application in Multi-Pose Face Synthesis. 14th British Machine Vision Conf, 2003.
[72]Samaria F S. Face recognition using hidden markov models. PhD thesis, Univ. of Cambridge,1994.
[73]A Martinez, R Benavente. The AR face database. CVC Tech, Report No. 24, 1998.
[2] Shashua A, Levin, Avidan S. Manifold pursuit: anew approach to appearance based recognition.16th International Conference on Pattern Recognition.pp:590~594. 2002.
[3] Jolliffe I T. Principal component analysis. Springer-Verlag, New Yok, 1986.
[4] Martinez, Kak A. PCA versus LDA. IEEE Trans. On Pattern Analysis and Machine Intelligence, 2001, 23(2):228~233.
[5] Balakrishnama S, Ganapathirraju A. Linear discriminate analysis. Institute for Signal and Information Processing, Mississippi State University,1998.Available at http://www.isip.msstate.edu/publications/reports/isip_internal/1998/linear_dicrim_analysis/lda_theory.pdf.
[6] Yu H, Yang J. A direct LDA algorithm for high dimensional data with applications to face recognition. Pattern recognition, 2001, 34:2001, 34:2067~2070.
[7] Cox T, Cox M. Multidimensional scaling. Chapman & Hall, London, 1994.
[8] He X F, Niyogi P. Locally preserving projections. Techinical Report TR-2002-09, University of Chicago Computer Science, Oct.2002.
[9] Tenenbaum J B, Silva V de, Langford J C. A global geometric framework for nonlinear dimensionality reduction. Science, 2000, 290(5500):2319~2323.
[10]Roweis S, Lawrence K. Nonlinear dimensionality reduction by locally linear embedding. Science, 2000, 290(5500):2323~2326.
[11]Kohonen T. Self-organizing maps (EDs.2). Springer, 1995.
[12]David L. Donoho, Carrie Grimes. Hessian eigenmaps :Locally linear embedding techniques for high-dimensional data. Proceedings of the National Academy of Sciences, 2003, 105(48):5591~5596.
[13]Belkin M, Niyogi P. Laplacian eigenmaps for dimensinality reduction and data representation. Neural Computation, 2003, 15(6):1373~1396.
[14]Lawrence K, Roweis S. An introduction to locally linear embedding. Technical Report, Gatsby Computational Neuscience Unit, UCL, 2001.
[15]Brans M, MERL. Charting a manifold. Neural Information Processing Systems: Natural and Synthetic, Vancouver, Canada, 2002.
[16]Hinton G, Roweis S. Stochastic neighbor embedding. Neural Information Proceeding Systems: Natural and Synthetic, Vancouver, Canada, 2002.
[17]Z Zhang, H Zha. Principal Manifolds and Nonlinear Dimensionality Reduction via Tangent Space Alignment. SIAM J. Scientific Computing, 26(1):313~338, 2004.
[18]Kouropteva O, Okun O, Pietikainen M. Supervised locally linear embedding algorithm for pattern recognition. Pattern Recognition and Image Analysis, IbPRIA2003, LNCS2652, 2003. 384~394.
[19]Alessandro L K, Robert S, Ching Y S. Recognition and verification of unconstrained handwritten words. IEEE Trans. On Pattern Analysis and Machine Intelligence. 2005, 27(10):1509~1522.
[20]Jackson J K. A user’s Guide to principal component analysis. Wiley Series in Probability and Mathematical Statistics, John Wiley and Sons, New York, London, Sydney, 1991.
[21]Fodor I K. A survey of dimension reduction techniques. Techinical report UCRL-ID-148494, LLNL, 2002.
[22]Kambhatla N, and Leen T K. Dimension reduction by local principal component analysis. Neural Component, Oct. 1997, 9(7):1493~1516.
[23]Yeung K Y, Ruzzo W L. An empirical study on Principal Component Analysis for clustering gene expression data. Technical Report UW-CSE-2000-11-03,2000.
[24]Tipping M E, Bishop C M. Mixture of probabilistic principal component analysis. Neural Comput, 1999, 11:443~482.
[25]Xu L. Multisets modeling learning: An unified theory for supervised and unsupervised learning. Proc. IEEE Internationa, Conf. on Neural Neworks, 1994, I: 315~320.
[26]Liu Z Y, Chiu K C, Xu L. Strip line detection and thinning by RPCL-based local PCA. Pattern Recognition Letters, ISSN: 0167-8655, Oct. 2003, 24(14): 2335~2344.
[27]Jordan M I, Xu L. Convergence results for the EM approach to mixtures of experts architectures, Neural Networks, 1995, 8(9): 1409~1431.
[28]Mardia K V, Kent J T, Bibby J M. Multivariate analysis. Probability and Mathematical Statics, Academic Press, 1995.
[29]Shepard R N. Multidimensional scaling, tree fitting, and clustering. Science, 1980, 210:390~398.
[30]Faloutsos C, Lin K I. FastMap: A fast algorithm for indexing, data-mining and visualization of traditional and multimedia datasets. Proc. 1995 ACM SIGMOD International Conf. on Management of Data, California, 1995. 163~174.
[31]Balasubramanian M, Schwatz E L, Tenenbaum J B, et al. The isomap algorithm and topological stability. Science, 2002, 295:7a.
[32]Silva V D, Tenenbaum J B. Global versus local methods in nonlinear dimensionality reduction. Neural Information Processing Systems: Natural and Synthetic, Vancouver, Canada, 2002.
[33]Jenkins O dest, Mataric Maja. A spatio-temporal extension to isomap nonlinear dimension reduction. The 21st International Conf. on Machine Learning, Banff, Canada, 2004.
[34]Lawrence K, Roweis S. An Introduction to Locally Linear Embedding. Technical Report, Gatsby Computational Neuroscience Unit, UCL, 2001.
[35]Ridder D de, Kouropteva O, Okun O, et al. Supervised locally linear embedding. In Proc. ICANN/ICONIP 2003, LNCS 2714, Springer-Verlag, 2003.333~341.
[36]Ridder D de, Loog M, Reinders M J. Local fisher embedding. 17th ICPR, Cambridge, UK, 2004.
[37]A. Hdid, O. Kouropteva, M. Pietikainen. Unsupervised learning using locally linear embedding: experiments in face pose analysis. Proc. 16th International Conference on Pattern Recognition, August 11-15, Quebec City, Canada, 1:111~114.
[38]O Jenkins, M Mataric. A Spaatio-temporal Extension to Isomap Nonlinear Dimansion Reduction. Proceedings of the Twenty-First International Conference on Machine Learning (ICML-2004),July 4-8, 2004, Banff, Alberta, Canada.
[39]D Kulpinski. LLE and Isomap Analysis of Spaectra and Color Images. Master Thesis, School of Computer Science, Simon Fraser University.
[40]De Silva V, Tenenbaum. Global versus local mathods in nonlinear dimensionality reduction. Advances in Neural Information Processing System 15. 2003. 705~712.
[41]J Zhang, S Z Li, J Wang. Manifold Learning ang application in Recognition. Intelligent Multimedia Processing with Soft Computing. Springer-Verlag, Heidelberg. 2004.
[42]J Rissanen. Modleing by shortest data description. Automatica, vol.14, pp.465~471, 1978.
[43]M Hansen, B Yu. Model selection and the minimum description length principle. Journal of the American Statistical Association, vol.96, pp. 746~774, 2001.
[44]A d’Aspremont, L E Ghaoui, M Jordan, G Lanckriet. A direct formulation of sparse PCA using semidefinite programming. SIAM Review, vol. 49, pp.434~448, 2007.
[45]K Huang, S Aviyente. Sparse representation for signal classification. In Advances in Neural Information Processing Systems (NIPS), 2006.
[46]V Vapnik. The Nature of Statistical Learning Theory. Springer,2000.
[47]T Cover. Geometrical and ststistical properties of systems of linear inequalities with applications in pattern recognition. IEEE Transactions on Electronic Computers, vol. 14, no.3, pp.326~334,1965.
[48]B Olshause and D Field. Sparse coding with an overcomplete basis set: Astrategy employed by V1? Vision Research, vol. 37,pp.3311~3325, 1997.
[49]T Serre. Learning a dictionary of shape-components in visual cortex: Comparison with neurons, human and machines. Ph.D. dissertation, MIT,2006.
[50]D Donoho. For most large underdetermined systems of linear equations the minimal L1-norm solution is also the sparsest slution. Comm. On Pure and Applied Math, vol.59,no.6, pp.797~829, 2006.
[51]E Candes, J Romberg, T Tao. Stable signal recovery from incomplete and inaccurate measurements. Comm. On Pure ang Applied Math, vol.59, no.8, pp.1207~1223, 2006
[52]E Candes, T Tao. Near0optimal signal recovery from random projections: Universal encoding strategies. IEEE Trans. Information Theory, vol.52, no.12, pp.5406~5425, 2006.
[53]P Zhao, B Yu. On model selection consistency of lasso. Journal of Machine Learning Research, no.7, pp. 2541~2567,2006.
[54]E Amaldi, V Kann. On the approximability of minimizing nonzero variables or unsatisfied relations in linear systems. Theoretical Computer Science, vol.209, pp. 237~260,1998.
[55]R Tibshirani. Regression shrinkage and selection via the LASSO. Journal of the Royal Statistical Society B, vol. 58, no.1, pp.267~288,1996.
[56]E Candes. Compressive sampling. In Proceedings of the Internatinal Cogress ofMathematicians, 2006.
[57]John Wright, Allen Y. Yang, Arvind Ganesh. Robust face recognition via sparse representation. Manuscript revised for IEEE Trans. Pattern Anal. Mach. Intell., 2008.
[58]RG Baraniuk. Compressive Sensing. Signal Processing Magazine, IEEE, 2007.
[59]S Chen, D Donoho, M Saunders. Atomic decomposition by basis pursuit. SIAM Review, vol. 43, no. 1, pp. 129~159.
[60]D Donoho, Y Tsaig. Fast solution of L1-norm minimization problems when the solution may be sparse. Preprint, http://www.stanford.edu/tsaig/research.html, 2006.
[61]D Donoho. Neighborly polytopes and sparse solution of underdetermined linear equations. Preprint, 2005.
[62]Y Sharon, J Wright, Y Ma. Computation and relaxation of conditions for equivalence between L1 and L0 minimization. Submitted to IEEE Transactions on Information Theory, 2007.
[63]A Martinez. Recognizing imprecisely localized, partially occluded, and expression variant faces from a single sample per class. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 6, pp.748~763, 2002.
[64]A Leonardis and H Bischof. Robust recognition using eigenimages. Computer Vision and Image Understanding, vol.78, no. 1, pp. 99~118,2000.
[65]F Sanja, D Skocaj, A Leonardis. Combining reconstructive and discriminative subspace methods for robust classification and regression by subsampling. IEEE Transactins on Pattern Analysis and Machine Intelligence. Vol. 28, no. 3, 3006.
[66]J Kim, J Choi, J Yi, M Turk. Effective representation using ICA for face recognition robust to local distortion and partial occlusion. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2005.
[67]S Li, X Hou, H Zhang, Q Cheng. Learning spatially localized, parts-based representation. In proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2001, pp.1~6.
[68]T Ahonen, A Halid, M Pietikainen. Face description with local binary patterns: Application to face recognition. IEEE Transactions on PAMI, vol.28, no.12, pp2037~2041, 2006.
[69]M Lades, J Vorbruggen, J Buhmann, J Lange. Distortion invariant object recognitionin dynamic link architecture. IEEE Tranctions on Computers, vol.42, pp.300~311, 1993.
[70]A Pentland, B Moghaddam, T Starner. View-based and modular eigenspaces for face recognition. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 1994.
[71]Jun Wang, Zhang Changshui, Kou Zhongbao. An Analytical Mapping for LLE and Its Application in Multi-Pose Face Synthesis. 14th British Machine Vision Conf, 2003.
[72]Samaria F S. Face recognition using hidden markov models. PhD thesis, Univ. of Cambridge,1994.
[73]A Martinez, R Benavente. The AR face database. CVC Tech, Report No. 24, 1998.