流形学习算法及若干应用研究
|
摘要
信息化技术的快速发展及其广泛应用,使具有高维数的非结构化数据信息大量出现。高维使得这些数据的内在规律不仅超出人们的直接感知能力,而且很难被现有机器学习和数据挖掘算法有效地处理。如何对高维数据进行有效维数约简,并由此发现其内在结构和规律已成为高维信息处理研究的关键问题之一。流形学习的主要目标是发现蕴含在高维数据集中的内在几何结构与规律性,是近年来机器学习和模式识别等领域一个新的研究热点。
本文对流形学习算法及其应用问题进行了研究,主要工作及研究成果总结如下:
①在对PCA和MVU算法分析的基础上,提出了有区别方差嵌入(DVE)算法。通过构造数据集的近邻图和非近邻图,DVE算法对样本方差采取了不同的处理方式,使低维表示全局方差最大的同时保持局部方差不变。DVE可以看作是PCA算法的非线性扩展,同时也可以看作是对MVU算法严格局部等距约束的松弛改进。DVE是一种全局维数约简算法,可以有效揭示蕴含在高维数据集中的全局几何结构和内在规律。与MVU和ISOMAP相比,DVE算法具有小的运算强度和存储需求。另外,DVE算法对具有等角映射特性的数据集有很好的降维效果,而ISOMAP和MVU的距离保持特性使得它们无法处理此类数据集。
②DVE算法需要对稠密矩阵进行特征分解,尽管与ISOMAP和MVU相比,算法的计算复杂度有了很大的降低,但仍无法满足对现实世界中海量高维数据的实时处理要求。针对这一问题,提出了基于基准点的DVE快速算法(LDVE)。在保持近邻点间距离和不变的条件下,LDVE算法通过使随机选取的基准点间的距离和最大在低维空间中展开高维数据流形,算法的求解也同时转化为稀疏矩阵的特征分解问题,从而有效降低了计算强度和存储需求。
③DVE算法得不到一个显式映射函数,无法对新增数据点进行有效处理,针对这一问题,通过对DVE算法进行线性逼近,提出了有区别方差投影(DVP)算法。和DVE算法一样,DVP算法在揭示数据集全局结构的同时有效保存它的局部结构,可以作为经典PCA和LPP的有效补充。
④DVP是一种非监督维数约简算法,它并不能保证不同类别的数据点在低维投影空间中可以被有效分开。针对这一问题,提出了监督有区别方差投影(SDVP)算法。通过构造数据集的类内近邻图和类间图,SDVP算法使得高维数据集在低维空间中投影的类内局部散度最小,同时类间全局散度最大。SDVP可以看作是线性判别分析(LDA)的局部化形式,而边际Fisher分析(MFA)又可以看作是SDVP的局部化形式。SDVP算法对具有多模态或嵌入流形结构的数据集有好的分类效果。在UCI机器学习数据库和一些标准人脸数据库上的分类实验证明了算法的优越性。
With the quick advancement and extensive application of information technology, more data with high dimension and complex structure occurs very quickly. High dimension not only makes the data hard to understand, and makes traditional machine learning and data mining techniques less effective. How to reduce the high dimensional data into the low dimensional space and discover the intrinsic structure have become the pivotal problem in high dimensional information processing. The main purpose of manifold learning algorithms is to detect the intrinsic structure embedded in the high dimensional data space, which have been a focused research field in machine learning and pattern recognition.
In this dissertation, some key issues on manifold learning have been studied, the main contributions are summarized as follows:
Baed on the analysis of PCA and MVU, we propose a new nonlinear dimensionality reduction method called distinguishing variance embedding (DVE). By constructing the neighborhood graph and non-neighborhood graph, DVE deals with the sample variance distinguishingly, that maximizes the global variance and simultaneously preserves the local variance. DVE can be viewed as the nonlinear counterpart of PCA, and also be viewed as a variant of MVU that relaxes the strict distance-preserving constraints. As a global algorithm for nonlinear dimensionality reduction, DVE can detect the global geometric structure of data set in the high demensional space. Compared with MVU and ISOMAP, the computation intensity and storage demands of DVE are drastically reduced. DVE can also effectively deal with the conformal data set while ISOMAP and MVU fail for their isometric property.
Despite the computational complexity of DVE greatly reduced when compared with ISOMAP and MVU, the eigen-decompose of dense matrix in DVE makes it can't meet the real-time data processing requirements in the real world. In order to solve this problem, the landmark version of DVE is proposed. Subject to the constraint that the total sum of distances between neighboring points remain unchanged, landmark DVE unfolds the data manifold in the low dimensional space by pull the randomly selected landmark points as far apart as possible. The main optimization of landmark DVE involves an eigen-decompose of spare matrix. Compared with DVE, the computation intensity and storage demands of landmark DVE are effectively reduced.
Like other manifold learning algorithms, DVE has no straightforward extension for out-of-sample examples as it can't get a mapping function. In order to solve this problem, the linear approximation of DVE is introduced, which is called distinguishing variance projection (DVP). Similar to DVE, DVP can detect the global structure of high dimensional data set and simultaneously preserve its local neighborhood information in a certain sense. DVP can be viewed as an effective complement of classical PCA and LPP.
As an unsupervised dimensionality reduction algorithm, DVP can't ensure that the data in different category be separated well in low dimensional subspace. As DVP deals with the data points in pairwise manner, the algorithm can be performed in supervised manner by taking the label information into account, called supervised distinguishing variance projection (SDVP). By constructing the intra-class neighborhood graph and inter-class graph, SDVP seeks the low dimensional subspace in which the intra-class local scatter of data is minimized and at the same time the inter-class scatter is maximized. SDVP can be viewed as a local variant of LDA, and MFA can be viewed as a local variant of SDVP. SDVP is suitable for the classification tasks of multi-modal and manifold data set. The experiments on the UCI machine learning databases and standard face databases prove the effectiveness of the algorithm.
本文对流形学习算法及其应用问题进行了研究,主要工作及研究成果总结如下:
①在对PCA和MVU算法分析的基础上,提出了有区别方差嵌入(DVE)算法。通过构造数据集的近邻图和非近邻图,DVE算法对样本方差采取了不同的处理方式,使低维表示全局方差最大的同时保持局部方差不变。DVE可以看作是PCA算法的非线性扩展,同时也可以看作是对MVU算法严格局部等距约束的松弛改进。DVE是一种全局维数约简算法,可以有效揭示蕴含在高维数据集中的全局几何结构和内在规律。与MVU和ISOMAP相比,DVE算法具有小的运算强度和存储需求。另外,DVE算法对具有等角映射特性的数据集有很好的降维效果,而ISOMAP和MVU的距离保持特性使得它们无法处理此类数据集。
②DVE算法需要对稠密矩阵进行特征分解,尽管与ISOMAP和MVU相比,算法的计算复杂度有了很大的降低,但仍无法满足对现实世界中海量高维数据的实时处理要求。针对这一问题,提出了基于基准点的DVE快速算法(LDVE)。在保持近邻点间距离和不变的条件下,LDVE算法通过使随机选取的基准点间的距离和最大在低维空间中展开高维数据流形,算法的求解也同时转化为稀疏矩阵的特征分解问题,从而有效降低了计算强度和存储需求。
③DVE算法得不到一个显式映射函数,无法对新增数据点进行有效处理,针对这一问题,通过对DVE算法进行线性逼近,提出了有区别方差投影(DVP)算法。和DVE算法一样,DVP算法在揭示数据集全局结构的同时有效保存它的局部结构,可以作为经典PCA和LPP的有效补充。
④DVP是一种非监督维数约简算法,它并不能保证不同类别的数据点在低维投影空间中可以被有效分开。针对这一问题,提出了监督有区别方差投影(SDVP)算法。通过构造数据集的类内近邻图和类间图,SDVP算法使得高维数据集在低维空间中投影的类内局部散度最小,同时类间全局散度最大。SDVP可以看作是线性判别分析(LDA)的局部化形式,而边际Fisher分析(MFA)又可以看作是SDVP的局部化形式。SDVP算法对具有多模态或嵌入流形结构的数据集有好的分类效果。在UCI机器学习数据库和一些标准人脸数据库上的分类实验证明了算法的优越性。
With the quick advancement and extensive application of information technology, more data with high dimension and complex structure occurs very quickly. High dimension not only makes the data hard to understand, and makes traditional machine learning and data mining techniques less effective. How to reduce the high dimensional data into the low dimensional space and discover the intrinsic structure have become the pivotal problem in high dimensional information processing. The main purpose of manifold learning algorithms is to detect the intrinsic structure embedded in the high dimensional data space, which have been a focused research field in machine learning and pattern recognition.
In this dissertation, some key issues on manifold learning have been studied, the main contributions are summarized as follows:
Baed on the analysis of PCA and MVU, we propose a new nonlinear dimensionality reduction method called distinguishing variance embedding (DVE). By constructing the neighborhood graph and non-neighborhood graph, DVE deals with the sample variance distinguishingly, that maximizes the global variance and simultaneously preserves the local variance. DVE can be viewed as the nonlinear counterpart of PCA, and also be viewed as a variant of MVU that relaxes the strict distance-preserving constraints. As a global algorithm for nonlinear dimensionality reduction, DVE can detect the global geometric structure of data set in the high demensional space. Compared with MVU and ISOMAP, the computation intensity and storage demands of DVE are drastically reduced. DVE can also effectively deal with the conformal data set while ISOMAP and MVU fail for their isometric property.
Despite the computational complexity of DVE greatly reduced when compared with ISOMAP and MVU, the eigen-decompose of dense matrix in DVE makes it can't meet the real-time data processing requirements in the real world. In order to solve this problem, the landmark version of DVE is proposed. Subject to the constraint that the total sum of distances between neighboring points remain unchanged, landmark DVE unfolds the data manifold in the low dimensional space by pull the randomly selected landmark points as far apart as possible. The main optimization of landmark DVE involves an eigen-decompose of spare matrix. Compared with DVE, the computation intensity and storage demands of landmark DVE are effectively reduced.
Like other manifold learning algorithms, DVE has no straightforward extension for out-of-sample examples as it can't get a mapping function. In order to solve this problem, the linear approximation of DVE is introduced, which is called distinguishing variance projection (DVP). Similar to DVE, DVP can detect the global structure of high dimensional data set and simultaneously preserve its local neighborhood information in a certain sense. DVP can be viewed as an effective complement of classical PCA and LPP.
As an unsupervised dimensionality reduction algorithm, DVP can't ensure that the data in different category be separated well in low dimensional subspace. As DVP deals with the data points in pairwise manner, the algorithm can be performed in supervised manner by taking the label information into account, called supervised distinguishing variance projection (SDVP). By constructing the intra-class neighborhood graph and inter-class graph, SDVP seeks the low dimensional subspace in which the intra-class local scatter of data is minimized and at the same time the inter-class scatter is maximized. SDVP can be viewed as a local variant of LDA, and MFA can be viewed as a local variant of SDVP. SDVP is suitable for the classification tasks of multi-modal and manifold data set. The experiments on the UCI machine learning databases and standard face databases prove the effectiveness of the algorithm.
引文
[1] D. L. Donoho. High-dimensional data analysis: the curses and blessings of dimensionality [C]. In Lecture, American Math Society on -- Math Challenges of the 21st Century, Los Angeles, 2000.
[2] M. Belkin, P. Niyogi, Semi-Supervised Learning on Riemannian Manifolds [J]. Machine Learning, 2004. 56(1): 209-239.
[3] I. T. Jolliffe. Principal Component Analysis [M]. New York: Springer Verlag, 1989.
[4] K. I. Diamantaras, S. Y. Kung, Principal component neural networks: theory and applications [M]. John Wiley & Sons, Inc., New York, NY, USA, 1996.
[5] H. Hotelling. Analysis of a Complex of Statistical Variables with Principal Components [J]. 1933, Journal of Educational Psychology, 24 (1933): 417-441 and 498-520.
[6] T. Cox, M. Cox. Multidimensional Scaling [M]. London: Chapman & Hall, 1994.
[7]张润楚,多元统计分析[M],北京:科学出版社, 2003.
[8] D. R. Hardoon, S. Szedmak, J. Shawe-Taylor. Canonical correlation analysis: An overview with application to learning methods [R]. Technical Report CSD-TR-03-02, 2008, Computer Science Department, Royal Holloway, University of London.
[9] M. Borga. Learning Multidimensional Signal Processing [D]. Linkping Studies in Science and Technology, 1998.
[10] A. Hyv?rinen, E. Oja. Independent Component Analysis: Algorithms and Applications [J]. Neural Networks, 2000, 13(4-5): 411-430.
[11] H. Hotelling. Relations between Two Sets of Variates [J]. Biometrika, 1936, 28: 321-377.
[12] C. Jutten, J. Hérault. Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture [J]. Signal Processing, 1991, 24:1-10.
[13] A. J. Bell, T.J. Sejnowski. An information-maximization approach to blind separation and blind deconvolution [J]. Neural Computation, 7(6): 1129-1159, 1996.
[14] R. O. Duda, P. E. Hart, D. G. Stork. Pattern Classification [M]. 2nd ed., John Wiley & Sons, Inc. 2000.
[15] R. A. Fisher. The use use of multiple measurements in taxonomic problems [R]. Annals of Eugenics, 7(2) : 179-188, 1936.
[16] S. Haykin, Neural Networks: A Comprehensive Foundation [M]. 2nd ed., Prentice Hall, NJ, USA, 1998.
[17] T. Kohonen, Self-organized formation of topologically correct feature maps [J]. BiologicalCybernetics, 43(1): 59-69, 1982.
[18] T. Kohonen, Self-organization and associative memory [M]. 3th ed., Springer-Verlag, Berlin, 1989.
[19] J. MAO, A. K. JAIN. A self-organizing network for hyperellipsoidal clustering (HEC) [J]. IEEE transactions on neural networks, TNN-7(1): 16-29, 1996.
[20] Stephen Grossberg. Competitive learning: From interactive activation to adaptive resonance [J]. Cognitive Science, 11(1): 23-63, 1987.
[21] Daniel S. Levine. Introduction to Neural and Cognitive Modeling [M]. 2nd ed., Mahwah, NJ: Lawrence Erlbaum Associates, 2000.
[22] D. G. Stork. Is backpropagation biologically plausible [C]. In proceedings of the International Joint Conference on Neural Networks (IJCNN),Ⅱ-241-246. IEEE, New York, 1989.
[23] B. E. Boser, I. M. Guyon, and V. N. Vapnik. A training algorithm for optimal margin classifiers [C]. In D. Haussler, editor, Proceedings of the 5th Annual ACM Workshop on Computational Learning Theory (COLT), 144-152, ACM Press, 1992.
[24] N. Cristianini, J. Shawe-Taylor. An Introduction to Support Vector Machine [M]. Cambridge: Cambridge University Press, 2000.
[25] B. Sch?lkopf, A.J. Smola, Learning with Kernels [M]. MIT Press, 2002.
[26] V. Vapnik, A. Chervonenkis. A note on one class of perceptrons [J]. Automation and Remote Control, 25, 1964.
[27] M. Aizerman, E. Braverman, and L. Rozonoer. Theoretical foundations of the potential function method in pattern recognition learning [J]. Automation and Remote Control, 1964, 25: 821-837.
[28] B. Scholkopf, A. Smola, K.-R. Muller, Nonlinear component analysis as a kernel eigenvalue problem [J], Neural Computation, 10(5): 1299-1319, 1998.
[29] S. Akaho. A kernel method for canonical correlation analysis [C]. In Proceedings of the International Meeting of the Psychometric Society (IMPS2001). Springer-Verlag, 2001.
[30] P. L. Lai, C. Fyfe. Kernel and Nonlinear Canonical Correlation Analysis [J]. International Journal of Neural Systems, 10(5): 365-377, 2000.
[31] F. R. Bach, M.I. Jordan. Kernel Independent Component Analysis [J]. Journal of Machine Learning Research, 2002, 3: 1-48.
[32] G. Baudat, F. Anouar, Generalized Discriminant Analysis Using a Kernel Approach [J]. Neural computation, 12(10): 2385-2404, 2000.
[33] S. Mika, G. R?tsch, J. Weston, B. SchQolkopf, K.-R. MQuller. Fisher discriminant analysis with kernels [C]. IEEE International Workshop on Neural Networks for Signal Processing IX,Madison, USA, August, 41–48, 1999.
[34] A. Vinokourov, D. Hardoon, J. Shawe-Taylor. Learning the semantics of multimedia content with applpcation to web image retrieval and classication [C]. In Proceedings of Fourth Internaational Symposium on Independent Component Analysis and Blind Source Separation, Nara, 2003.
[35] D. R. Hardoon, J. Shawe-Taylor. KCCA for different level precision in content-based image retrieval [C]. In: Third International Workshop on Content-Based Multimedia Indexing, 2004, IRISA, Rennes, France.
[36] J. P. Vert, M. Kanehisa. Graph-Driven Features Extraction from Microarray Data using Diffusion Kernels and Kernel CCA [C]. NIPS15, 2002, MIT Press, 2003.
[37] Y. Yamanishi, J.P. Vert. Extraction of correlated gene clusters from multiple genomic data by generalized kernel canonical correlation analysis [R]. Bioinformatics, 2003.
[38] J. Shawe-Taylor, N. Cristianini, Kernel Methods for Pattern Analysis [M]. New York, NY: Cambridge University Press, 2004.
[39] H. S. Seung, D.D. Lee. The manifold ways of perception [J]. Science, 290: 2268-2269, 2000.
[40] J. B. Tenenbaum, V. de Silva, J.C. Langford. A global geometric framework for nonlinear dimensionality reduction [J]. Science, 290: 2319-2323, 2000.
[41] S. T. Roweis, L.K. Saul. Nonlinear dimensionality reduction by locally linear embedding [J]. Science, 290: 2323-2326, 2000.
[42] S. K. Nayar, S.A. Nene, H. Murase. Subspace methods for robot vision [J]. IEEE Transactions on Rbotics and Automation, 1996, 2(5): 750-758.
[43] C. Bregler, S.M. Omohundro. Nonlinear manifold learning for visual speech recognition [C]. ICCV 1995, 494-499.
[44] L. K. Saul, S.T. Roweis, Think globally, fit locally: Unsupervised learning of low dimensional manifold [J]. Journal of Machine Learning Research 4, 119-155, 2003.
[45] M. Belkin, P. Niyogi. Laplacian eigenmaps and spectral techniques for embedding and clustering [A]. In T.G. Dietterich, S. Becker, and Z. Ghahramani, editors, Advances in Neural Information Processing Systems 14 [C], Cambridge, MA, 585-591. MIT Press, 2002.
[46] M. Belkin, P. Niyogi. Laplacian eigenmaps for dimensionality reduction and data representation [J]. Neural Computation, 2003, 15 (6): 1373-1396.
[47] K. Q. Weinberger, F. Sha, L. K. Saul. Learning a kernel matrix for nonlinear dimensionality reduction [C]. In Proceedings of the Twenty First International Confernence on Machine Learning (ICML-04), 839-846. Banff, Canada, 2004.
[48] K. Q. Weinberger, L. K. Saul. Unsupervised learning of image manifolds by semidefiniteprogramming [C]. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR-04), vol. 2, 988-995. Washington D.C., 2004.
[49] K. Q. Weinberger, B. D. Packer, and L. K. Saul. Nonlinear dimensionality reduction by semidefinite programming and kernel matrix factorization [A]. In Z. Ghahramani and R. Cowell (eds.), Proceedings of the Tenth International Workshop on Artificial Intelligence and Statistics [C], 381-388. Barbados, West Indies, 2005.
[50] K. Q. Weinberger, L. K. Saul. An introduction to nonlinear dimensionality reduction by maximum variance unfolding [C]. In Proceedings of the Twenty First National Conference on Artificial Intelligence (AAAI-06), Boston, MA, 2006.
[51] K. Q. Weinberger, L. K. Saul. Unsupervised learning of image manifolds by semidefinite programming [J]. International Journal of Computer Vision 70(1): 77-90, 2006.
[52] Z. Y. Zhang, H. Y. Zha, Principal manifolds and nonlinear dimensionality reduction via tangent space alignment [J]. SIAM Journal of Scientific Computing, 2004, 26 (1): 313-338.
[53]张振跃,查宏远.线性低秩逼近与非线性降维[J].中国科学A辑数学,2005, 35(3): 372-285.
[54] M. Balasubramanian, E.L. Schwartz, J.B. Tenenbaum, V. de Silva, J.C. Langford, The ISOMAP Algorithm and Topological Stability [J]. Science, 2002. 295(5552): 7-7.
[55] V. De Silva, Joshua B. Tenenbaum. Global versus local methods in nonlinear dimensionality reduction [C]. in Advances in Neural Information Processing Systems. 2003.
[56] M. Bernstein, S. V. de, J. C. Langford, J.B. Tenenbaum, Graph approximations to geodesics on embedded manifolds [R]. 2000, Department of Psychology, Stanford University.
[57] H. Y. Zha, Z.Y. Zhang. Isometric embedding and continuum ISOMAP [C]. In Proceedings of the Twentieth International Conference on Machine Learning (ICML2003), 2003.
[58] H. Y. Zha, Z.Y. Zhang. Continuum ISOMAP for manifold learnings [J]. Computational Statistics & Data Analysis, 2006.
[59] H. Chen, G. Jiang, K. Yoshihira. Robust Nonlinear Dimensionality Reduction for Manifold Learning [C]. In proceedings of the 18th International Conference on Pattern Recognition (ICPR'06), 2006.
[60] L. Yang. Building k edge-disjoint spanning trees of minimum total length for isometric data embedding [J]. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 2005. 27(10): 1680-1683.
[61]何力,张军平,周志华.基于放大因子和延伸方向研究流形学习算法[J].计算机学报,2005. 28(12): 2000-2009.
[62] J. A. Lee, A. Lendasse, M. Verleysen, Curvilinear distance analysis versus ISOMAP [C].Proceedings of European Symposium on Artificial Neural Networks, 19th, 2002, 185-192.
[63] O. Kouropteva, O. Okun, M. Pietikainen. Selection of the optimal parameter value for the locally linear embedding algorithm [C]. In Proc. of the 1st int. Conf. on Fuzzy Systems and Knowledge Discovery, Singapore. 2002.
[64] H. Chang, D. Y. Yeung, Robust Locally Linear Embedding [J]. Pattern recognition, 2006. 39(6): 1053-1065.
[65] O. Kouropteva, O. Okun, M. Pietikainen. Classification of handwritten digits using supervised locally linear embedding algorithm and support vector machine [C]. In Proc. of the 11th European Symp. on Artificial Neural Networks, Bruges, Belgium. 2003.
[66] Y. Chang, C. Hu, M. Turk. Manifold of facial expression [C]. in Analysis and Modeling of Faces and Gestures, 2003. AMFG 2003. IEEE International Workshop on. 2003.
[67] N. Mekuz, C. Bauckhage, J. K. Tsotsos. Face recognition with weighted locally linear embedding [C]. in Computer and Robot Vision, 2005. Proceedings. The 2nd Canadian Conference on. 2005.
[68]张军平,何力.监督流形学习[A].见:周志华,王珏编.机器学习及其应用2007 [M]北京:清华大学出版社. 2007, 10: 194-231.
[69]詹德川,周志华,基于流形学习的多示例回归算法[J].计算机学报,2006. 29(11): 1948-1955.
[70] Jürgen Jost. Riemannian Geometry and Geometric Analysis [M]. Berlin: Springer-Verlag, 2002.
[71] Mikhail Belkin, Partha Niyogi. Semi-supervised learning on manifolds [R], Technical Report, 2002.
[72] M.Belkin, P. Niyogi, Semi-Supervised Learning on Riemannian Manifolds [J]. Machine Learning, 2004. 56(1): 209-239.
[73] M. Belkin, P. Niyogi, V. Sindhwani. Manifold Regularization: a Geometric Framework for Learning from Labeled and Unlabeled Examples [J]. Journal of Machine Learning Research, 2006, 7(Nov): 2399-2434.
[74] M. Brand, Continuous nonlinear dimensionality reduction by kernel eigenmaps [C]. in Int. Joint Conf. Artif. Intel. 2003.
[75] D. L. Donoho, C. Grimes, Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data [J]. Proceedings of the National Academy of Sciences, 2003. 100(10): 5591-5596.
[76] F. S. Beckman, D. A. Quarles. On isometries of Euclidean space [C]. Proc. Amer. Math. Soc., 4 (1953): 810-815.
[77] L. K. Saul, K. Q. Weinberger, J. H. Ham, F. Sha, and D. D. Lee. Spectral methods for dimensionality reduction [A]. In O. Chapelle, B. Schoelkopf, and A. Zien (eds.), Semisupervised Learning [M]. Cambridge, MA: MIT Press, 2006.
[78] C. J. C. Burges, Geometric methods for feature extraction and dimensional reduction - a guided tour [A]. in: O. Maimon, L. Rokach (eds.), The Data Mining and Knowledge Discovery Handbook [C], Springer, 2005, 59-92.
[79]罗四维,赵连伟.基于谱图理论的流形学习算法[J].计算机研究与发展, 2006, 43(7): 1173-1179.
[80] J. Ham, D. D. Lee, S. Mika, and B. Sch?lkopf. A kernel view of the dimensionality reduction of manifolds [C]. In Proceedings of the Twenty First International Conference on Machine Learning (ICML-04), 369-376, 2004.
[81] Y. Bengio, J. F. Paiement, P. Vincent. Out-of-Sample Extensions for LLE, ISOMAP, MDS, Eigenmaps, and Spectral Clustering [C]. in Advances in Neural Information Processing Systems. 2004.
[82] M. Brand. Charting a manifold [C]. In Advances in Neural Information Processing Systems 15, 961-968, Cambridge, MA, USA. The MIT Press, 2003.
[83] Y. W. Teh, S. Roweis. Automatic alignment of local representations [C]. In: Advances in Neural Information Processing Systems 15. MIT Press, 2003, 15: 841-848.
[84] J. H. Ham, D. D. Lee, and L. K. Saul. Semisupervised alignment of manifolds [A]. In Z. Ghahramani and R. Cowell (eds.), Proceedings of the Tenth International Workshop on Artificial Intelligence and Statistics [C], 2005, 120-127. Barbados, West Indies
[85] M. H. C. Law, N. Zhang, A. K. Jain, Nonlinear manifold learning for data stream [C]. Proceedings of SIAM Data Mining, Florida, USA, 2004, 33-44.
[86] J. Bruske, G. Sommer, Topology representing networks for intrinsic dimensionality estimation [C]. Wulfram Gerstner, Alain Germond, Martin Hasler, et al, ICANN’97, Springer LNCS 1327, 1997, 595-600.
[87] X. Yang, H. Fu, H. Zha, et al. Semi-Supervised Nonlinear Dimensionality Reduction [C]. Proceedings of the 23rd international conference on Machine learning. ACM Press New York, NY, USA, 2006. 1065-1072.
[88]张军平.流形学习及应用[D].北京:中国科学院自动化研究所,复杂系统与智能科学重点实验室, 2003.
[89] M. Belkin, P. Niyogi. Using Manifold Structure for Partially Labeled Classification [C]. Advances in Neural Information Processing Systems, 2003, 15:929-936.
[90] J. Ye, R. Janardan, Q. Li. GPCA: an efficient dimension reduction scheme for imagecompression and retrieval [A]. In Won Kim, Ron Kohavi, Johannes Gehrke, and William DuMouchel, editors, The Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining [C], 354-363. ACM, 2004.
[91] H. Chang, D. Y. Yeung. Locally linear metric adaptation with application to semi-supervised clustering and image retrieval [J]. Pattern Recognition. 2006, 39(7): 1253-1264.
[92] R. Souvenir, R. Pless. Manifold clustering [C]. Proceedings of the International Conference on Computer Vision, 2005, 1: 648-653.
[93] A. E Seward, B. Bodenheimer. Using nonlinear dimensionality reduction in 3D figure animation [C]. Proceedings of the ACM Regional Conference, 2005, 2: 388-392.
[94] J. Banfield, A. Raftery. Ice Floe Identification in Satellite Images Using Mathematical Morphology and Clustering about Principal Curves [J]. Journal of the American Statistical Association, 1992, 87: 7-16.
[95] B. Kégl, A. Krzyzak. Piecewise Linear Skeletonization Using Principal Curves [J]. IEEE Transaction on Pattern Analysis and Machine Intelligence, 2002, 24(1): 59-74.
[96] R. D. de, O. Kouropteva, O. Okun, et al. Supervised locally linear embedding [C]. in Proc. Joint Int. Conf. ICANN/ICONIP. 2003.
[97] X. Geng, D. C. Zhan, Z.H. Zhou. Supervised nonlinear dimensionality reduction for visualization and classification [J]. IEEE Transactions on Systems, Man, and Cybernetics - Part B: Cybernetics, 2005, 35(6): 1098-1107.
[98] S. WENG, C. ZHANG, Z. LIN, Exploring the structure of supervised data by Discriminant Isometric Mapping [J]. Pattern recognition, 2005. 38(4): 599-601.
[99] H. Y. Li, L. Teng, W. Chen, et al. Supervised Learning on Local Tangent Space [C]. in: Advances in Neural Networks-ISNN2005, Lecture Notes in Computer Science, Heidelberg: Springer Berlin, 2005, 3496: 546-551.
[100] M. Belkin and P. Niyogi, Using Manifold Structure for Partially Labelled Classification [A], Advances in NIPS [C], vol. 15, 2003.
[101]陈省身,陈维桓.微分几何讲义[M].北京:北京大学出版社, 2001.
[102]陈维桓.微分流形初步[M].北京:高等教育出版社, 2001.
[103]陈维桓.流形上的微积分[M].见:萧树铁编.大学数学.北京:高等教育出版社, 2003.
[104] Michael Spivak,齐民友,路见可译.流形上的微积分——高等微积分中的一些经典定理的现代化处理(双语版)[M],北京:人民邮电出版社, 2006.
[105]施恩伟.流形上的微积分[M].北京:科学出版社, 2004.
[106] R. A. Horn, C. R. Johnson,杨奇译.矩阵分析[M],北京:机械工业出版社, 2005.
[107] J. H. Friedman, Regularized Discriminant Analysis [J]. Journal of the American StatisticalAssociation, 1989. 84(405): 165-175.
[108] P. N. Belhumeur, J.P. Hespanha, D.J. Kriegman. Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection [J]. IEEE Trans. Pattern Analysis and Machine Intelligence, 1997, 19(7): 711-720.
[109] L. F. Chen, H.Y.M. Liao, M.T. Ko, et al. A New LDA-Based Face Recognition System Which can Solve the Small Sample Size Problem [J]. Pattern Recognition, 2000, 33(10): 1713-1726.
[110]梁毅雄.基于子空间分析的人脸特征提取及识别研究[D].重庆大学研究生院, 2005.
[111] Z. Jin, J. Y. Yang, Z. S. Hu, et al. Face Recognition Based on the Uncorrelated Discriminant Transformation [J]. Pattern Recognition, 2001, 34(7): 1405-1416.
[112] M. Skuirchina, R. P. W. Duin. Bagging, Boosting, and the Random Subspace Method for Linear Classifiers [J]. Pattern Analysis and Applications, 2002, 5:121-135.
[113]赵连伟,罗四维,赵艳敞,刘蕴辉.高维数据流形的低维嵌入及嵌入维数研究[J].软件学报, 2005, 16(8): 1423-1430.
[114]王式安,数理统计[M],北京:北京理工大学出版社, 2004.
[115] J. B. Shi, J. Malik. Normalized Cuts and Image Segmentation [J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2000, 22(8): 888-905.
[116] L. Zelnik-Manor, P. Perona. Self-tuning spectral clustering [A]. In L. K. Saul, Y.Weiss, and L. Bottou, editors, Advances in Neural Information Processing Systems 17 [C], Cambridge, MA: MIT Press, 2005: 1601–1608.
[117] S. A. Nene, S. K. Nayar, H. Murase, Columbia object image library (COIL20) [R], Technical Report CUCS-005-96, Columbia University, New York, 1996.
[118] Y. LeCun, L. Bottou, Y. Bengio, et al. Gradient-based learning applied to document recognition [C]. Proceedings of the IEEE, 86(11): 2278-2324, November 1998.
[119] X. F. He, P. Niyogi. Locality Preserving Projections [C]. Advances in Neural Information Processing Systems 16, Vancouver, Canada, 2003.
[120] X. F. He. Locality Preserving Projections [D]. Illinois, American: Computer Science Department, The University of Chicago, 2005
[121] D. Cai, S. C. Yan, et al. Neighborhood preserving embedding [C]. IEEE International Conference on Computer Vision (ICCV), Beijing, China, 2005.
[122] T. H. Zhang, J. Yang, D. L. Zhao, et al. Linear Local Tangent Space Alignment and Application to Face Recognition [J]. Neurocomputing, 2007, 70(7-9): 1547-1553.
[123] D. Cai, X. F. He, J.W. Han. Spectral Regression: A Unified Subspace Learning Framework for Content-Based Image Retrieval [J]. ACM Multimedia 2007, Augsburg, Germany, Sep. 2007.
[124] S. C. Yan, D. Xu, B.Y Zhang, et al, Graph Embedding and Extension: a General Framework for Dimensionality Reduction [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2007, 29(1): 40-51.
[125] A. Tikhonov, V. Arsenin. Solution of ill-posed problems [M]. Washington: Winston & Sons, 1977.
[126] X. F. He, S. C. Yan, Y. X. Hu, et al. Face recognition using laplacianfaces [J]. IEEE Trans. Pattern Anal. Mach. Intell. 27 (3): 328-340, 2005.
[127] J. Hull, A database for handwritten text recognition research [J], Transactions on Pattern Analysis and Machine Intelligence 16 (5): 550-554, May 1994.
[128] F. S. Samaria, A. C. Harter. Parameterisation of a stochastic model for human face identification [C]. Proceedings of 2nd IEEE Workshop on Applications of Computer Vision, Sarasota FL, 1994.
[129] Yale Univ. Face database [Z]. http://cvc.yale.edu/projects/yalefaces/yalefaces.html.
[130] T. Sim, S. Baker, M. Bsat. The CMU Pose, Illumination, and Expression Database [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2003. 25(12): 1615-1618.
[131] M. Turk, A. Pentland. Eigenfaces for recognition [J]. Journal of Cognitive Neuroscience, 1991, 3 (1): 71-86.
[2] M. Belkin, P. Niyogi, Semi-Supervised Learning on Riemannian Manifolds [J]. Machine Learning, 2004. 56(1): 209-239.
[3] I. T. Jolliffe. Principal Component Analysis [M]. New York: Springer Verlag, 1989.
[4] K. I. Diamantaras, S. Y. Kung, Principal component neural networks: theory and applications [M]. John Wiley & Sons, Inc., New York, NY, USA, 1996.
[5] H. Hotelling. Analysis of a Complex of Statistical Variables with Principal Components [J]. 1933, Journal of Educational Psychology, 24 (1933): 417-441 and 498-520.
[6] T. Cox, M. Cox. Multidimensional Scaling [M]. London: Chapman & Hall, 1994.
[7]张润楚,多元统计分析[M],北京:科学出版社, 2003.
[8] D. R. Hardoon, S. Szedmak, J. Shawe-Taylor. Canonical correlation analysis: An overview with application to learning methods [R]. Technical Report CSD-TR-03-02, 2008, Computer Science Department, Royal Holloway, University of London.
[9] M. Borga. Learning Multidimensional Signal Processing [D]. Linkping Studies in Science and Technology, 1998.
[10] A. Hyv?rinen, E. Oja. Independent Component Analysis: Algorithms and Applications [J]. Neural Networks, 2000, 13(4-5): 411-430.
[11] H. Hotelling. Relations between Two Sets of Variates [J]. Biometrika, 1936, 28: 321-377.
[12] C. Jutten, J. Hérault. Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture [J]. Signal Processing, 1991, 24:1-10.
[13] A. J. Bell, T.J. Sejnowski. An information-maximization approach to blind separation and blind deconvolution [J]. Neural Computation, 7(6): 1129-1159, 1996.
[14] R. O. Duda, P. E. Hart, D. G. Stork. Pattern Classification [M]. 2nd ed., John Wiley & Sons, Inc. 2000.
[15] R. A. Fisher. The use use of multiple measurements in taxonomic problems [R]. Annals of Eugenics, 7(2) : 179-188, 1936.
[16] S. Haykin, Neural Networks: A Comprehensive Foundation [M]. 2nd ed., Prentice Hall, NJ, USA, 1998.
[17] T. Kohonen, Self-organized formation of topologically correct feature maps [J]. BiologicalCybernetics, 43(1): 59-69, 1982.
[18] T. Kohonen, Self-organization and associative memory [M]. 3th ed., Springer-Verlag, Berlin, 1989.
[19] J. MAO, A. K. JAIN. A self-organizing network for hyperellipsoidal clustering (HEC) [J]. IEEE transactions on neural networks, TNN-7(1): 16-29, 1996.
[20] Stephen Grossberg. Competitive learning: From interactive activation to adaptive resonance [J]. Cognitive Science, 11(1): 23-63, 1987.
[21] Daniel S. Levine. Introduction to Neural and Cognitive Modeling [M]. 2nd ed., Mahwah, NJ: Lawrence Erlbaum Associates, 2000.
[22] D. G. Stork. Is backpropagation biologically plausible [C]. In proceedings of the International Joint Conference on Neural Networks (IJCNN),Ⅱ-241-246. IEEE, New York, 1989.
[23] B. E. Boser, I. M. Guyon, and V. N. Vapnik. A training algorithm for optimal margin classifiers [C]. In D. Haussler, editor, Proceedings of the 5th Annual ACM Workshop on Computational Learning Theory (COLT), 144-152, ACM Press, 1992.
[24] N. Cristianini, J. Shawe-Taylor. An Introduction to Support Vector Machine [M]. Cambridge: Cambridge University Press, 2000.
[25] B. Sch?lkopf, A.J. Smola, Learning with Kernels [M]. MIT Press, 2002.
[26] V. Vapnik, A. Chervonenkis. A note on one class of perceptrons [J]. Automation and Remote Control, 25, 1964.
[27] M. Aizerman, E. Braverman, and L. Rozonoer. Theoretical foundations of the potential function method in pattern recognition learning [J]. Automation and Remote Control, 1964, 25: 821-837.
[28] B. Scholkopf, A. Smola, K.-R. Muller, Nonlinear component analysis as a kernel eigenvalue problem [J], Neural Computation, 10(5): 1299-1319, 1998.
[29] S. Akaho. A kernel method for canonical correlation analysis [C]. In Proceedings of the International Meeting of the Psychometric Society (IMPS2001). Springer-Verlag, 2001.
[30] P. L. Lai, C. Fyfe. Kernel and Nonlinear Canonical Correlation Analysis [J]. International Journal of Neural Systems, 10(5): 365-377, 2000.
[31] F. R. Bach, M.I. Jordan. Kernel Independent Component Analysis [J]. Journal of Machine Learning Research, 2002, 3: 1-48.
[32] G. Baudat, F. Anouar, Generalized Discriminant Analysis Using a Kernel Approach [J]. Neural computation, 12(10): 2385-2404, 2000.
[33] S. Mika, G. R?tsch, J. Weston, B. SchQolkopf, K.-R. MQuller. Fisher discriminant analysis with kernels [C]. IEEE International Workshop on Neural Networks for Signal Processing IX,Madison, USA, August, 41–48, 1999.
[34] A. Vinokourov, D. Hardoon, J. Shawe-Taylor. Learning the semantics of multimedia content with applpcation to web image retrieval and classication [C]. In Proceedings of Fourth Internaational Symposium on Independent Component Analysis and Blind Source Separation, Nara, 2003.
[35] D. R. Hardoon, J. Shawe-Taylor. KCCA for different level precision in content-based image retrieval [C]. In: Third International Workshop on Content-Based Multimedia Indexing, 2004, IRISA, Rennes, France.
[36] J. P. Vert, M. Kanehisa. Graph-Driven Features Extraction from Microarray Data using Diffusion Kernels and Kernel CCA [C]. NIPS15, 2002, MIT Press, 2003.
[37] Y. Yamanishi, J.P. Vert. Extraction of correlated gene clusters from multiple genomic data by generalized kernel canonical correlation analysis [R]. Bioinformatics, 2003.
[38] J. Shawe-Taylor, N. Cristianini, Kernel Methods for Pattern Analysis [M]. New York, NY: Cambridge University Press, 2004.
[39] H. S. Seung, D.D. Lee. The manifold ways of perception [J]. Science, 290: 2268-2269, 2000.
[40] J. B. Tenenbaum, V. de Silva, J.C. Langford. A global geometric framework for nonlinear dimensionality reduction [J]. Science, 290: 2319-2323, 2000.
[41] S. T. Roweis, L.K. Saul. Nonlinear dimensionality reduction by locally linear embedding [J]. Science, 290: 2323-2326, 2000.
[42] S. K. Nayar, S.A. Nene, H. Murase. Subspace methods for robot vision [J]. IEEE Transactions on Rbotics and Automation, 1996, 2(5): 750-758.
[43] C. Bregler, S.M. Omohundro. Nonlinear manifold learning for visual speech recognition [C]. ICCV 1995, 494-499.
[44] L. K. Saul, S.T. Roweis, Think globally, fit locally: Unsupervised learning of low dimensional manifold [J]. Journal of Machine Learning Research 4, 119-155, 2003.
[45] M. Belkin, P. Niyogi. Laplacian eigenmaps and spectral techniques for embedding and clustering [A]. In T.G. Dietterich, S. Becker, and Z. Ghahramani, editors, Advances in Neural Information Processing Systems 14 [C], Cambridge, MA, 585-591. MIT Press, 2002.
[46] M. Belkin, P. Niyogi. Laplacian eigenmaps for dimensionality reduction and data representation [J]. Neural Computation, 2003, 15 (6): 1373-1396.
[47] K. Q. Weinberger, F. Sha, L. K. Saul. Learning a kernel matrix for nonlinear dimensionality reduction [C]. In Proceedings of the Twenty First International Confernence on Machine Learning (ICML-04), 839-846. Banff, Canada, 2004.
[48] K. Q. Weinberger, L. K. Saul. Unsupervised learning of image manifolds by semidefiniteprogramming [C]. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR-04), vol. 2, 988-995. Washington D.C., 2004.
[49] K. Q. Weinberger, B. D. Packer, and L. K. Saul. Nonlinear dimensionality reduction by semidefinite programming and kernel matrix factorization [A]. In Z. Ghahramani and R. Cowell (eds.), Proceedings of the Tenth International Workshop on Artificial Intelligence and Statistics [C], 381-388. Barbados, West Indies, 2005.
[50] K. Q. Weinberger, L. K. Saul. An introduction to nonlinear dimensionality reduction by maximum variance unfolding [C]. In Proceedings of the Twenty First National Conference on Artificial Intelligence (AAAI-06), Boston, MA, 2006.
[51] K. Q. Weinberger, L. K. Saul. Unsupervised learning of image manifolds by semidefinite programming [J]. International Journal of Computer Vision 70(1): 77-90, 2006.
[52] Z. Y. Zhang, H. Y. Zha, Principal manifolds and nonlinear dimensionality reduction via tangent space alignment [J]. SIAM Journal of Scientific Computing, 2004, 26 (1): 313-338.
[53]张振跃,查宏远.线性低秩逼近与非线性降维[J].中国科学A辑数学,2005, 35(3): 372-285.
[54] M. Balasubramanian, E.L. Schwartz, J.B. Tenenbaum, V. de Silva, J.C. Langford, The ISOMAP Algorithm and Topological Stability [J]. Science, 2002. 295(5552): 7-7.
[55] V. De Silva, Joshua B. Tenenbaum. Global versus local methods in nonlinear dimensionality reduction [C]. in Advances in Neural Information Processing Systems. 2003.
[56] M. Bernstein, S. V. de, J. C. Langford, J.B. Tenenbaum, Graph approximations to geodesics on embedded manifolds [R]. 2000, Department of Psychology, Stanford University.
[57] H. Y. Zha, Z.Y. Zhang. Isometric embedding and continuum ISOMAP [C]. In Proceedings of the Twentieth International Conference on Machine Learning (ICML2003), 2003.
[58] H. Y. Zha, Z.Y. Zhang. Continuum ISOMAP for manifold learnings [J]. Computational Statistics & Data Analysis, 2006.
[59] H. Chen, G. Jiang, K. Yoshihira. Robust Nonlinear Dimensionality Reduction for Manifold Learning [C]. In proceedings of the 18th International Conference on Pattern Recognition (ICPR'06), 2006.
[60] L. Yang. Building k edge-disjoint spanning trees of minimum total length for isometric data embedding [J]. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 2005. 27(10): 1680-1683.
[61]何力,张军平,周志华.基于放大因子和延伸方向研究流形学习算法[J].计算机学报,2005. 28(12): 2000-2009.
[62] J. A. Lee, A. Lendasse, M. Verleysen, Curvilinear distance analysis versus ISOMAP [C].Proceedings of European Symposium on Artificial Neural Networks, 19th, 2002, 185-192.
[63] O. Kouropteva, O. Okun, M. Pietikainen. Selection of the optimal parameter value for the locally linear embedding algorithm [C]. In Proc. of the 1st int. Conf. on Fuzzy Systems and Knowledge Discovery, Singapore. 2002.
[64] H. Chang, D. Y. Yeung, Robust Locally Linear Embedding [J]. Pattern recognition, 2006. 39(6): 1053-1065.
[65] O. Kouropteva, O. Okun, M. Pietikainen. Classification of handwritten digits using supervised locally linear embedding algorithm and support vector machine [C]. In Proc. of the 11th European Symp. on Artificial Neural Networks, Bruges, Belgium. 2003.
[66] Y. Chang, C. Hu, M. Turk. Manifold of facial expression [C]. in Analysis and Modeling of Faces and Gestures, 2003. AMFG 2003. IEEE International Workshop on. 2003.
[67] N. Mekuz, C. Bauckhage, J. K. Tsotsos. Face recognition with weighted locally linear embedding [C]. in Computer and Robot Vision, 2005. Proceedings. The 2nd Canadian Conference on. 2005.
[68]张军平,何力.监督流形学习[A].见:周志华,王珏编.机器学习及其应用2007 [M]北京:清华大学出版社. 2007, 10: 194-231.
[69]詹德川,周志华,基于流形学习的多示例回归算法[J].计算机学报,2006. 29(11): 1948-1955.
[70] Jürgen Jost. Riemannian Geometry and Geometric Analysis [M]. Berlin: Springer-Verlag, 2002.
[71] Mikhail Belkin, Partha Niyogi. Semi-supervised learning on manifolds [R], Technical Report, 2002.
[72] M.Belkin, P. Niyogi, Semi-Supervised Learning on Riemannian Manifolds [J]. Machine Learning, 2004. 56(1): 209-239.
[73] M. Belkin, P. Niyogi, V. Sindhwani. Manifold Regularization: a Geometric Framework for Learning from Labeled and Unlabeled Examples [J]. Journal of Machine Learning Research, 2006, 7(Nov): 2399-2434.
[74] M. Brand, Continuous nonlinear dimensionality reduction by kernel eigenmaps [C]. in Int. Joint Conf. Artif. Intel. 2003.
[75] D. L. Donoho, C. Grimes, Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data [J]. Proceedings of the National Academy of Sciences, 2003. 100(10): 5591-5596.
[76] F. S. Beckman, D. A. Quarles. On isometries of Euclidean space [C]. Proc. Amer. Math. Soc., 4 (1953): 810-815.
[77] L. K. Saul, K. Q. Weinberger, J. H. Ham, F. Sha, and D. D. Lee. Spectral methods for dimensionality reduction [A]. In O. Chapelle, B. Schoelkopf, and A. Zien (eds.), Semisupervised Learning [M]. Cambridge, MA: MIT Press, 2006.
[78] C. J. C. Burges, Geometric methods for feature extraction and dimensional reduction - a guided tour [A]. in: O. Maimon, L. Rokach (eds.), The Data Mining and Knowledge Discovery Handbook [C], Springer, 2005, 59-92.
[79]罗四维,赵连伟.基于谱图理论的流形学习算法[J].计算机研究与发展, 2006, 43(7): 1173-1179.
[80] J. Ham, D. D. Lee, S. Mika, and B. Sch?lkopf. A kernel view of the dimensionality reduction of manifolds [C]. In Proceedings of the Twenty First International Conference on Machine Learning (ICML-04), 369-376, 2004.
[81] Y. Bengio, J. F. Paiement, P. Vincent. Out-of-Sample Extensions for LLE, ISOMAP, MDS, Eigenmaps, and Spectral Clustering [C]. in Advances in Neural Information Processing Systems. 2004.
[82] M. Brand. Charting a manifold [C]. In Advances in Neural Information Processing Systems 15, 961-968, Cambridge, MA, USA. The MIT Press, 2003.
[83] Y. W. Teh, S. Roweis. Automatic alignment of local representations [C]. In: Advances in Neural Information Processing Systems 15. MIT Press, 2003, 15: 841-848.
[84] J. H. Ham, D. D. Lee, and L. K. Saul. Semisupervised alignment of manifolds [A]. In Z. Ghahramani and R. Cowell (eds.), Proceedings of the Tenth International Workshop on Artificial Intelligence and Statistics [C], 2005, 120-127. Barbados, West Indies
[85] M. H. C. Law, N. Zhang, A. K. Jain, Nonlinear manifold learning for data stream [C]. Proceedings of SIAM Data Mining, Florida, USA, 2004, 33-44.
[86] J. Bruske, G. Sommer, Topology representing networks for intrinsic dimensionality estimation [C]. Wulfram Gerstner, Alain Germond, Martin Hasler, et al, ICANN’97, Springer LNCS 1327, 1997, 595-600.
[87] X. Yang, H. Fu, H. Zha, et al. Semi-Supervised Nonlinear Dimensionality Reduction [C]. Proceedings of the 23rd international conference on Machine learning. ACM Press New York, NY, USA, 2006. 1065-1072.
[88]张军平.流形学习及应用[D].北京:中国科学院自动化研究所,复杂系统与智能科学重点实验室, 2003.
[89] M. Belkin, P. Niyogi. Using Manifold Structure for Partially Labeled Classification [C]. Advances in Neural Information Processing Systems, 2003, 15:929-936.
[90] J. Ye, R. Janardan, Q. Li. GPCA: an efficient dimension reduction scheme for imagecompression and retrieval [A]. In Won Kim, Ron Kohavi, Johannes Gehrke, and William DuMouchel, editors, The Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining [C], 354-363. ACM, 2004.
[91] H. Chang, D. Y. Yeung. Locally linear metric adaptation with application to semi-supervised clustering and image retrieval [J]. Pattern Recognition. 2006, 39(7): 1253-1264.
[92] R. Souvenir, R. Pless. Manifold clustering [C]. Proceedings of the International Conference on Computer Vision, 2005, 1: 648-653.
[93] A. E Seward, B. Bodenheimer. Using nonlinear dimensionality reduction in 3D figure animation [C]. Proceedings of the ACM Regional Conference, 2005, 2: 388-392.
[94] J. Banfield, A. Raftery. Ice Floe Identification in Satellite Images Using Mathematical Morphology and Clustering about Principal Curves [J]. Journal of the American Statistical Association, 1992, 87: 7-16.
[95] B. Kégl, A. Krzyzak. Piecewise Linear Skeletonization Using Principal Curves [J]. IEEE Transaction on Pattern Analysis and Machine Intelligence, 2002, 24(1): 59-74.
[96] R. D. de, O. Kouropteva, O. Okun, et al. Supervised locally linear embedding [C]. in Proc. Joint Int. Conf. ICANN/ICONIP. 2003.
[97] X. Geng, D. C. Zhan, Z.H. Zhou. Supervised nonlinear dimensionality reduction for visualization and classification [J]. IEEE Transactions on Systems, Man, and Cybernetics - Part B: Cybernetics, 2005, 35(6): 1098-1107.
[98] S. WENG, C. ZHANG, Z. LIN, Exploring the structure of supervised data by Discriminant Isometric Mapping [J]. Pattern recognition, 2005. 38(4): 599-601.
[99] H. Y. Li, L. Teng, W. Chen, et al. Supervised Learning on Local Tangent Space [C]. in: Advances in Neural Networks-ISNN2005, Lecture Notes in Computer Science, Heidelberg: Springer Berlin, 2005, 3496: 546-551.
[100] M. Belkin and P. Niyogi, Using Manifold Structure for Partially Labelled Classification [A], Advances in NIPS [C], vol. 15, 2003.
[101]陈省身,陈维桓.微分几何讲义[M].北京:北京大学出版社, 2001.
[102]陈维桓.微分流形初步[M].北京:高等教育出版社, 2001.
[103]陈维桓.流形上的微积分[M].见:萧树铁编.大学数学.北京:高等教育出版社, 2003.
[104] Michael Spivak,齐民友,路见可译.流形上的微积分——高等微积分中的一些经典定理的现代化处理(双语版)[M],北京:人民邮电出版社, 2006.
[105]施恩伟.流形上的微积分[M].北京:科学出版社, 2004.
[106] R. A. Horn, C. R. Johnson,杨奇译.矩阵分析[M],北京:机械工业出版社, 2005.
[107] J. H. Friedman, Regularized Discriminant Analysis [J]. Journal of the American StatisticalAssociation, 1989. 84(405): 165-175.
[108] P. N. Belhumeur, J.P. Hespanha, D.J. Kriegman. Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection [J]. IEEE Trans. Pattern Analysis and Machine Intelligence, 1997, 19(7): 711-720.
[109] L. F. Chen, H.Y.M. Liao, M.T. Ko, et al. A New LDA-Based Face Recognition System Which can Solve the Small Sample Size Problem [J]. Pattern Recognition, 2000, 33(10): 1713-1726.
[110]梁毅雄.基于子空间分析的人脸特征提取及识别研究[D].重庆大学研究生院, 2005.
[111] Z. Jin, J. Y. Yang, Z. S. Hu, et al. Face Recognition Based on the Uncorrelated Discriminant Transformation [J]. Pattern Recognition, 2001, 34(7): 1405-1416.
[112] M. Skuirchina, R. P. W. Duin. Bagging, Boosting, and the Random Subspace Method for Linear Classifiers [J]. Pattern Analysis and Applications, 2002, 5:121-135.
[113]赵连伟,罗四维,赵艳敞,刘蕴辉.高维数据流形的低维嵌入及嵌入维数研究[J].软件学报, 2005, 16(8): 1423-1430.
[114]王式安,数理统计[M],北京:北京理工大学出版社, 2004.
[115] J. B. Shi, J. Malik. Normalized Cuts and Image Segmentation [J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2000, 22(8): 888-905.
[116] L. Zelnik-Manor, P. Perona. Self-tuning spectral clustering [A]. In L. K. Saul, Y.Weiss, and L. Bottou, editors, Advances in Neural Information Processing Systems 17 [C], Cambridge, MA: MIT Press, 2005: 1601–1608.
[117] S. A. Nene, S. K. Nayar, H. Murase, Columbia object image library (COIL20) [R], Technical Report CUCS-005-96, Columbia University, New York, 1996.
[118] Y. LeCun, L. Bottou, Y. Bengio, et al. Gradient-based learning applied to document recognition [C]. Proceedings of the IEEE, 86(11): 2278-2324, November 1998.
[119] X. F. He, P. Niyogi. Locality Preserving Projections [C]. Advances in Neural Information Processing Systems 16, Vancouver, Canada, 2003.
[120] X. F. He. Locality Preserving Projections [D]. Illinois, American: Computer Science Department, The University of Chicago, 2005
[121] D. Cai, S. C. Yan, et al. Neighborhood preserving embedding [C]. IEEE International Conference on Computer Vision (ICCV), Beijing, China, 2005.
[122] T. H. Zhang, J. Yang, D. L. Zhao, et al. Linear Local Tangent Space Alignment and Application to Face Recognition [J]. Neurocomputing, 2007, 70(7-9): 1547-1553.
[123] D. Cai, X. F. He, J.W. Han. Spectral Regression: A Unified Subspace Learning Framework for Content-Based Image Retrieval [J]. ACM Multimedia 2007, Augsburg, Germany, Sep. 2007.
[124] S. C. Yan, D. Xu, B.Y Zhang, et al, Graph Embedding and Extension: a General Framework for Dimensionality Reduction [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2007, 29(1): 40-51.
[125] A. Tikhonov, V. Arsenin. Solution of ill-posed problems [M]. Washington: Winston & Sons, 1977.
[126] X. F. He, S. C. Yan, Y. X. Hu, et al. Face recognition using laplacianfaces [J]. IEEE Trans. Pattern Anal. Mach. Intell. 27 (3): 328-340, 2005.
[127] J. Hull, A database for handwritten text recognition research [J], Transactions on Pattern Analysis and Machine Intelligence 16 (5): 550-554, May 1994.
[128] F. S. Samaria, A. C. Harter. Parameterisation of a stochastic model for human face identification [C]. Proceedings of 2nd IEEE Workshop on Applications of Computer Vision, Sarasota FL, 1994.
[129] Yale Univ. Face database [Z]. http://cvc.yale.edu/projects/yalefaces/yalefaces.html.
[130] T. Sim, S. Baker, M. Bsat. The CMU Pose, Illumination, and Expression Database [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2003. 25(12): 1615-1618.
[131] M. Turk, A. Pentland. Eigenfaces for recognition [J]. Journal of Cognitive Neuroscience, 1991, 3 (1): 71-86.