基于模块神经网络和流形学习的模式识别中若干高复杂问题的研究
|
摘要
目前，模式识别已经发展了很多分类方法。然而，随着应用领域的不断推广和处理数据的不断庞大，模式识别系统的学习速度、分类精度和代价不断受到挑战。特别是在实际应用过程中往往会出现一些特殊的复杂问题，比如大样本集问题、训练集类别不对称问题、有类别标记样本少而无类别标记样本较多的问题等等。对于这些特殊问题，已有的模式识别方法已经不能够满足实际应用中对识别精度和学习速度的要求，因此，有必要深入探讨和研究针对这些特殊问题的特定的分类模型。本论文主要针对模块神经网络解决大样本集学习问题、训练集中类别不对称问题、以及流形学习应用于半监督分类等课题来展开全面而系统的研究。研究成果丰富了针对这些特殊的复杂问题的分类模型，提高了它们的分类效率。全文的主要工作体现在以下几个方面：
1、提出了一种基于新的任务分解技术的矩阵模块神绎网络分类系统，它将一个复杂分类任务分解为多个简单的子任务来解决，每个子任务只是在两个子空间内进行，且由一个具有简单结构的神经网络模块来完成，所有网络模块将组成一个神绎网络矩阵，最终将该神经网络矩阵的输出矩阵集成得到最终分类结果。通过理论分析和模拟实验证明，该矩阵模块神经网络能节省学刊时间，提高分类精度。
2、成功将矩阵模块神经网络应用于人脸和掌纹识别系统。对于掌纹识别问题，提出了一种有效的2DPCA(w／o3)+PCA特征抽取技术，此特征提取方法比其它特征提取方法花费较少的抽取时间，却能取得更好的分类精度。
3、提出用矩阵模块神经网络来解决非对称模式分类问题的模型结构。它将非对称模式分类问题分解为一系列对称的两类问题来解决，每个两类问题由一个结构简单的网络来解决，并且仅使用简单的网络学习算法就能够取得较好的分类结果。该矩阵模块神经网络能有效地减少非对称分类问题的学习时间，提高其分类精度。
4、提出一种改进的基于黎曼流形和最小误差类别映射的半监督学习算法，使其能直接应用于多类半监督学习问题。该改进算法在保持与原算法相同的分类精度的基础上，能够大大提高学习速度。
5、提出了一种光谱映射的改进算法——半监督光谱映射用于半监督分类，取得了较好的分类效果。该算法在映射时添加了类别信息，并且用沿着流形表而的测地距离取代了原来的欧氏距离作为样本点之间差异性的测度。该改进算法提高了映射的性能，并且取得了较好的分类结果。
At present, there have been many classification techniques well developed in pattern recognition field. However, the broadening of fields and the enlarging of sizes of datasets dealt with in real applications are challenging the learning speeds and classification accuracies of all sorts of pattern classification systems. Specially, some special complex cases such as the classification problems with large size of training set, unbalanced training set and partially labeled training set demand designing some special and more effective classification models. Therefore, it is necessary for us to thoroughly investigate the classification models so as to solve those highly complex problems. This thesis is focused on comprehensively and systemically solving the classification problems with large size of training set and imbalanced training set by using modular neural networks, as well as semi-supervised classification problems by using manifold learning. The obtained results enrich and perfect the classification models and enhance the classification performance for these complex problems. The main works in the thesis can be stated as follows:
1. A classification structure of matrix modular neural network based on a novel task decomposition technique was proposed, which can decompose a complex task into several easier subtasks between subspace pairs. Each subtask is then solved by a simple perceptron. All of these perceptron modules form a perceptron matrix structure, which produces a matrix of outputs that will be fed to an integration machine so that a classification decision can be efficiently made. This method can greatly speed up training of neural networks and obviously enhance the generalization capability for distinguishing unknown samples according to our experiments and theoretical analyses.
2. The proposed matrix modular neural network was successfully applied into face recognition and palmprint recognition. Furthermore, for palmprint recognition, a feature extraction technique called '2DPCA(w/o3)+PCA', which consumes less eatraction time but obtains better classification performance than other feature extraction techniques, was also proposed.
3. A structure of matrix modular neural network was proposed to deal with the imbalanced pattern classification problems. By this matrix modular neural network, an imbalanced classification problem can be transformed into a set of symmetrical two-class problems, each of which can be solved easily by a simple network. The experimental results showed that the matrix modular neural network could reduce the CPU consumption for the training, and also improve the classification performance.
4. A modified version to semi-supervised learning algorithm based on Riemannian manifold and mapping for minimum error sum was proposed to make it applicable to multi-classes semi-supervised learning. The modified algorithm largely increases the learning speed, and at the same time attains the satisfying classification performance, which is not lower than that of the original algorithm.
5. An improved version to spectral mapping, referred to as semi-supervised spectral mapping, was proposed to implement semi-supervised learning. This new method adds the label information into the mapping process, and adopts the geodesic distance rather than Euclidean distance as the measure of the difference between two data points. The experimental results show that the proposed method yields significant benefits for partially labeled classification with respect to the previous methods.
1、提出了一种基于新的任务分解技术的矩阵模块神绎网络分类系统，它将一个复杂分类任务分解为多个简单的子任务来解决，每个子任务只是在两个子空间内进行，且由一个具有简单结构的神经网络模块来完成，所有网络模块将组成一个神绎网络矩阵，最终将该神经网络矩阵的输出矩阵集成得到最终分类结果。通过理论分析和模拟实验证明，该矩阵模块神经网络能节省学刊时间，提高分类精度。
2、成功将矩阵模块神经网络应用于人脸和掌纹识别系统。对于掌纹识别问题，提出了一种有效的2DPCA(w／o3)+PCA特征抽取技术，此特征提取方法比其它特征提取方法花费较少的抽取时间，却能取得更好的分类精度。
3、提出用矩阵模块神经网络来解决非对称模式分类问题的模型结构。它将非对称模式分类问题分解为一系列对称的两类问题来解决，每个两类问题由一个结构简单的网络来解决，并且仅使用简单的网络学习算法就能够取得较好的分类结果。该矩阵模块神经网络能有效地减少非对称分类问题的学习时间，提高其分类精度。
4、提出一种改进的基于黎曼流形和最小误差类别映射的半监督学习算法，使其能直接应用于多类半监督学习问题。该改进算法在保持与原算法相同的分类精度的基础上，能够大大提高学习速度。
5、提出了一种光谱映射的改进算法——半监督光谱映射用于半监督分类，取得了较好的分类效果。该算法在映射时添加了类别信息，并且用沿着流形表而的测地距离取代了原来的欧氏距离作为样本点之间差异性的测度。该改进算法提高了映射的性能，并且取得了较好的分类结果。
At present, there have been many classification techniques well developed in pattern recognition field. However, the broadening of fields and the enlarging of sizes of datasets dealt with in real applications are challenging the learning speeds and classification accuracies of all sorts of pattern classification systems. Specially, some special complex cases such as the classification problems with large size of training set, unbalanced training set and partially labeled training set demand designing some special and more effective classification models. Therefore, it is necessary for us to thoroughly investigate the classification models so as to solve those highly complex problems. This thesis is focused on comprehensively and systemically solving the classification problems with large size of training set and imbalanced training set by using modular neural networks, as well as semi-supervised classification problems by using manifold learning. The obtained results enrich and perfect the classification models and enhance the classification performance for these complex problems. The main works in the thesis can be stated as follows:
1. A classification structure of matrix modular neural network based on a novel task decomposition technique was proposed, which can decompose a complex task into several easier subtasks between subspace pairs. Each subtask is then solved by a simple perceptron. All of these perceptron modules form a perceptron matrix structure, which produces a matrix of outputs that will be fed to an integration machine so that a classification decision can be efficiently made. This method can greatly speed up training of neural networks and obviously enhance the generalization capability for distinguishing unknown samples according to our experiments and theoretical analyses.
2. The proposed matrix modular neural network was successfully applied into face recognition and palmprint recognition. Furthermore, for palmprint recognition, a feature extraction technique called '2DPCA(w/o3)+PCA', which consumes less eatraction time but obtains better classification performance than other feature extraction techniques, was also proposed.
3. A structure of matrix modular neural network was proposed to deal with the imbalanced pattern classification problems. By this matrix modular neural network, an imbalanced classification problem can be transformed into a set of symmetrical two-class problems, each of which can be solved easily by a simple network. The experimental results showed that the matrix modular neural network could reduce the CPU consumption for the training, and also improve the classification performance.
4. A modified version to semi-supervised learning algorithm based on Riemannian manifold and mapping for minimum error sum was proposed to make it applicable to multi-classes semi-supervised learning. The modified algorithm largely increases the learning speed, and at the same time attains the satisfying classification performance, which is not lower than that of the original algorithm.
5. An improved version to spectral mapping, referred to as semi-supervised spectral mapping, was proposed to implement semi-supervised learning. This new method adds the label information into the mapping process, and adopts the geodesic distance rather than Euclidean distance as the measure of the difference between two data points. The experimental results show that the proposed method yields significant benefits for partially labeled classification with respect to the previous methods.
引文
[1] R. Bajcsy and S. Kovaclc, "Multiresolution elastic matching," Computer Vision Graphics Image Processing, Vol. 46, pp. 1-21, 1989.
[2] L. Devroye, L. Gyorfi and G. Lugosi, A probabilistic theory of pattern recognition, Berlin: Springer-Verlag, 1996.
[3] R. O. Duda and P. B. Hart, Pattern Classification and Scene Analysis, New York: John Wiley & Sons, 1973.
[4] K. S. Fu, Syntactic Pattern Recognition and Applications, Englewood Cliffs, N.J.: Prentice-Hall, 1982.
[5] F. Rosenblatt, "The perceptron: A perceiving and recognizing automation (project PARA)," Cornell Aeronautical Laboratory Report, 85-406-Ⅰ, 1959.
[6] 黄德双，《神经网络模式识别系统理论》，电子工业出版社，1996年5月。
[7] W. W. McCulloch and W. Pitts "A logic calculus of the ideas imminent in neurons activity," Bulletin of Mathematical Biophysics, Vol. 5, pp. 115-133, 1943.
[8] P. J. Werbos, "Beyond regression; new tool for prediction and analysis in the behavior science," Ph.D. Thesis, Harvard University, 1975.
[9] J. J. Hopfield, "Neural networks and physical systems with emergent collective computational abilities," Proc. Acad. Sci., U.S.A, Vol. 79, pp. 2554-2558, 1982.
[10] D. E. Rumelhart, G. E. Hinton and R. J. Williams, "Learning representations by back-propagating errors," Nature, Vol. 323, pp. 533-535, 1986.
[11] J. E. Moody and C. J. Darken, "Fast learning in networks &locally tuned processing units," Neural Computation, Vol. 1, pp. 281-293, 1989.
[12] R.D. Short and K. Fukunaga, "The optimal distance measure for nearest neighbor classification," IEEE Trans. Inf Theory, Vol. 27(5), pp. 622-627, 1981.
[13] K. Fukunaga and T.E. Flick, "An optimal global nearest neighbor metric," IEEE Trans. PAMI., Vol. 6(3), pp. 314-318, 1984.
[14] M. R. Azimi-Sadjadi and Ren-Jean Lion, "Fast learning process of multilayer Neural Networks using recursive least squares method," IEEE Trans. Signal Processing, Vol. 40(2), pp.446-450, 1992.
[15] R. Battiti, "Accelerated backpropagation learning: two optimization methods," Complex Systems, Vol. 3, pp. 331-342, 1989.
[16] E. M. Johansson, F. U. Dowla and D. M. Goodman, "Backpropagation learning for multi-layer feed-forward neural networks using the conjugate gradient method," International Journal of Neural Systems, Vol. 2(4), pp. 291-302, 1991.
[17] M. F. Moller, "A scaled conjugate gradient algorithm for fast supervised learning," Neural Networks, Vol.6, pp. 525-533, 1993.
[18] B. Efron and R. Tibshirani, An introduction to the Bootstrap, Chapman and Hall, 1993.
[19] L. Breiman, Heuristics of instability in model selection. Technical Report, Statistics Department, University of California at Berkeley, 1994.
[20] Y. Freund and R.E. Schapire, "Experiments with a new boosting algorithm", In International Conference on Machine Learning, pp. 148-156, 1996.
[21] Y. Freund and R.E. Schapire, "A decision-theoretic generalization of on-line learning and an application to boosting", J. Comput. Syst. Sci. Vol. 55(1), pp. 119-139, 1997.
[22] N. J. Nilsson, Learning Machines." Foundations of Trainable Pattern-Classifying Systems. New York: McGraw-Hill, 1965.
[23] R. Anand, K. Mehrotra and C.K. Mohan etc, "Efficient classification for multiclass problems using modular neural networks," IEEE Trans. Neural Networks, Vol. 6(1), pp. 117-124, 1995.
[24] R. Anand, K. G. Mehrotra, C. K. Mohan and S. Ranka, "An improved algorithm for neural network classification of imbalanced training sets," IEEE Trans. Neural Networks, Vol. 4(6), pp. 962-969, 1993.
[25] P. Foster, "Machine learning from imbalanced data sets 101," Invited paper for the AAAI'2000 Workshop on Imbalanced Data Sets, 2000.
[26] Z. H. Zhou and X. Y. Liu, "Training cost-sensitive neural networks with methods addressing the class imbalance problem," IEEE Trans. on Knowledge and Data Engineering, Vol. 18(1), 63-77, 2006.
[27] R. Liu, J. Z. Zhou and M. Liu, "A graph-based semi-supervised learning algorithm for web page classification," Proceedings of the Sixth International Conference on Intelligent Systems Design and Applications (ISDA'06), 2006.
[28] K. Nigam, et al, "Text classification from labeled and unlabeled documents using EM, " Machine Learning, Vol. 39(2/3), pp.103-134, 2000.
[29] 鲁珂，赵继东，叶娅兰，曾家智，“一种用于图像检索的新型半监督学习算法，”《电子科技大学学报》，卷34(5)，第669-671页，2000年．
[30] M. F. Balcan, A. Blum, P. P. Choi, J. Lafferty, B. Pantano, M. R. Rwebangira and X. Zhu, "Person identification in webcam images: An application of semi-supervised learning," ICML2005 Workshop on Learning with Partially Classified Training Data.
[31] K. Nigam, A. K. McCallum, S. Thrun and T. Mitchell, "Text classification from labeled and unlabeled documents using EM," Machine Learning, Vol. 39, pp. 103-134.2000.
[32] S. Baluja, "Probabilistic modeling for face orientation discrimination: Learning from labeled and unlabeled data," Neural Information Processing Systems, pp. 854-860, 1998.
[33] C. Rosenberg, M. Hebert and H. Schneiderman, "Semi-supervised self-training of object detection models," Seventh IEEE Workshop on Applications of Computer Vision, 2005.
[34] D. Yarowsky, "Unsupervised word sense disambiguation rivaling supervised methods," Proceedings of the 33rd Annual Meeting of the Association for Computational Linguistics, pp. 189-196, 1995.
[35] A. Blum and T. Mitchell, "Combining labeled and unlabeled data with co-training," COLT: Proceedings of the Workshop on Computational Learning Theory, 1998.
[36] Y. Joachims, "Transductive inference for text classification using support vector machines," Proc. 16th International Conf. on Machine Learning Morgan Kaufmann, San Francisco, CA, pp. 200-209, 1999.
[37] D.Y. Zhou, O, Bousquet, T.N. Lal, J. Weston and B. Scholkopf. Learning with local and global consistency, Max Planck Institute for Biological Cybernetics Technical Report, 2003.
[38] W. Du, K. Inoue, and K. Urahama, "Dimensionality reduction for semi-supervised face recognition," Lecture notes in computer science, Vol.3614, 1-10, 2005.
[39] M. Belkin and P. Niyogi, "Semi-supervised learning on Riemannian manifolds," Machine Learning, Vol. 56, 209-239, 2004.
[40] X. Zhu, Z. Ghahramani and J. Lafferty, "Semi-supervised learning using Gaussian field and harmonic functions," Proceeding of the 20th International Conference on Machine Learning (ICML-2003), Washington DC, 2003.
[41] A. Demiriz, K. Bennett and M. Embrechts, "Semi-supervised clustering using genetic algorithms," Proceedings of Artificial Neural Networks in Engineering, 1999.
[42] R. Dara, S. Kremer and D. Stacey, "Clsutering unlabeled data with SOMs improves classification of labeled real-world data," Proceedings of the World Congress on Computational Intelligence (WCCI), 2002.
[43] M. Belkin and P. Niyogi, "Laplacian eigenmaps for dimensionality reduction and data representation," Neural Computation, Vol. 15(6), pp. 1373-1396, 2003.
[1] E. Alpaydin, "Multiple networks for function learning," International Conference on Neural networks, USA, Vol. 1, pp. 9-14, 1993.
[2] L. Xu, A. Krzyzak and C. Y. Suen, "Methods of combining multiple classifiers and their applications to handwriting recognition," IEEE Trans. System, Man, and Cybernetics, Vol. 22(3),pp. 418-433, 1992.
[3] V. D. Mazurov, A. I. Krivonogov and V. L. Kazantsev, "Solving of optimization and identification problems by the committee methods," Pattern Recognition, Vol. 20(4), pp. 371-378,1987.
[4] R. Battiti and A. Colla, "Democracy in neural nets: voting schemes for classification," Neural Networks, Vol. 7(4), pp. 691-707, 1994.
[5] D. Black, The Theory of Committees and Elections, 2" ed. London: Cambridge University Press, 1963.
[6] M. F. Moller, "A scaled Conjugate Gradient algorithm for fast supervised learning," Neural Networks, Vol.6, pp. 525-533, 1993.
[7] J. X. Sun, Modern Pattern Recognition, China: National University of Defense Technology Publishers, 2002.
[8] J. Kittler, M. Hatef, R. P. W. Duin and J. Matas, "On combining classifier," IEEE Trans.Pattern Analysis and Machine Intelligence, Vol. 20, pp. 226-239, 1998.
[9] R. A. Fisher, "The use of multiple measurements in taxonomic problems," Annals of Eugenics,Vol. 7, pp. 179-188, 1936.
[10] Z. H. Zhou, S. F. Chen and Z. Q. Chen, "FANNC: A fast adaptive neural network classifier",Knowledge and Information Systems, pp. 115-129, 2000.
[11] E. Backer, Computer-assisted Reasoning in Cluster Analysis, New York: Prentice Hall, 1995.
[12] O. D. Richard, E. H. Peter and G. S. David, Pattern Classification, New York: Wiley, 2001.
[1] X. Jia and M.S. Nixon, "Extending the feature vector for automatic face recognition" IEEE Transaction on Pattern Analysis and Machine Intelligence, Vol. 17(12), pp. 1167-1176,1995.
[2] N. Roder and X. Li, Pattern Recognition, Vol. 29, pp.143-157, 1996.
[3] D.S Penev and J. J. Atick, Network Computation & Neural Network, Vol.7, pp.477-490, 1996.
[4] B. Moghaddam and A. Pentland, "Probabilistic visual learning for object representation," IEEE Transaction on Pattern Analysis and Machine Intelligence, Vol. 19(7), pp.696-710, 1997.
[5] M.A. Turk and A. R Pentland, CVPR'91,586-591.
[6] M. Turk and A. Pentland, "Eigenfaces for Recognition," J. Cognitive Neuroscience, Vol.3(1), 1991.
[7] M. Turk and A Pentland, "Face recognition using eigenfaces," Proc. IEEE Conf. On Computer Vision and Pattern Recognition, pp.586-591, 1991.
[8] P, Hallinan, "A low-dimensional representation of human faces for arbitrary lighting conditions," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp.995-999, 1994.
[9] H. Murase and S. Nayar, "Visual learning and recognition of 3-D objects from appearance," Int'l J. Computer Vision, Vol. 14, 5-24.
[10] R. Chellappa, C. Wilson, and S. Sirohey, "Human and machine recognition of faces: A survey." Proc. IEEE, Vol. 83(5), pp.705-740, 1995.
[11] R. A. Fisher, "The use of multiple measures in taxonomic problems, "Ann. Eugenics, Vol.7, pp.179-188, 1936.
[12] M. F. Moller, "A scaled conjugate gradient algorithm for fast supervised learning," Neural Networks, Vol.6, pp. 525-533, 1993.
[13] D. Zhang and W. Shu, "Two Novel Characteristics in Palmprint Verification: Datum Point Invariance and Line Feature Matching", Pattern Recognition, Vol. 32(4), pp. 691-702, 1999.
[14] W. Li, D. Zhang and Z. Xu, "Palmprint identification by fourier transform", International Journal of Pattern Recognition and Artificial Intelligence, Vol. 16(4), pp. 417-432, 2002.
[15] G.M. Lu, D. Zhang and K. Wang, "Palmprint recognition using eigenpalms features", Pattern Recognition Letters, Vol. 24, no. 9-10, pp. 1463-1467, 2003.
[16] D. Zhang, W. Kong, J. You and M. Wong, "On-line palmprint identification", IEEE Trans. on PAMI, Vol. 25(9), pp. 1041-1050, 2003.
[17] 黄德双，神经网络模式识别系统理论，电子工业出版社，北京，1996。
[18] X.Q. Wu, D. Zhang and K.Q Wang, "Fisherpalms based palmprint recognition," Pattern Recognition Letter, Vol. 24(15), pp.2829-2838, 2003.
[19] J. Yang, D. Zhang, A.E Frangi and J.Y. Yang, "Two-dimensional PCA: A new approach to appearance-based face representation and recognition", IEEE Trans. PAMI, Vol.26(1), pp. 1-7,2004.
[20] P.N. Belhumeur, J,P. Hespanha and D.J. Kriengman, "Eigenfaces vs. Fisherfaces:Recognition using class specific linear projection", IEEE Trans. PAMI, vol. 19, no. 7, pp. 711-720,1997.
[21] PolyU Palmprint Database, http://www.comp.polyu.edu.hk/～biometrics/
[22] D.E. Rumelhart, G.E. Hinton and R.J. Williams, "Learning representations by back-propagating errors," Nature, Vol. 323, pp. 533-535, 1986.
[1] G.E. Cook, R.J. Barnett, et al., "Modeling and control using artificial neural networks," IEEE Trans. Industry Applications, Vol. 31, pp. 1484-1491, 1995.
[2] G. van Schoor, J.D. van Wyk and I.S. Shaw, "Optimal control of a hybrid power compensator using an artificial neural network controller," IEEE Trans. Industry Applications, Vol. 38, pp.467-475, 2002.
[3] P.J. Werbos, Beyond regression. New tools for predictions and analysis in the behavioral science, Ph D Thesis, Harvard University, 1974.
[4] D. E. Rumelhart, G. E. Hinton and R. J. Williams, "Learning representations by back-propagating errors," Nature, Vol. 323, pp. 533-535, 1986.
[5] J. Kowalik and M. R. Osborne, Methods for Unconstrained Optimization Problems. American Elsevier, 1968.
[6] R. Anand, K. G. Mehrotra, C. K. Mohan, and S. Ranka, " An improved algorithm for neural network classification of imbalanced training Sets," IEEE Trans. Neural Networks, Vol. 4(6), pp.962-969, 1993.
[7] D. T. Phamand and D. Karaboga, Intelligent Optimization Techniques, Genetic Algorithms, Tabu Search, Simulated Annealing and Neural Networks. New York: Springer-Verlag, 2000.
[8] Z. Michalewicz, Genetic Algorithm + Data Structures = Evolution Pro-grams, 2~(nd) extended. New York: Springer-Verlag, 1994.
[9] B. D. Liu, C. Y. Chert, and J. Y. Tsao, "Design of adaptive fuzzy logic controller based on linguistic-hedge concepts and genetic algorithms," IEEE Trans. Syst., Man, Cybern. B, Vol. 31, pp. 32-53, 2001.
[10] E. Alpaydin, "Multiple networks for function learning," International Conference on Neural networks, USA, Vol. 1, pp. 9-14, 1993.
[11] L. Xu, A. Krzyzak and C. Y. Suen, "Methods of Combining Multiple Classifiers and Their Applications to Handwriting Recognition," IEEE Trans. System, Man, and Cybernetics, Vol. 22(3), pp. 418-433, 1992.
[12] 陈国良，王煦法，庄镇泉，王东升，遗传算法及其应用．北京：人民邮电出版社，1996．
[13] 张文修，梁怡，遗传算法的数学基础．西安：西安交通大学出版社，2000．
[14] R. A. Fisher, "The use of multiple measurements in taxonomic problems," Annals of Eugenics, Vol. 7, pp. 179-188, 1936.
[15] P. Simard, Y. Le Curt, J. Denker, and B. Victorri, "Transformation invariance in pattern recognition -- Tangent distance and tangent propagation," Neural networks." tricks of the trade, volume 1524 of Lecture Notes in Computer Science, Springer, Heidelberg, pp. 239-274, 1998.
[16] P. Simard, Y. Le Curt, and J. Denker, "Efficient pattern recognition using a new transformation distance," Advances in Neural lnf Proe. Systems, volume 5, Morgan Kaufmnann, San Mateo CA, pp. 50-58, 1993.
[17] B. Scholkopf, Support Vector Learning, Ph.D. Thesis, Berlin, 1997.
[18] R.A. Jacobs, "Increased rates of convergence through learning rate adaptation," Neural Networks, Vol. 1, pp. 295-307, 1988.
[19] T. Tollenaere, "SuperSAB: Fast adaptive back propagation with good scaling properties," Neural Networks, Vol. 3, pp. 561-573, 1990.
[20] Y. S. Zhou and L. Y. Lai, "Optimal design for fuzzy controllers by genetic algorithms," IEEE Trans. Ind. Applicat., Vol. 36, pp. 93-97, 2000.
[21] C. F. Juang, J. Y. Lin, and C. Y. Lin, "Genetic reinforcement learning through symbiotic evolution for fuzzy controller design," IEEE Trans. Syst., Man. Cybern. B, Vol. 30, pp. 290-302,2000.
[22] K. Belarbi and F. Titel, "Genetic algorithm for the design of a class of fuzzy controllers: An alternative approach," IEEE Trans. Fuzzy Systems, Vol. 8, pp. 398-405, 2000. [23] H. Juidette and H. Youlal, "Fuzzy dynamic path planning using genetic algorithms," Electron. Lett., Vol. 36(4), pp. 374-376, 2000.
[24] M. Setnes and H. Roubos, "GA-fuzzy modeling and classification: Complexity and performance," IEEE Trans. Fuzzy Syst., Vol. 8, pp. 509-522, 2000.
[25] D. E. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning, Addison Wesley, 1989.
[1] A. Dempster, N. Laird and D. Rubin, "Maximum likelihood from incomplete data via the EM algorithm," Journal of the Royal Statistical Society B, Vol. 39, pp. 1-38, 1977.
[2] M. Seeger, "Learning with labeled and unlabeled data," Technical report, University of Edinburgh, 2000.
[3] W. Du, K. Inoue, and K. Urahama, "Dimensionality reduction for semi-supervised face recognition," Lecture notes in computer science, Vol. 3614, pp. 1-10,2005.
[4] Herve Abdi, "Multivariate Analysis," Encyclopedia of Social Sciences Research Methods, Thousand Oaks (CA): Sage, 2003.
[5] M. Belkin and P. Niyogi, "Laplacian eigenmaps for dimensionality reduction and data representation," Neural Computation, Vol. 15(6), 1373-1396, 2003.
[6] A. Y. Ng, M. I. Jordan and Y. Weiss, "On spectral clustering: Analysis and an algorithm, " Proc. NIPS' 01, pp. 849-856,2001.
[7] M. Turk and A. Petland, "Face recognition using eigenfaces", J. cognitive Neurosci., Vol.3, pp. 71-86, 1991.
[8] D.Y. Zhou, O. Bousquet, T.N. Lal, J. Weston and B. Scholkopf, "Learning with local and global consistency," Max Planck Institute for Biological Cybernetics Technical Report, 2003.
[9] X. Zhu, Z. Ghahramani and J. Lafferty, "Semi-supervised learning using Gaussian field and harmonic functions," Proceeding of the 20th International Conference on Machine Learning (ICML-2003), Washington DC, 2003.
[10] M. Belkin and R Niyogi, "Semi-supervised learning on Riemannian manifolds," Machine Learning, Vol. 56, pp. 209-239, 2004.
[11] R. A. Horn and C.R. Johnson, Matrix Analysis, Cambridge Univ. Press, Cambridge, 1990.
[1] M. Belkin and P. Niyogi, "Laplacian eigenmaps for dimensionality reduction and data representation," Neural Computation, Vol. 15(6), pp. 1373-1396, 2003.
[2] M. Belkin and P. Niyogi, "Semi-supervised learning on Riemannian manifolds," Machine Learning, Vol. 56, pp. 209-239, 2004.
[3] W. Du, K. Inoue and K. Urahama, "Dimensionality reduction for semi-supervised face recognition," Lecture notes in computer science, Vol. 3614, pp. 1-10,2005.
[4] D.Y. Zhou, O. Bousquet, T.N. Lal, J. Weston and B. Scholkopf, "Learning with local and global consistency," Max Planck lnstitute for Biological Cybernetics Technical Report, 2003.
[5] X. Zhu, Z. Ghahramani and J. Lafferty, "Semi-supervised learning using Gaussian field and harmonic functions," Proceeding of the 20th International Conference on Machine Learning (ICML-2003), Washington DC, 2003.
[6] R. A. Horn and C.R. Johnson, Matrix Analysis, Cambridge Univ. Press, Cambridge, 1990.
[7] Xin Geng, De-Chuan Zhan and Zhi-Hua Zhou, "Supervised Nonlinear Dimensionality Reduction for Visualization and Classification," IEEE Trans. on Systems, Man, and Cybernetics—PART B: Cybernetics, Vol. 35(6), pp. 1098-1106, 2005.
[8] M. Turk and A. Petland, "Face recognition using eigenfaces," J. cognitive Neurosci., Vol.3, pp. 71-86, 1991.
[9] M. J. Er, S. Wu, J. Lu and H. L. Toh, "Face Recognition With Radial Basis Function (RBF) Neural Networks," IEEE Trans. Neural Networks, Vol. 13, No.3, pp. 697-710, 2002.
[10] S. Rosenberg, The Laplacian on a Riemannian Manifold, Cambridge University Press, 1997.
[2] L. Devroye, L. Gyorfi and G. Lugosi, A probabilistic theory of pattern recognition, Berlin: Springer-Verlag, 1996.
[3] R. O. Duda and P. B. Hart, Pattern Classification and Scene Analysis, New York: John Wiley & Sons, 1973.
[4] K. S. Fu, Syntactic Pattern Recognition and Applications, Englewood Cliffs, N.J.: Prentice-Hall, 1982.
[5] F. Rosenblatt, "The perceptron: A perceiving and recognizing automation (project PARA)," Cornell Aeronautical Laboratory Report, 85-406-Ⅰ, 1959.
[6] 黄德双，《神经网络模式识别系统理论》，电子工业出版社，1996年5月。
[7] W. W. McCulloch and W. Pitts "A logic calculus of the ideas imminent in neurons activity," Bulletin of Mathematical Biophysics, Vol. 5, pp. 115-133, 1943.
[8] P. J. Werbos, "Beyond regression; new tool for prediction and analysis in the behavior science," Ph.D. Thesis, Harvard University, 1975.
[9] J. J. Hopfield, "Neural networks and physical systems with emergent collective computational abilities," Proc. Acad. Sci., U.S.A, Vol. 79, pp. 2554-2558, 1982.
[10] D. E. Rumelhart, G. E. Hinton and R. J. Williams, "Learning representations by back-propagating errors," Nature, Vol. 323, pp. 533-535, 1986.
[11] J. E. Moody and C. J. Darken, "Fast learning in networks &locally tuned processing units," Neural Computation, Vol. 1, pp. 281-293, 1989.
[12] R.D. Short and K. Fukunaga, "The optimal distance measure for nearest neighbor classification," IEEE Trans. Inf Theory, Vol. 27(5), pp. 622-627, 1981.
[13] K. Fukunaga and T.E. Flick, "An optimal global nearest neighbor metric," IEEE Trans. PAMI., Vol. 6(3), pp. 314-318, 1984.
[14] M. R. Azimi-Sadjadi and Ren-Jean Lion, "Fast learning process of multilayer Neural Networks using recursive least squares method," IEEE Trans. Signal Processing, Vol. 40(2), pp.446-450, 1992.
[15] R. Battiti, "Accelerated backpropagation learning: two optimization methods," Complex Systems, Vol. 3, pp. 331-342, 1989.
[16] E. M. Johansson, F. U. Dowla and D. M. Goodman, "Backpropagation learning for multi-layer feed-forward neural networks using the conjugate gradient method," International Journal of Neural Systems, Vol. 2(4), pp. 291-302, 1991.
[17] M. F. Moller, "A scaled conjugate gradient algorithm for fast supervised learning," Neural Networks, Vol.6, pp. 525-533, 1993.
[18] B. Efron and R. Tibshirani, An introduction to the Bootstrap, Chapman and Hall, 1993.
[19] L. Breiman, Heuristics of instability in model selection. Technical Report, Statistics Department, University of California at Berkeley, 1994.
[20] Y. Freund and R.E. Schapire, "Experiments with a new boosting algorithm", In International Conference on Machine Learning, pp. 148-156, 1996.
[21] Y. Freund and R.E. Schapire, "A decision-theoretic generalization of on-line learning and an application to boosting", J. Comput. Syst. Sci. Vol. 55(1), pp. 119-139, 1997.
[22] N. J. Nilsson, Learning Machines." Foundations of Trainable Pattern-Classifying Systems. New York: McGraw-Hill, 1965.
[23] R. Anand, K. Mehrotra and C.K. Mohan etc, "Efficient classification for multiclass problems using modular neural networks," IEEE Trans. Neural Networks, Vol. 6(1), pp. 117-124, 1995.
[24] R. Anand, K. G. Mehrotra, C. K. Mohan and S. Ranka, "An improved algorithm for neural network classification of imbalanced training sets," IEEE Trans. Neural Networks, Vol. 4(6), pp. 962-969, 1993.
[25] P. Foster, "Machine learning from imbalanced data sets 101," Invited paper for the AAAI'2000 Workshop on Imbalanced Data Sets, 2000.
[26] Z. H. Zhou and X. Y. Liu, "Training cost-sensitive neural networks with methods addressing the class imbalance problem," IEEE Trans. on Knowledge and Data Engineering, Vol. 18(1), 63-77, 2006.
[27] R. Liu, J. Z. Zhou and M. Liu, "A graph-based semi-supervised learning algorithm for web page classification," Proceedings of the Sixth International Conference on Intelligent Systems Design and Applications (ISDA'06), 2006.
[28] K. Nigam, et al, "Text classification from labeled and unlabeled documents using EM, " Machine Learning, Vol. 39(2/3), pp.103-134, 2000.
[29] 鲁珂，赵继东，叶娅兰，曾家智，“一种用于图像检索的新型半监督学习算法，”《电子科技大学学报》，卷34(5)，第669-671页，2000年．
[30] M. F. Balcan, A. Blum, P. P. Choi, J. Lafferty, B. Pantano, M. R. Rwebangira and X. Zhu, "Person identification in webcam images: An application of semi-supervised learning," ICML2005 Workshop on Learning with Partially Classified Training Data.
[31] K. Nigam, A. K. McCallum, S. Thrun and T. Mitchell, "Text classification from labeled and unlabeled documents using EM," Machine Learning, Vol. 39, pp. 103-134.2000.
[32] S. Baluja, "Probabilistic modeling for face orientation discrimination: Learning from labeled and unlabeled data," Neural Information Processing Systems, pp. 854-860, 1998.
[33] C. Rosenberg, M. Hebert and H. Schneiderman, "Semi-supervised self-training of object detection models," Seventh IEEE Workshop on Applications of Computer Vision, 2005.
[34] D. Yarowsky, "Unsupervised word sense disambiguation rivaling supervised methods," Proceedings of the 33rd Annual Meeting of the Association for Computational Linguistics, pp. 189-196, 1995.
[35] A. Blum and T. Mitchell, "Combining labeled and unlabeled data with co-training," COLT: Proceedings of the Workshop on Computational Learning Theory, 1998.
[36] Y. Joachims, "Transductive inference for text classification using support vector machines," Proc. 16th International Conf. on Machine Learning Morgan Kaufmann, San Francisco, CA, pp. 200-209, 1999.
[37] D.Y. Zhou, O, Bousquet, T.N. Lal, J. Weston and B. Scholkopf. Learning with local and global consistency, Max Planck Institute for Biological Cybernetics Technical Report, 2003.
[38] W. Du, K. Inoue, and K. Urahama, "Dimensionality reduction for semi-supervised face recognition," Lecture notes in computer science, Vol.3614, 1-10, 2005.
[39] M. Belkin and P. Niyogi, "Semi-supervised learning on Riemannian manifolds," Machine Learning, Vol. 56, 209-239, 2004.
[40] X. Zhu, Z. Ghahramani and J. Lafferty, "Semi-supervised learning using Gaussian field and harmonic functions," Proceeding of the 20th International Conference on Machine Learning (ICML-2003), Washington DC, 2003.
[41] A. Demiriz, K. Bennett and M. Embrechts, "Semi-supervised clustering using genetic algorithms," Proceedings of Artificial Neural Networks in Engineering, 1999.
[42] R. Dara, S. Kremer and D. Stacey, "Clsutering unlabeled data with SOMs improves classification of labeled real-world data," Proceedings of the World Congress on Computational Intelligence (WCCI), 2002.
[43] M. Belkin and P. Niyogi, "Laplacian eigenmaps for dimensionality reduction and data representation," Neural Computation, Vol. 15(6), pp. 1373-1396, 2003.
[1] E. Alpaydin, "Multiple networks for function learning," International Conference on Neural networks, USA, Vol. 1, pp. 9-14, 1993.
[2] L. Xu, A. Krzyzak and C. Y. Suen, "Methods of combining multiple classifiers and their applications to handwriting recognition," IEEE Trans. System, Man, and Cybernetics, Vol. 22(3),pp. 418-433, 1992.
[3] V. D. Mazurov, A. I. Krivonogov and V. L. Kazantsev, "Solving of optimization and identification problems by the committee methods," Pattern Recognition, Vol. 20(4), pp. 371-378,1987.
[4] R. Battiti and A. Colla, "Democracy in neural nets: voting schemes for classification," Neural Networks, Vol. 7(4), pp. 691-707, 1994.
[5] D. Black, The Theory of Committees and Elections, 2" ed. London: Cambridge University Press, 1963.
[6] M. F. Moller, "A scaled Conjugate Gradient algorithm for fast supervised learning," Neural Networks, Vol.6, pp. 525-533, 1993.
[7] J. X. Sun, Modern Pattern Recognition, China: National University of Defense Technology Publishers, 2002.
[8] J. Kittler, M. Hatef, R. P. W. Duin and J. Matas, "On combining classifier," IEEE Trans.Pattern Analysis and Machine Intelligence, Vol. 20, pp. 226-239, 1998.
[9] R. A. Fisher, "The use of multiple measurements in taxonomic problems," Annals of Eugenics,Vol. 7, pp. 179-188, 1936.
[10] Z. H. Zhou, S. F. Chen and Z. Q. Chen, "FANNC: A fast adaptive neural network classifier",Knowledge and Information Systems, pp. 115-129, 2000.
[11] E. Backer, Computer-assisted Reasoning in Cluster Analysis, New York: Prentice Hall, 1995.
[12] O. D. Richard, E. H. Peter and G. S. David, Pattern Classification, New York: Wiley, 2001.
[1] X. Jia and M.S. Nixon, "Extending the feature vector for automatic face recognition" IEEE Transaction on Pattern Analysis and Machine Intelligence, Vol. 17(12), pp. 1167-1176,1995.
[2] N. Roder and X. Li, Pattern Recognition, Vol. 29, pp.143-157, 1996.
[3] D.S Penev and J. J. Atick, Network Computation & Neural Network, Vol.7, pp.477-490, 1996.
[4] B. Moghaddam and A. Pentland, "Probabilistic visual learning for object representation," IEEE Transaction on Pattern Analysis and Machine Intelligence, Vol. 19(7), pp.696-710, 1997.
[5] M.A. Turk and A. R Pentland, CVPR'91,586-591.
[6] M. Turk and A. Pentland, "Eigenfaces for Recognition," J. Cognitive Neuroscience, Vol.3(1), 1991.
[7] M. Turk and A Pentland, "Face recognition using eigenfaces," Proc. IEEE Conf. On Computer Vision and Pattern Recognition, pp.586-591, 1991.
[8] P, Hallinan, "A low-dimensional representation of human faces for arbitrary lighting conditions," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp.995-999, 1994.
[9] H. Murase and S. Nayar, "Visual learning and recognition of 3-D objects from appearance," Int'l J. Computer Vision, Vol. 14, 5-24.
[10] R. Chellappa, C. Wilson, and S. Sirohey, "Human and machine recognition of faces: A survey." Proc. IEEE, Vol. 83(5), pp.705-740, 1995.
[11] R. A. Fisher, "The use of multiple measures in taxonomic problems, "Ann. Eugenics, Vol.7, pp.179-188, 1936.
[12] M. F. Moller, "A scaled conjugate gradient algorithm for fast supervised learning," Neural Networks, Vol.6, pp. 525-533, 1993.
[13] D. Zhang and W. Shu, "Two Novel Characteristics in Palmprint Verification: Datum Point Invariance and Line Feature Matching", Pattern Recognition, Vol. 32(4), pp. 691-702, 1999.
[14] W. Li, D. Zhang and Z. Xu, "Palmprint identification by fourier transform", International Journal of Pattern Recognition and Artificial Intelligence, Vol. 16(4), pp. 417-432, 2002.
[15] G.M. Lu, D. Zhang and K. Wang, "Palmprint recognition using eigenpalms features", Pattern Recognition Letters, Vol. 24, no. 9-10, pp. 1463-1467, 2003.
[16] D. Zhang, W. Kong, J. You and M. Wong, "On-line palmprint identification", IEEE Trans. on PAMI, Vol. 25(9), pp. 1041-1050, 2003.
[17] 黄德双，神经网络模式识别系统理论，电子工业出版社，北京，1996。
[18] X.Q. Wu, D. Zhang and K.Q Wang, "Fisherpalms based palmprint recognition," Pattern Recognition Letter, Vol. 24(15), pp.2829-2838, 2003.
[19] J. Yang, D. Zhang, A.E Frangi and J.Y. Yang, "Two-dimensional PCA: A new approach to appearance-based face representation and recognition", IEEE Trans. PAMI, Vol.26(1), pp. 1-7,2004.
[20] P.N. Belhumeur, J,P. Hespanha and D.J. Kriengman, "Eigenfaces vs. Fisherfaces:Recognition using class specific linear projection", IEEE Trans. PAMI, vol. 19, no. 7, pp. 711-720,1997.
[21] PolyU Palmprint Database, http://www.comp.polyu.edu.hk/～biometrics/
[22] D.E. Rumelhart, G.E. Hinton and R.J. Williams, "Learning representations by back-propagating errors," Nature, Vol. 323, pp. 533-535, 1986.
[1] G.E. Cook, R.J. Barnett, et al., "Modeling and control using artificial neural networks," IEEE Trans. Industry Applications, Vol. 31, pp. 1484-1491, 1995.
[2] G. van Schoor, J.D. van Wyk and I.S. Shaw, "Optimal control of a hybrid power compensator using an artificial neural network controller," IEEE Trans. Industry Applications, Vol. 38, pp.467-475, 2002.
[3] P.J. Werbos, Beyond regression. New tools for predictions and analysis in the behavioral science, Ph D Thesis, Harvard University, 1974.
[4] D. E. Rumelhart, G. E. Hinton and R. J. Williams, "Learning representations by back-propagating errors," Nature, Vol. 323, pp. 533-535, 1986.
[5] J. Kowalik and M. R. Osborne, Methods for Unconstrained Optimization Problems. American Elsevier, 1968.
[6] R. Anand, K. G. Mehrotra, C. K. Mohan, and S. Ranka, " An improved algorithm for neural network classification of imbalanced training Sets," IEEE Trans. Neural Networks, Vol. 4(6), pp.962-969, 1993.
[7] D. T. Phamand and D. Karaboga, Intelligent Optimization Techniques, Genetic Algorithms, Tabu Search, Simulated Annealing and Neural Networks. New York: Springer-Verlag, 2000.
[8] Z. Michalewicz, Genetic Algorithm + Data Structures = Evolution Pro-grams, 2~(nd) extended. New York: Springer-Verlag, 1994.
[9] B. D. Liu, C. Y. Chert, and J. Y. Tsao, "Design of adaptive fuzzy logic controller based on linguistic-hedge concepts and genetic algorithms," IEEE Trans. Syst., Man, Cybern. B, Vol. 31, pp. 32-53, 2001.
[10] E. Alpaydin, "Multiple networks for function learning," International Conference on Neural networks, USA, Vol. 1, pp. 9-14, 1993.
[11] L. Xu, A. Krzyzak and C. Y. Suen, "Methods of Combining Multiple Classifiers and Their Applications to Handwriting Recognition," IEEE Trans. System, Man, and Cybernetics, Vol. 22(3), pp. 418-433, 1992.
[12] 陈国良，王煦法，庄镇泉，王东升，遗传算法及其应用．北京：人民邮电出版社，1996．
[13] 张文修，梁怡，遗传算法的数学基础．西安：西安交通大学出版社，2000．
[14] R. A. Fisher, "The use of multiple measurements in taxonomic problems," Annals of Eugenics, Vol. 7, pp. 179-188, 1936.
[15] P. Simard, Y. Le Curt, J. Denker, and B. Victorri, "Transformation invariance in pattern recognition -- Tangent distance and tangent propagation," Neural networks." tricks of the trade, volume 1524 of Lecture Notes in Computer Science, Springer, Heidelberg, pp. 239-274, 1998.
[16] P. Simard, Y. Le Curt, and J. Denker, "Efficient pattern recognition using a new transformation distance," Advances in Neural lnf Proe. Systems, volume 5, Morgan Kaufmnann, San Mateo CA, pp. 50-58, 1993.
[17] B. Scholkopf, Support Vector Learning, Ph.D. Thesis, Berlin, 1997.
[18] R.A. Jacobs, "Increased rates of convergence through learning rate adaptation," Neural Networks, Vol. 1, pp. 295-307, 1988.
[19] T. Tollenaere, "SuperSAB: Fast adaptive back propagation with good scaling properties," Neural Networks, Vol. 3, pp. 561-573, 1990.
[20] Y. S. Zhou and L. Y. Lai, "Optimal design for fuzzy controllers by genetic algorithms," IEEE Trans. Ind. Applicat., Vol. 36, pp. 93-97, 2000.
[21] C. F. Juang, J. Y. Lin, and C. Y. Lin, "Genetic reinforcement learning through symbiotic evolution for fuzzy controller design," IEEE Trans. Syst., Man. Cybern. B, Vol. 30, pp. 290-302,2000.
[22] K. Belarbi and F. Titel, "Genetic algorithm for the design of a class of fuzzy controllers: An alternative approach," IEEE Trans. Fuzzy Systems, Vol. 8, pp. 398-405, 2000. [23] H. Juidette and H. Youlal, "Fuzzy dynamic path planning using genetic algorithms," Electron. Lett., Vol. 36(4), pp. 374-376, 2000.
[24] M. Setnes and H. Roubos, "GA-fuzzy modeling and classification: Complexity and performance," IEEE Trans. Fuzzy Syst., Vol. 8, pp. 509-522, 2000.
[25] D. E. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning, Addison Wesley, 1989.
[1] A. Dempster, N. Laird and D. Rubin, "Maximum likelihood from incomplete data via the EM algorithm," Journal of the Royal Statistical Society B, Vol. 39, pp. 1-38, 1977.
[2] M. Seeger, "Learning with labeled and unlabeled data," Technical report, University of Edinburgh, 2000.
[3] W. Du, K. Inoue, and K. Urahama, "Dimensionality reduction for semi-supervised face recognition," Lecture notes in computer science, Vol. 3614, pp. 1-10,2005.
[4] Herve Abdi, "Multivariate Analysis," Encyclopedia of Social Sciences Research Methods, Thousand Oaks (CA): Sage, 2003.
[5] M. Belkin and P. Niyogi, "Laplacian eigenmaps for dimensionality reduction and data representation," Neural Computation, Vol. 15(6), 1373-1396, 2003.
[6] A. Y. Ng, M. I. Jordan and Y. Weiss, "On spectral clustering: Analysis and an algorithm, " Proc. NIPS' 01, pp. 849-856,2001.
[7] M. Turk and A. Petland, "Face recognition using eigenfaces", J. cognitive Neurosci., Vol.3, pp. 71-86, 1991.
[8] D.Y. Zhou, O. Bousquet, T.N. Lal, J. Weston and B. Scholkopf, "Learning with local and global consistency," Max Planck Institute for Biological Cybernetics Technical Report, 2003.
[9] X. Zhu, Z. Ghahramani and J. Lafferty, "Semi-supervised learning using Gaussian field and harmonic functions," Proceeding of the 20th International Conference on Machine Learning (ICML-2003), Washington DC, 2003.
[10] M. Belkin and R Niyogi, "Semi-supervised learning on Riemannian manifolds," Machine Learning, Vol. 56, pp. 209-239, 2004.
[11] R. A. Horn and C.R. Johnson, Matrix Analysis, Cambridge Univ. Press, Cambridge, 1990.
[1] M. Belkin and P. Niyogi, "Laplacian eigenmaps for dimensionality reduction and data representation," Neural Computation, Vol. 15(6), pp. 1373-1396, 2003.
[2] M. Belkin and P. Niyogi, "Semi-supervised learning on Riemannian manifolds," Machine Learning, Vol. 56, pp. 209-239, 2004.
[3] W. Du, K. Inoue and K. Urahama, "Dimensionality reduction for semi-supervised face recognition," Lecture notes in computer science, Vol. 3614, pp. 1-10,2005.
[4] D.Y. Zhou, O. Bousquet, T.N. Lal, J. Weston and B. Scholkopf, "Learning with local and global consistency," Max Planck lnstitute for Biological Cybernetics Technical Report, 2003.
[5] X. Zhu, Z. Ghahramani and J. Lafferty, "Semi-supervised learning using Gaussian field and harmonic functions," Proceeding of the 20th International Conference on Machine Learning (ICML-2003), Washington DC, 2003.
[6] R. A. Horn and C.R. Johnson, Matrix Analysis, Cambridge Univ. Press, Cambridge, 1990.
[7] Xin Geng, De-Chuan Zhan and Zhi-Hua Zhou, "Supervised Nonlinear Dimensionality Reduction for Visualization and Classification," IEEE Trans. on Systems, Man, and Cybernetics—PART B: Cybernetics, Vol. 35(6), pp. 1098-1106, 2005.
[8] M. Turk and A. Petland, "Face recognition using eigenfaces," J. cognitive Neurosci., Vol.3, pp. 71-86, 1991.
[9] M. J. Er, S. Wu, J. Lu and H. L. Toh, "Face Recognition With Radial Basis Function (RBF) Neural Networks," IEEE Trans. Neural Networks, Vol. 13, No.3, pp. 697-710, 2002.
[10] S. Rosenberg, The Laplacian on a Riemannian Manifold, Cambridge University Press, 1997.