摘要
半监督学习(semi-supervised learning)是利用未标记样本进行学习的主流技术,是目前机器学习中非常活跃的研究方向之一。本文侧重于样本的几何信息在半监督学习中的应用研究,主要工作包括几何信息和标记信息的融合、基于图的归纳式算法、流形仿射对齐、多分辨率RandomWalk图像分割、几何信息对半监督学习影响的对比分析等。
半监督学习主要靠挖掘未标记样本中的隐藏信息以提高分类器的精度,通常需要将标记信息和样本的几何信息融合在一起,其中的融合系数一般是固定的。本文通过研究两部分融合信息的关系,提出随着标记比例改变融合系数,以有效的提高学习精度。
基于图的学习是近几年来半监督学习中一个相当活跃的方向,它用图来描述样本空间,利用近邻点的位置来控制标记信息的传播。由于图的特性的限制,大多数此类算法是直推式的,虽然推导过程直观、分类效果精度高,却没有给出显式的映射关系。本文提出了半监督局部线性调和算法,将混合模型和局部线性调和引入半监督学习之中,通过局部线性映射,实现基于图的归纳式的学习,给出了显式的映射关系。
流形对齐是寻找两个或两个以上的数据集中的隐空间,并根据一些监督信息,将这些隐空间对齐在一起,以寻找数据集间对应点的相互联系。大多数流形对齐算法只能给出了训练集上的预测值,而没有给出整个数据空间上的映射关系。本文提出了一种流形仿射对齐算法,能通过线性变换实现流形对齐,便于直接映射新的数据点。
由于内存消耗和分割时间的限制,大多数半监督图像分割的算法不能直接应用到大图像上,为此本文提出了基于多分辨率Random-walk的半监督图像分割算法。该算法利用低频子图的分割概率近似原始图像上的分割概率,同时迅速找出分割带有争议性的区域,然后在争议区域上进行精确分割。该算法较好地解决了大图片的分割问题,减少了内存消耗、缩短了分割时间,且具有一定的鲁棒性,在复杂复杂背景图像上也能取得较好的分割效果。
为更好的研究几何信息与半监督学习效果的关系,本文提出了一个几何信息与标记信息的融合框架,并通过中间变量将局部混合模型也纳入到该框架内。最后通过实验对比和分析了几种几何信息对半监督学习效果的影响。
Semi-supervised learning is the primary method of making use of unlabeled samples for learning, and is a very active research field in machine learning. This article focuses on the applied research of geometric information in the semi-supervised learning. The main work includes the integration of geometric information and tag information, graph-based inductive algorithm, affine manifold alignment, image segmentation by multi-resolution Random-Walk, comparative effects and analysis of geometric information in semi-supervised learning.
Semi-supervised learning mainly digs the hidden information in the unlabeled data to improve the classifier accuracy, and usually contains the integration of label information and geometric information. In most algorithms the coefficient for melting the two parts is fixed. In this paper, we point out that the weights of the labeled information and manifold structural information could be changed with the proportion of the labeled points in order to effectively improve the learning accuracy.
Graph-based learning is a very active direction of semi-supervised learning in recent years. It describes the sample space by graph, and uses neighbors to spread label information in point cloud. For the restriction of the graph feature, most of these algorithms are transductive that they can't produce an explicit mapping. By introducing the mixed model and local linear coordinate into the semi-supervised learning, we propose the semi-supervised local linear coordination algorithm. The algorithm is an inductive graph-based method, and achieves better performance than linear methods by local linear transformation.
Manifold alignment is to find the hidden space of two or more data sets, and align them in a global coordination where the corresponding pairwise relationship could be found easily. Most of manifold alignment algorithms can only give the predictive value of the training set instead of producing a mapping defined everywhere. We present a manifold affine alignment algorithm, which facilitates direct mapping of new data points.
As the constraints of memory consumption and time of segmentation, most of the semi-supervised image segmentation algorithms can not be directly applied to large images. In this paper, a semi-supervised image segmentation algorithm based on multi-resolution Random-Walk is proposed. Low frequency sub-division is used here to approximate the segmentation probability of the original image, while the controversial area is quickly identified, then the accurate segmentation is imposed on the disputed area. The algorithm offers a better solution to semi-supervised segmentation on large images, and is robust in complex background environment.
In order to better study the relationship between the geometric information and the semi-supervised learning effect, a framework for integrating the geometric information and tag information is present here. The methods based on mixed model are also incorporated into the framework by the definition of middle variable. The comparison and analysis of geometric information's impact on semi-supervised learning is performed through the experiments.
引文
邓超,郭茂祖.2008.基于Tri-Training和数据剪辑的半监督聚类算法[J].软件学报,19(3):663-673.
宫秀军,史忠植.2002.基于Bayes潜在语义模型的半监督Web挖掘[J].软件学报,13(8):1508-1514.
黄树成,朱宇光,董逸生.2007.基于半监督学习的数据流分类算法[J].计算机研究与发展,44(z2).
李和平,胡占义,吴毅红等2007..基于半监督学习的行为建模与异常检测[J].软件学报,18(3):527-537.
金骏,张道强.2008.半监督鲁棒联机聚类算法[J].计算机研究与发展.45(3):496-502.
王玲,薄列峰,焦李成.2007.密度敏感的半监督谱聚类[J].软件学报.18(10):2412-2422.
杨剑,王珏,钟宁.2007.流形上的Laplacian半监督回归[J].计算机研究与发展.
44(7):1121·1127.
杨剑,李伏欣,王珏.2005 一种改进的局部切空间排列算法[J].软件学报,16(9):1584-1589.
尹清波,王慧强,张汝波等.2006.半监督在线增量自学习异常检测方法研究[J],计算机研究与发展.43(z2).
朱付平,田捷,林瑶等.2002.基于Level Set方法的医学图像分割[J].软件学报,13(9):1866-1872.
Abdella M, Marwala T.2005. The use of genetic algorithms and neural networks to approximate missing data in database[C]. In Proceedings of the IEEE International Conference on Computational Cybernetics, pages 207-212.
Agrafiotis D.K.2003. Stochastic proximity embedding[J]. Journal of Computational Chemistry, 24(10)1215-1221.
Agrawala A. K.1970. Learning with a probabilistic teacher[J]. IEEE Transactions on Information Theory,16:373-379.
Aldous, D. and Fill, J. (in preparation). Reversible Markov Chains and Random Walks on Graphs [M]. online version available at http://www.stat.berkeley.edu/users/aldous/RWG/book.html.
Alex Pothen, and Fan Chin-Ju.1990. Computing the Block Triangular Form of a Sparse Matrix[J]. ACM Transactions on Mathematical Software,16(4):303-324.
Anderson W.N. and Morley T.D.1985. Eigenvalues of the Laplacian of a graph[J]. Linear and Multilinear Algebra,18:141-145.
Bai X, Yu H, Hancock E R.2004. Graph Matching Using Spectral Embedding and Alignment[C]. 17th International Conference on Pattern Recognition, Cambridge:398-401.
Balasubramanian M. and Schwartz E.L.2002. The Isomap algorithm and topological stability[J]. Science,295(5552):7.
Baudat G. and Anouar F.2000. Generalized discriminant analysis using a kernel approach[J]. Neural Computation,12(10):2385-2404.
Belkin, M., Niyogi, P., Sindhwani, V.2004. Manifold regularization:A geometric framework for learning from examples[R]. Technical Report TR-2004-06. University of Chicago.
Belkin M, Niyogi P.2002. Laplacian Eigenmaps and spectral techniques for embedding and clustering[C] Advances in Neural Information Processing Systems 16, Vancouver, Canada: The MIT Press,:585-591.
Belkin M, Niyogi P.2004. Semi-supervised learning on Riemannian manifolds [J]. Machine Learning,56 (Special Issue on Clustering):209-239.
Belkin M, Niyogi P, Sindhwani V.2006. Manifold regularization:A geometric framework for learning from examples[J].Journal of Machine Learning Research,7(11):2399-2431
Bengio Y., Delalleau O., Le Roux N., Paiement J.-F. Vincent P., and Ouimet M.2004. Learning eigenfunctions links spectral embedding and Kernel PCA[J]. Neural Computation, 16(10):2197-2219.
Bengio Y. and LeCun Y.2007. Scaling learning algorithms towards AI[M]. In L. Bottou, O. Chapelle, D. DeCoste, and J. Weston, editors, Large-Scale Kernel Machines, pages 321-360. MIT Press.
Bengio Y. and Monperrus M.2004. Non-local manifold tangent learning[C]. In Advances in Neural Information Processing Systems, volume 17, pages 129-136, Cambridge, MA, USA. The MIT Press.
Bengio Y, Paiement J.-F., Vincent P., Delalleau O., Le Roux N., and Ouimet M.2004. Out-of-sample extensions for LLE, Isomap, MDS, eigenmaps, and spectral clustering[C]. In Advances in Neural Information Processing Systens, volume 16, Cambridge, MA, USA. The MIT Press.
Betechuoh B.L. Marwala T. and Tettey T.2006. Autoencoder networks for HIV classification[J]. Current Science,91(11):1467-1473.
Blum A., Chawla S.2001.Learning from labeled and unlabeled data using graph mincuts[C]. In: Proceedings of 18th International Conf. on Machine Learning. Williams College, Williamstown, MA, USA,19-26.
Borchers B. and Young J.G.2007. Implementation of a primaldual method for SDP on a shared memory parallel architecture[J]. Computational Optimization and Applications, 37(3):355-369.
Brand M.2002. Charting a manifold[C]. In Advances in Neural Information Processing Systems, volume 15, pages 985-992, Cambridge, MA, USA. The MIT Press.
Brand M.2004. From subspaces to submanifolds[C]. In Proceedings of the 15th British Machine Vision Conference, London, UK. British Machine Vision Association.
Braun M.L., Lange T., and Buhmann J.M.2006. Model selection in kernel methods based on a spectral analysis of label information[C]. In Proceedings of the 28th DAGM Symposium, pages 344-353, Berlin, Germany. Springer.
Brun A., Park H.-J., Knutsson H., and Westin C.-F.2003. Coloring of dt-mri fiber traces using laplacian eigenmaps[C]. In Proccedings of the Eurocast 2003, Neuro Image Workshop.
Burges C.J.C.2005. Data Mining and Knowledge Discovery Handbook:A Complete Guide for Practitioners and Researchers, chapter Geometric Methods for Feature Selection and Dimensional Reduction:A Guided Tour[M]. Kluwer Academic Publishers.
Cai D., He X., and Han J.2007. Spectral regression for efficient regularized subspace learning[C]. IEEE 11th International Conference on Computer Vision, Rio de Janeiro, Brazil: 1-8.
Cai D., He X., and Han J.2007. Spectral regression:A unified approach for sparse subspace learning[C] IEEE 7th International Conference on Data Mining, Omaha, Nebraska:73-82.
Camastra F. and Vinciarelli A.2002. Estimating the intrinsic dimension of data with a fractal-based method[J] IEEE Trans. on Pattern Analysis and Machine Intelligence,24(10): 1404-1407.
Chang H., Yeung D.-Y., and Xiong Y.2004. Super-resolution through neighbor embedding[C]. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition, volume 1, pages 275-282.
Chapelle O., Weston J., Scholkopf B.2002. Cluster kernels for semi-supervised learning[C]. In: Advances in Neural Information Processing Systems.15:585-592.
Chapelle O, Zien A.2005. Semi-supervised classification by low density separation[C]. Tenth International Workshop on Artificial Intelligence and Statistics, Barbados:6-8.
Chapelle, O., Zien, A., and Sch"olkopf, B.2006. Semi-supervised learning[M].MIT Press.
Chapelle O, M. Chi, Zien A.2006. A continuation method for semi-supervised svms[C].23rd International Conference on Machine Learning Carnegie Mellon, Pittsburgh, Pennsylvania: 25-29.
Chapelle O, Sindhwani V, Keerthi S.2007. Branch and Bound for Semi-Supervised Support Vector Machines[C]. Advances in Neural Information Processing Systems 19:Proceedings of the 2006 Conference, MIT Press, Cambridge, Mass:217-224.
Choi H. and Choi S.2007. Robust kernel Isomap[J]. Pattern Recognition,40(3):853-862.
Collobert R, Sinz F, Weston J, Bottou L.2006. Large scale transductive SVMs. Journal of Machine Learning Research, Submitted, www.kyb.tuebingen.mpg.de/bs/people/fabee/universvm.html.
Costa J.A. and Hero A.O.2005. Classification constrained dimensionality reduction[C]. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, volume 5, pages 1077-1080.
Cristianini, N., Shawe-Taylor, J. and Kandola, J.2001. On kernel-target alignment[C]. In Advances in NIPS.
de Silva V. and Tenenbaum J.B.2003. Global versus local methods in nonlinear dimensionality reduction[C]. In Advances in Neural Information Processing Systems, volume 15, pages 721-728, Cambridge, MA, USA. The MIT Press.
DeMers D. and Cottrell G.1993. Non-linear dimensionality reduction[C]. In Advances in Neural Information Processing Systems, volume 5, pages 580-587, San Mateo, CA, USA. Morgan Kaufmann.
Dempster A., Laird N., and Rubin D.1977. Maximum likelihood from incomplete data via the EM algorithm[J]. Journal of the Royal Statistical Society, Series B,39(1):1-38.
Ding, C., He, X., Zha, H., Gu, M., and Simon, H.2001. A min-max cut algorithm for graph partitioning and data clustering[C]. In Proceedings of the first IEEE International Conference on Data Mining (ICDM). Washington, DC, USA:IEEE Computer Society:107-114.
Ding, C.2004. A tutorial on spectral clustering. Talk presented at ICML. (Slides available at http://crd.lbl.gov/_cding/Spectral/)
Donoho D.L. and Grimes C.2005. Hessian eigenmaps:New locally linear embedding techniques for high-dimensional data[C]. Proceedings of the National Academy of Sciences, 102(21):7426-7431.
Duchenne O, Audibert J, Keriven R, et al.2008. Segmentation by transduction[C]. Computer Vision and Pattern Recognition, Page(s):1-8.
Duraiswami R. and Raykar V.C.2005. The manifolds of spatial hearing[C]. In Proceedings of International Conference on Acoustics, Speech and Signal Processing, volume 3, pages 285-288.
Erdo" s P. and Re'nyi A.1959. On random graphs[J]. Publicationes Mathematicae,6:290-297.
Fralick S. C.1967. Learning to recognize patterns without a teacher[J]. IEEE Transactions on Information Theory,13:57-64.
Fujino, A., Ueda, N., Saito, K.2005. A hybrid generative/discriminative approach to semi-supervised classifier design[C]. AAAI-05, The Twentieth National Conference on Artificial Intelligence.
Garcke, J.,& Griebel, M.2005. Semi-supervised learning with sparse grids[C]. Proc. of the 22nd ICMLWorkshop on Learning with Partially Classified Training Data. Bonn, Germany.
Ghahramani Z. and Hinton G.E.1996. The EM algorithm for mixtures of factor analyzers[R]. Technical Report CRG-TR-96-1, Department of Computer Science, University of Toronto.
Graepel T.2002. Kernel matrix completion by semidefinite programming[C]. In Proceedings of the International Conference on Artificial Neural Networks, pages 694-699, Berlin, Germany. Springer-Verlag.
Graf A.B.A. and Wichmann F.A.2002. Gender classification of human faces[C]. In Biologically Motivated Computer Vision 2002, LNCS 2525, pages 491-501.
Grau V, Mewes A, Alcaniz M, et al.2004. Improved watershed transform for medical image segmentation using prior information[J]. IEEE Transactions on Medical Imaging,23 (4): 447-458.
Haffari G.2005. A Survey on Inductive Semi-Supervised Learning[R]. Technical report, School of Computing Science, Simon Fraser University.
Hagen, L. and Kahng, A.1992. New spectral methods for ratio cut partitioning and clustering. IEEE Trans. Computer-Aided Design,11 (9):1074-1085.
Ham J., Lee D., Mika S., and Scho" lkopf B.2003. A kernel view of the dimensionality reduction of manifolds. Technical Report TR-110, Max Planck Institute for Biological Cybernetics, Germany.
Ham J., Lee D., and Saul L.2005. Semisupervised Alignment of Manifolds[C]Proceedings of the Tenth International Workshop on Artificial Intelligence and Statistics, Barbados:120-127.
He X., Cai, D. Yan S., and Zhang H.-J.2005. Neighborhood preserving embedding[C]. In Proceedings of the 10th IEEE International Conference on Computer Vision, pages 1208-1213.
He X.2004. Incremental semi-supervised subspace learning for image retrieval [C]. In: Proceedings of the ACM Conference on Multimedia. New York,10.
He X. and Niyogi P.2004. Locality preserving projections[C]. In Advances in Neural Information Processing Systems, volume 16, page 37, Cambridge, MA, USA. The MIT Press.
He X., Yan S., Hu Y., Niyogi P., and Zhang H.-J.2005. Face recognition using laplacianfaces[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,27(3):328-340.
Hinton G.E.2002. Training products of experts by minimizing contrastive divergence [J]. Neural Computation,14(8):1771-1800.
Hinton G.E. and Roweis S.T.2002. Stochastic Neighbor Embedding[C]. In Advances in Neural Information Processing Systems, volume 15, pages 833-840, Cambridge, MA, USA. The MIT Press.
Hinton G.E. and Salakhutdinov R.R.2006. Reducing the dimensionality of data with neural networks[J].Science,313(5786):504-507.
Hoffmann H.2007. Kernel PCA for novelty detection[J]. Pattern Recognition,40(3):863-874.
Huber R., Ramoser H., Mayer K., Penz H., and Rubik M.2005. Classification of coins using an eigenspace approach[J]. Pattern Recognition Letters,26(1):61-75.
Hughes N.P. and Tarassenko L.2003. Novel signal shape descriptors through wavelet transforms and dimensionality reduction[C]. In Wavelet Applications in Signal and Image Processing X, pages 763-773.
Jain A. K. and Dubes R. C.1988. Algorithms for clustering data[M], Prentice Hall, Englewood Cliffs, NJ,.
Jenkins O.C. and Mataric M.J.2002. Deriving action and behavior primitives from human motion data[C]. In International Conference on Intelligent Robots and Systems, volume 3, pages 2551-2556.
Joachims T.1999. Transductive inference for text classification using support vector machines[C]. The Sixteenth International Conference on Machine Learning. Bled, Slovenia: 27-30.
Johnson Rie and Zhang Tong.2007. On the Effectiveness of Laplacian Normalization for Graph Semisupervised Learning. Journal of Machine Learning Research,8:1489-1517.
Kambhatla N. and Leen T.K.1997. Dimension reduction by local principal component analysis[J]. Neural Computation,9(7):1493-1516.
Kegl B.2002. Intrinsic dimension estimation based on packing numbers[C]. In Advances in Neural Information Processing Systems. Cambridge, MA, USA. The MIT Press.15:833-840.
Kim K.I., Jung K., and Kim H.J.2002. Face recognition using kernel principal component analysis[J]. IEEE Signal Processing Letters,9(2):40-42.
Kirby M.2001. Geometric Data Analysis:An Empirical Approach to Dimensionality Reduction and the Study of Patterns, Wiley-Interscience.
Kokiopoulou E. and Saad Y.2007. Orthogonal Neighborhood Preserving Projections:A projection-based dimensionality reduction technique[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,29(12):2134-2156.
Kondor R.I. and Lafferty J,.2002. Diffusion kernels on graphs and other discrete input spaces[C]. Proc.19th International Conf. on Machine Learning.
Lafon S, Lee A B.2006. Diffusion maps and coarse-graining:A unified framework for dimensionality reduction, graph partitioning, and data set parameterization [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,28(9):1393-1403.
Lafon S., Keller Y, and Coifman R. R.2006. Data Fusion and Multicue Data Matching by Diffusion Maps[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence.28(11): 1784-1797.
Lafon S, Lee A.B.2006. Diffusion maps and coarse-graining:A unified framework for dimensionality reduction, graph partitioning, and data set parameterization[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,28(9):1393-1403.
Law M.H. and Jain A.K.2006. Incremental nonlinear dimensionality reduction by manifold learning[J]. IEEE Transactions of Pattern Analysis and Machine Intelligence,28(3):377-391.
Lee J.A. and Verleysen M.2005. Nonlinear dimensionality reduction of data manifolds with essential loops[J]. Neurocomputing,67:29-53.
Leo Grady.2006. Random walks for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,28(11):1768-1783.
L. Grady and G. Funka-Lea.2004. Multi-Label Image Segmentation for Medical Applications Based on Graph-Theoretic Electrical Potentials[C]. Proc. Workshop Computer Vision and Math. Methods in Medical and Biomedical Image Analysis, Volume 3117:230-245.
Levina E. and Bickel P.J.2004. Maximum likelihood estimation of intrinsic dimension[C]. In Advances in Neural Information Processing Systems, volume 17, Cambridge, MA, USA. The MIT Press.
Li Bo, and Huang D.S.2008a. Locally linear discriminant embedding:An efficient method for face recognition[J], Pattern Recognition,41(12):3813-3821.
Li Bo, Huang D.S., Wang Chao and Liu Kun-Hong.2008b. Feature extraction using constrained maximum variance mapping[J]. Pattern Recognition,41(11):3287-3294.
Li H., Teng L., Chen W., and Shen I.-F.2005. Supervised learning on local tangent space[M]. In Lecture Notes on Computer Science, volume 3496, pages 546-551, Berlin, Germany. Springer Verlag.
Lim I.S., Ciechomski P.H., Sarni S., and Thalmann D.2003. Planar arrangement of high-dimensional biomedical data sets by Isomap coordinates[C]. In Proceedings of the 16th IEEE Symposium on Computer-Based Medical Systems, pages 50-55.
Lima A., Zen H., Nankaku Y., Miyajima C., Tokuda K., and Kitamura T.2004. On the use of Kernel PCA for feature extraction in speech recognition[J]. IEICE Transactions on Information Systems, E87-D(12):2802-2811.
Liu Juan, Moulin P.2001. Information-Theoretic Analysis of Interscale and Intrascale Dependencies Between Image Wavelet Coefficients[J]. IEEE Transactions on Image Processing,10(11):1647-1658.
Lovasz L.1993. Random walks on graphs:a survey[M]. In Combinatorics, Paul Erdos is eighty (pp.353-397). Budapest:Janos Bolyai Math. Soc.
von Luxburg, U., Belkin, M., and Bousquet, O.2004. Consistency of spectral clustering[R]. Technical Report 134, Max Planck Institute for Biological Cybernetics
von Luxburg, U., Bousquet, O., and Belkin, M.2004. On the convergence of spectral clustering on random samples:the normalized case[C], Proceedings of the 17th Annual Conference on Learning Theory (COLT). Springer, New York:457-471.
von Luxburg, U., Bousquet, O., and Belkin, M.2005. Limits of spectral clustering[C]. Advances in Neural Information Processing Systems (NIPS) 17. Cambridge, MA:MIT Press:857-864.
von Luxburg. U.2007. A tutorial on spectral clustering. Statistics and Computing, 17(4):395-416.
Mallat SG.1989. A theory for multiresolution signal decomposition:The wavelet representation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,11(7): 674-693.
Mekuz N. and Tsotsos J.K.2006. Parameterless Isomap with adaptive neighborhood selection[C]. In Proceedings of the 28th DAGM Symposium, pages 364-373, Berlin, Germany. Springer.
Meytlis M. and Sirovich L.2007. On the dimensionality of face space[J]. IEEE Transactions of Pattern Analysis and Machine Intelligence,29(7):1262-1267.
Nadler B., Lafon S., Coifman R.R., and Kevrekidis I.G.2006. Diffusion maps, spectral clustering and the reaction coordinates of dynamical systems[J]. Applied and Computational Harmonic Analysis:Special Issue on Diffusion Maps and Wavelets,21:113-127.
Nam K., Je H., and Choi S.2004. Fast Stochastic Neighbor Embedding:A trust-region algorithm[C]. In Proceedings of the IEEE International Joint Conference on Neural Networks 2004, volume 1, pages 123-128, Budapest, Hungary.
Ng A., Jordan M., and Weiss Y.2001. On spectral clustering:Analysis and an algorithm[C]. In Advances in Neural Information Processing Systems, volume 14, pages 849-856, Cambridge, MA, USA. The MIT Press.
Nigam K.2001. Using unlabeled data to improve text classification [D]. Carnegie Mellon University.
Niskanen M. and Silve'n O.2003. Comparison of dimensionality reduction methods for wood surface inspection[C]. In Proceedings of the 6th International Conference on Quality Control by Artificial Vision, pages 178-188, Gatlinburg, TN, USA. International Society for Optical Engineering.
Park J.-H., Zhang Z., Zha H., and Kasturi R.2004. Local smoothing for manifold learning[J]. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition, volume 2, pages 452-459.
Partridge M. and Calvo R.1997. Fast dimensionality reduction and Simple PCA[J]. Intelligent Data Analysis,2(3):292-298.
Patwari N. and Hero A.O.2004. Manifold learning algorithms for localization in wireless sensor networks[C]. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, volume 3, pages 857-860.
Penrose M.2003. Random Geometric Graphs[M]. Oxford University Press.
Platt J.C.2005. FastMap, MetricMap, and Landmark MDS are all Nystro m algorithms[C]. In Proceedings of the 10th International Workshop on Artificial Intelligence and Statistics, pages 261-268.
Pohle R, Toennies KD.2001. Segmentation of medical images using adaptive region growing[C]. Proc. SPIE Medical Imaging,2(27):1337-1346.
Rajpoot N.M., Arif M., and Bhalerao A.H.2007. Unsupervised learning of shape manifolds[C]. In Proceedings of the British Machine Vision Conference.
Raytchev B., Yoda I., and Sakaue K.2004. Head pose estimation by nonlinear manifold learning[C]. In Proceedings of the 17th ICPR, pages 462-466.
Riloff, E., Wiebe, J., Wilson, T.2003. Learning subjective nouns using extraction pattern bootstrapping[C]. Proceedings of the Seventh Conference on Natural Language Learning (CoNLL-2003).
Rosenberg S..1997. The Laplacian on a Riemannian Manifold[M]. Cambridge University Press, Cambridge, UK.
Rosenberg, C., Hebert, M., Schneiderman, H.2005. Semi-supervised selftraining of object detection models[C]. Seventh IEEE Workshop on Applications of Computer Vision.
Roweis S.T.1997. EM algorithms for PCA and SPCA[C]. In Advances in Neural Information Processing Systems, volume 10, pages 626-632.
Roweis S.T., Saul L., and Hinton G.2001. Global coordination of local linear models[C]. In Advances in Neural Information Processing Systems, volume 14, pages 889-896, Cambridge, MA, USA. The MIT Press.
Roweis S.T. and Saul L.K.2000. Nonlinear dimensionality reduction by Locally Linear Embedding[J]. Science,290(5500):2323-2326.
Samko O., Marshall A.D., and Rosin P.L.2006. Selection of the optimal parameter value for the Isomap algorithm[J]. Pattern Recognition Letters,27(9):968-979.
Saul L.K., Weinberger K.Q., Ham J.H., Sha F., and Lee D.D.2006. Spectral methods for dimensionality reduction[M]. In Semisupervised Learning, Cambridge, MA, USA. The MIT Press.
Saxena A., Gupta A., and Mukerjee A.2004. Non-linear dimensionality reduction by locally linear isomaps[M]. Lecture Notes in Computer Science,3316:1038-1043.
Scho" lkopf B., Smola A.J., and Mu ller K.-R.1998. Nonlinear component analysis as a kernel eigenvalue problem[J]. Neural Computation, 10(5):1299-1319.
Scudder H. J.1965. Probability of error of some adaptive pattern-recognition machines[J]. IEEE Transactions on Information Theory,11:363-371.
Sha F. and Saul L.K.2005. Analysis and extension of spectral methods for nonlinear dimensionality reduction[C]. In Proceedings of the 22nd International Conference on Machine Learning, pages 785-792.
Shawe-Taylor J. and Christianini N.2004. Kernel Methods for Pattern Analysis[M]. Cambridge University Press, Cambridge, UK.
Shi J. and Malik J.2000. Normalized cuts and image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,22(8):888-905.
Shon A. P., Grochow K., Hertzmann A., and Rao R.2006. Learning shared latent structure for image synthesis and robotic imitation[C]. Advances in Neural Information Processing Systems 20, Vancouver, Canada:1233-1240.
Smola, A., Kondor, R.2003. Kernels and regularization on graphs[C]. Proceedings of the International Conference on Computer Learning Theory and Seventh Kernel Workshop, Lecture Notes in Computer Science, vol.2777, Springer, Berlin,144-158.
Sindhwani V, Keerthi S, Chapelle O.2006. Deterministic annealing for semi-supervised kernel machines[C].23rd International Conference on Machine Learning, Carnegie Mellon, Pittsburgh, Pennsylvania:25-29.
Stoer, M. and Wagner, F. 1997. A simple min-cut algorithm[J]. Journal of the ACM.44 (4): 585-591.
Suykens J.A.K.2007. Data visualization and dimensionality reduction using kernel maps with a reference point. Technical Report 07-22, ESAT-SISTA, K.U. Leuven.
Szummer, M.,& Jaakkola, T.2001. Partially labeled classification with Markov random walks. Advances in Neural Information Processing Systems,14.
Szummer M. and Jaakkola T.2002. Information regularization with partially labeled data[C]. In Advances in Neural Information Processing Systems, volume 15. MIT Press.
Teh Y.W. and Roweis S.T.2002. Automatic alignment of hidden representations[C]. In Advances in Neural Information Processing Systems, volume 15, pages 841-848, Cambridge, MA, USA. The MIT Press.
Tenenbaum J.B.1998. Mapping a manifold of perceptual observations[C]. In Advances in Neural Information Processing Systems, volume 10, pages 682-688, Cambridge, MA, USA. The MIT Press.
Tenenbaum J.B., de Silva V., and Langford J.C.2000. A global geometric framework for nonlinear dimensionality reduction[J]. Science,290(5500):2319-2323.
Teng L., Li H., Fu X., Chen W., and Shen I.-F.2005. Dimension reduction of microarray data based on local tangent space alignment[C]. In Proceedings of the 4th IEEE International Conference on Cognitive Informatics, pages 154-159.
Tipping M.E.2000. Sparse kernel principal component analysis[C]. In Advances in Neural Information Processing Systems, volume 13, pages 633-639, Cambridge, MA, USA. The MIT Press.
Tipping M.E. and Bishop C.M.1999. Mixtures of probabilistic principal component analysers[J]. Neural Computation,11 (2):443-482.
Tsang, I.,& Kwok, J.2006. Large-scale sparsified manifold regularization[C]. Advances in Neural Information Processing Systems (NIPS) 19.
Van der Maaten L. J. P., Postma E. O., and van den Herik H. J.2007. Dimensionality reduction: A comparative review[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence.
Verbeek, J., Vlassis,N.2006. Gaussian fields for semi-supervised regression and correspondence learning[J]. Pattern Recognition,39(10):1864-1875.
Venkatarajan M.S. and Braun W.2004. New quantitative descriptors of amino acids based on multidimensional scaling of a large number of physicalchemical properties[J]. Journal of Molecular Modeling,7(12):445-453.
Venna J.2007. Dimensionality reduction for visual exploration of similarity structures[D]. PhD thesis, Helsinki University of Technology.
Verbeek J.2006. Learning nonlinear image manifolds by global alignment of local linear models[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(8):1236-1250.
Verveer P. and Duin R.1995. An evaluation of intrinsic dimensionality estimators[J]. IEEE Trans. on Pattern Analysis and Machine Intelligence,17(1):81-86.
Wang Fei, Wang Jingdong, Zhang Changshui and Shen Helen C.2006. Semi-Supervised Classification Using Linear Neighborhood Propagation[C], IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR2006), New York University, New York, New York, USA. June 17-22.
Wang Jingdong, Wang Fei, Zhang Changshui, Shen Helen C., Quan Long.2009. Linear Neighborhood Propagation and Its Applications[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI),31(9):1600-1615.
Wang J., Zhang Z., and Zha H.2005. Adaptive manifold learning[C]. In Advances in Neural Information Processing Systems, volume 17, pages 1473-1480, Cambridge, MA, USA. The MIT Press.
Weinberger K.Q., Packer B.D., and Saul L.K.2005. Nonlinear dimensionality reduction by semidefinite programming and kernel matrix factorization[C]. In Proceedings of the 10th International Workshop on AI and Statistics, Barbados, WI. Society for Artificial Intelligence and Statistics.
Weinberger K.Q, Sha F., and Saul L.K.2004. Learning a kernel matrix for nonlinear dimensionality reduction[C]. In Proceedings of the 21st International Confernence on Machine Learning.
Weinberger K.Q., Sha F., Zhu Q., and Saul L.K.2007. Graph Laplacian regularization for large-scale semidefinite programming[C]. In Advances in Neural Information Processing Systems, volume 19.
Weiss Y.1999. Segmentation using eigenvectors:a unifying view[C]. In Proceedings of the IEEE International Conference on Computer Vision, volume 2, pages 975-982, Los Alamitos, CA, USA. IEEE Computer Society Press.
Welling M., Rosen-Zvi M., and Hinton G.E.2004. Exponential family harmoniums with an application to information retrieval[C]. In Advances in Neural Information Processing Systems, volume 17, pages 1481-1488.
Xiao L., Sun J., and Boyd S.2006. A duality view of spectral methods for dimensionality reduction[C]. In Proceedings of the 23rd International Conference on Machine Learning, pages 1041-1048.
Xu R., Damelin S., and Wunsch D.C.2007. Applications of diffusion maps in gene expression data-based cancer diagnosis analysis[C]. In Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pages 4613-4616.
Yang G., Xu X., Tu L.2009. Manifold Alignment via Local Block Coordinate[A]. In:Knowledge Discovery and Data Mining[C]. Moscow,697-700.
Yang G., Xu X., Zhang J.2008. Manifold Alignment via Local Tangent Space Alignment[A]. In: International Conference on Computer Science and Software Engineering[C]. Wuhan,1, 928-931.
Yang X., Fu H., Zha H., and Barlow J. L.2006. Semisupervised nonlinear dimensionality reduction[A]. In ICML'06[C], Pittsburgh, PA,1065-1072.
Yarowsky D.1995. Unsupervised word sense-disambiguation rivaling supervised methods[C]. In Meeting of the Association for Computational Linguistics, pages:189-196.
Yin JS, Hu DW, Zhou ZT.2007. Growing locally linear embedding for manifold learning[J]. Journal of Pattern Recognition Research,2(1):1-16.
Zhang D., Zhou Z.H., Chen S.2007. Semi-supervised dimensionality reduction[A]. In: Proceedings of the 7th SIAM International Conference on Data Mining[C]. Minneapolis, MN,
Zhang T., Yang J., Zhao D., and Ge X.2007. Linear local tangent space alignment and application to face recognition[J]. Neurocomputing,70:1547-1533.
Zhang Z Y, Zha H Y.2002. Principal Manifolds and Nonlinear Dimension Reduction via Local Tangent Space Alignment[R]. CSE-02-019, Technical Report, CSE, Penn State Univ.
Zhang Z. and Zha H.2003. Local linear smoothing for nonlinear manifold learning[R]. Technical Report CSE-03-003, Department of Computer Science and Engineering, Pennsylvania State University, University Park, PA, USA.
Zhang Z. and Zha H.2004. Principal manifolds and nonlinear dimensionality reduction via local tangent space alignment[J]. SIAM Journal of Scientific Computing,26(1):313-338.
Zhou, D., Bousquet, O., Lal, T., Weston, J., Sch(?)lkopf, B.2004. Learning with local and global consistency[C]. Advances in Neural Information Processing System 16.
Zhou, D., Huang, J., Schoelkopf, B.2006. Learning with hypergraphs:Clustering, classification, and embedding[C]. Advances in Neural Information Processing Systems (NIPS) 19.
Zhou, Z.-H.,& Li, M.2005a. Semi-supervised regression with co-training[C]. International Joint Conference on Artificial Intelligence (IJCAI).
Zhou, Z.-H.,& Li, M.2005b. Tri-training:exploiting unlabeled data using three classifiers[J]. IEEE Transactions on Knowledge and Data Engineering,17,1529-1541.
Zhou, Z.-H.,& Xu, J.-M.2007. On the relation between multi-instance learning and semi-supervised learning[C]. The 24th International Conference on Machine Learning.
Zhou, Z.-H., Zhan, D.-C.,& Yang, Q.2007. Semi-supervised learning with very few labeled training examples[C]. Twenty-Second AAAI Conference on Artificial Intelligence (AAAI-07).
Zhu Xiaojin, Ghahramani Z.2002. Learning from labeled and unlabeled data with label propagation[R]. Technical Report 02-107, CMU-CALD, USA:Carnegie Mellon University.
Zhu X., Ghahramani Z., Lafferty J.2003 Semi-supervised learning using Gaussian fields and harmonic functions[C]. In:Proceedings of the 20th International Conference on Machine Learning. Washington, DC, USA,912-919.
Zhu Xiaojin, Kandola Jaz, Ghahramani Zoubin, and Lafferty John.2004. Non-parametric Transforms of Graph Kernels for Semi-Supervised Learning[C]. Neural Information Processing Systems, NIPS 2004, Vancouver, British Columbia, Canada,13-18.
Zhu X.2005, Semi-supervised learning with graphs [D].Carnegie Mellon University. CMU-LTI-05-192.2005.
Zhu, X., Lafferty, J.2005. Harmonic mixtures:combining mixture models and graph-based methods for inductive and scalable semi-supervised learning[A]. In:The 22nd International Conference on Machine Learning[C]. Bonn, Germany,1052-1059.
Zhu X.2005. Semi-supervised learning literature survey[R]. Technical Report,1530, Computer Sciences, University of Wisconsin-Madison.