Unsupervised learning of Dirichlet process mixture models with missing data

详细信息    查看全文

• 作者：Xunan Zhang ; Shiji Song ; Lei Zhu ; Keyou You ; Cheng Wu
• 关键词：Dirichlet processes ; missing data ; clustering ; variational Bayesian ; image analysis
• 刊名：SCIENCE CHINA Information Sciences
• 出版年：2016
• 出版时间：January 2016
• 年：2016
• 卷：59
• 期：1
• 页码：1-14
• 全文大小：1,163 KB
• 参考文献：1.Li C Z, Xu Z B, Qiao C, et al. Hierarchical clustering driven by cognitive features. Sci China Inf Sci, 2014, 57: 012109MathSciNet
  2.Wu C M, Chou S C, Liaw H T. A trend based investment decision approach using clustering and heuristic algorithm. Sci China Inf Sci, 2014, 57: 092117MathSciNet
  3.McLachlan G, Peel D. Finite Mixture Models. Hoboken: John Wiley and Sons, 2004MATH
  4.Fan W, Bouguila N. Variational learning of a Dirichlet process of generalized Dirichlet distributions for simultaneous clustering and feature selection. Pattern Recogn, 2013, 46: 2754–2769CrossRef MATH
  5.Figueiredo M A T, Jain A K. Unsupervised learning of finite mixture models. IEEE Trans Pattern Anal Mach Intell, 2002, 24: 381–396CrossRef
  6.Neal R M. Markov chain sampling methods for Dirichlet process mixture models. J Comput Graph Stat, 2000, 9: 249–265MathSciNet
  7.Blei D M, Jordan M I. Variational inference for Dirichlet process mixtures. Bayesian Anal, 2006, 1: 121–143MathSciNet CrossRef MATH
  8.Kim S, Tadesse M G, Vannucci M. Variable selection in clustering via Dirichlet process mixture models. Biometrika, 2006, 93: 877–893MathSciNet CrossRef
  9.Orbanz P, Buhmann J M. Nonparametric Bayesian image segmentation. Int J Comput Vision, 2008, 77: 25–45CrossRef
  10.García-Laencina P J, Sancho-Gómez J L, Figueiras-Vidal A R. Pattern classification with missing data: a review. Neural Comput Appl, 2010, 19: 263–282CrossRef
  11.Wang C, Liao X, Carin L, et al. Classification with incomplete data using Dirichlet process priors. J Mach Learn Res, 2010, 11: 3269–3311MathSciNet MATH
  12.Williams D, Liao X J, Xue Y, et al. On classification with incomplete data. IEEE Trans Pattern Anal Mach Intell, 2007, 29: 427–436CrossRef
  13.Schafer J L, Graham J W. Missing data: our view of the state of the art. Psychol Method, 2002, 7: 147–177CrossRef
  14.Little R J A, Rubin D B. Statistical Analysis with Missing Data. 2nd ed. Hoboken: John Wiley and Sons, 2002CrossRef MATH
  15.Chechik G, Heitz G, Elidan G, et al. Max-margin classification of data with absent features. J Mach Learn Res, 2008, 9: 1–21MATH
  16.Fidler S, Skocaj D, Leonardis A. Combining reconstructive and discriminative subspace methods for robust classification and regression by subsampling. IEEE Trans Pattern Anal Mach Intell, 2006, 28: 337–350CrossRef
  17.Chan K, Lee T W, Sejnowski T J. Variational learning of clusters of undercomplete nonsymmetric independent components. J Mach Learn Res, 2003, 3: 99–114MathSciNet MATH
  18.Teh Y W, Jordan M I, Beal M J, et al. Hierarchical dirichlet processes. J Amer Stat Assoc, 2006, 101: 1566–1581MathSciNet CrossRef MATH
  19.Sethuraman J. A constructive definition of Dirichlet priors. Stat Sin, 1994, 4: 639–650MathSciNet MATH
  20.Ghahramani Z, Beal M J. Propagation algorithms for variational Bayesian learning. In: Leen T K, Dietterich T, Tresp V, eds. Advances in Neural Information Processing Systems. Cambridge: MIT Press, 2001. 507–513
  21.Hughes M C, Sudderth E. Memoized online variational inference for Dirichlet process mixture models. In: Burges C J C, Bottou L, Welling M, et al, eds. Advances in Neural Information Processing Systems. Cambridge: MIT Press, 2013. 1133–1141
  22.Bishop C M. Pattern Recognition and Machine Learning. New York: springer, 2006MATH
  23.Lin T I, Lee J C, Ho H J. On fast supervised learning for normal mixture models with missing information. Pattern Recogn, 2006, 39: 1177–1187CrossRef MATH
  24.Collins L M, Schafer J L, Kam C M. A comparison of inclusive and restrictive strategies in modern missing data procedures. Psychol Method, 2001, 6: 330–351CrossRef
  25.Meng X L, Rubin D B. Maximum likelihood estimation via the ECM algorithm: a general framework. Biometrika, 1993, 80: 267–278MathSciNet CrossRef MATH
  26.Ueda N, Nakano R. Deterministic annealing EM algorithm. Neural Netw, 1998, 11: 271–282CrossRef
  27.Barnard K, Duygulu P, Forsyth D, et al. Matching words and pictures. J Mach Learn Res, 2003, 3: 1107–1135MATH
  28.Herzig P M, Hannington M D. Polymetallic massive sulfides at the modern seafloor a review. Ore Geol Rev, 1995, 10: 95–115CrossRef
  
• 作者单位：Xunan Zhang (1)
  Shiji Song (1)
  Lei Zhu (2)
  Keyou You (1)
  Cheng Wu (1)
  
  1. Department of Automation, Tsinghua University, Beijing, 100084, China
  2. China Ocean Mineral Resources R&D Association, Beijing, 100860, China
  
• 刊物类别：Computer Science
• 刊物主题：Chinese Library of Science
  Information Systems and Communication Service
  
• 出版者：Science China Press, co-published with Springer
• ISSN：1869-1919

文摘

This study presents a novel approach to unsupervised learning for clustering with missing data. We first extend a finite mixture model to the infinite case by considering Dirichlet process mixtures, which can automatically determine the number of mixture components or clusters. Furthermore, we view the missing features as latent variables and compute the posterior distributions using the variational Bayesian expectation maximization algorithm, which optimizes the evidence lower bound on the complete-data log marginal likelihood. We demonstrate the performance on several artificial data sets with missing values. The experimental results indicate that the proposed method outperforms some classic imputation methods. We finally present an application to seabed hydrothermal sulfide color images analysis problem.