A Kernel Method to Extract Common Features Based on Mutual Information

详细信息    查看全文

• 关键词：Kernel canonical correlation analysis ; mutual information ; Parallel Tempering
• 刊名：Lecture Notes in Computer Science
• 出版年：2014
• 出版时间：2014
• 年：2014
• 卷：8835
• 期：1
• 页码：26-34
• 全文大小：218 KB
• 参考文献：1. Akaho, S.: A kernel method for canonical correlation analysis. In: Proceedings of the International Meeting of the Psychometric Society (IMPS 2001) (2001)
  2. Araki, T., Ikeda, K.: Adaptive Markov chain Monte Carlo for auxiliary variable method and its application to Parallel Tempering. Neural Networks?43, 33-0 (2013) CrossRef
  3. Bach, F.R., Jordan, M.I.: Kernel independent component analysis. The Journal of Machine Learning Research?3, 1-8 (2003)
  4. Faivishevsky, L., Goldberger, J.: ICA based on a smooth estimation of the differential entropy. In: Advances in Neural Information Processing Systems, pp. 433-40 (2008)
  5. Geyer, C.: Markov chain Monte Carlo maximum likelihood. In: Proc. 23rd Symp. Interface Comput. Sci. Statist., pp. 156-16 (1991)
  6. Hino, H., Murata, N.: A conditional entropy minimization criterion for dimensionality reduction and multiple kernel learning. Neural Computation?22(11), 2887-923 (2010) CrossRef
  7. Hukushima, K., Nemoto, K.: Exchange Monte Carlo method and application to spin glass simulations. Journal of the Physical Society of Japan?65(6), 1604-608 (1996) CrossRef
  8. Melzer, T., Reiter, M.K., Bischof, H.: Nonlinear feature extraction using generalized canonical correlation analysis. In: Dorffner, G., Bischof, H., Hornik, K. (eds.) ICANN 2001. LNCS, vol.?2130, pp. 353-60. Springer, Heidelberg (2001) CrossRef
  9. Robert, C., Casella, G.: Monte Carlo Statistical Methods. Springer (2004)
  10. Sch?lkopf, B., Herbrich, R., Smola, A.J.: A generalized representer theorem. In: Helmbold, D.P., Williamson, B. (eds.) COLT/EuroCOLT 2001. LNCS (LNAI), vol.?2111, pp. 416-26. Springer, Heidelberg (2001) CrossRef
  11. Sch?lkopf, B., Smola, A.: Learning with kernels. MIT Press, Cambridge (2002)
  
• 作者单位：Takamitsu Araki (20)
  Hideitsu Hino (21)
  Shotaro Akaho (20)
  
  20. Human Technology Research Institute, National Institute of Advanced Industrial Science and Technology, Central 2, 1-1-1 Umezono, Tsukuba, Ibaraki, 3058568, Japan
  21. Department of Computer Science, University of Tsukuba, 1-1-1 Tennodai, Tsukuba, Ibaraki, 3058573, Japan
  
• ISSN：1611-3349

文摘

Kernel canonical correlation analysis (CCA) aims to extract common features from a pair of multivariate data sets by maximizing a linear correlation between nonlinear mappings of the data. However, the kernel CCA tends to obtain the features that have only small information of original multivariates in spite of their high correlation, because it considers only statistics of the extracted features and the nonlinear mappings have high degree of freedom. We propose a kernel method for common feature extraction based on mutual information that maximizes a new objective function. The objective function is a linear combination of two kinds of mutual information, one between the extracted features and the other between the multivariate and its feature. A large value of the former mutual information provides strong dependency to the features, and the latter prevents loss of the feature’s information related to the multivariate. We maximize the objective function by using the Parallel Tempering MCMC in order to overcome a local maximum problem. We show the effectiveness of the proposed method via numerical experiments.