Asymptotic analysis of the learning curve for Gaussian process regression

详细信息    查看全文

• 作者：Loic Le Gratiet (1) (2)
  Josselin Garnier (3)
  
  1. Universit茅 Paris Diderot ; 75205聽 ; Paris Cedex 13 ; France
  2. CEA ; DAM ; DIF ; 91297聽 ; Arpajon ; France
  3. Laboratoire de Probabilites et Modeles Aleatoires & Laboratoire Jacques-Louis Lions ; Universite Paris Diderot ; 75205聽 ; Paris Cedex 13 ; France
  
• 关键词：Gaussian process regression ; Asymptotic mean squared error ; Learning curves ; Generalization error ; Convergence rate
• 刊名：Machine Learning
• 出版年：2015
• 出版时间：March 2015
• 年：2015
• 卷：98
• 期：3
• 页码：407-433
• 全文大小：459 KB
• 参考文献：1. Abramowitz, M., & Stegun, I. A. (1965). / Handbook of mathematical functions. New York: Dover.
  2. Berger, J. O., De Oliveira, V., & Sans, B. (2001). Objective bayesian analysis of spatially correlated data objective bayesian analysis of spatially correlated data. / Journal of the American Statistical Association, / 96, 1361鈥?374. CrossRef
  3. Bozzini, M., & Rossini, M. (2003). Numerical differentiation of 2d functions from noisy data. / Computer and Mathematics with Applications, / 45, 309鈥?27. CrossRef
  4. Bronski, J. C. (2003). Asymptotics of Karhunen鈥揕o猫ve eigenvalues and tight constants for probability distributions of passive scalar transport. / Communications in Mathematical Physics, / 238, 563鈥?82. CrossRef
  5. Fang, K. T., Li, R., & Sudjianto, A. (2006). / Design and modeling for computer experiments. Computer science and data analysis series. London: Chapman & Hall.
  6. Fernex, F., Heulers, L., Jacquet, O., Miss, J., Richet, Y. (2005). The Moret 4b monte carlo code new features to treat complex criticality systems. In: MandC International Conference on Mathematics and Computation Supercomputing, Reactor and Nuclear and Biological Application, Avignon, France.
  7. Gneiting, T., Kleiber, W., & Schlater, M. (2010). Mat茅rn cross-covariance functions for multivariate random fields. / Journal of the American Statistical Association, / 105, 1167鈥?177. CrossRef
  8. Harville, D. A. (1997). / Matrix algebra from statistician鈥檚 perspective. New York: Springer-Verlag. CrossRef
  9. Laslett, G. M. (1994). Kriging and splines: An empirical comparison of their predictive performance in some applications kriging and splines: An empirical comparison of their predictive performance in some applications. / Journal of the American Statistical Association, / 89, 391鈥?00. CrossRef
  10. Mercer, J. (1909). Functions of positive and negative type and their connection with the theory of integral equations. / Philosophical Transactions of the Royal Society A, / 209, 441鈥?58. CrossRef
  11. Nazarov, A. I., & Nikitin, Y. Y. (2004). Exact \(\text{ l }_2\) -small ball behaviour of integrated Gaussian processes and spectral asymptotics of boundary value problems. / Probability Theory and Related Fields, / 129, 469鈥?94. CrossRef
  12. Opper, M., & Vivarelli, F. (1999). General bounds on Bayes errors for regression with Gaussian processes. / Advances in Neural Information Processing Systems, / 11, 302鈥?08.
  13. Picheny, V. (2009). / Improving accuracy and compensating for uncertainty in surrogate modeling. PhD thesis, Ecole Nationale Sup茅rieure des Mines de Saint Etienne.
  14. Pusev, R. S. (2011). Small deviation asymptotics for Mat茅rn processes and fields under weighted quadratic norm. / Theory of Probability and its Applications, / 55, 164鈥?72.
  15. Rasmussen, C. E., & Williams, C. K. I. (2006). / Gaussian processes for machine learning. Cambridge: MIT Press.
  16. Ritter, K. (2000a). Almost optimal differentiation using noisy data. / Journal of Approximation Theory, / 86, 293鈥?09. CrossRef
  17. Ritter, K. (2000b). / Average-case analysis of numerical problems. Berlin: Springer Verlag. CrossRef
  18. Sacks, J., & Ylvisaker, D. (1981). Variance estimation for approximately linear models. / Series Statistics, / 12, 147鈥?62. CrossRef
  19. Sacks, J., Welch, W. J., Mitchell, T. J., & Wynn, H. P. (1989). Design and analysis of computer experiments. / Statistical Science, / 4, 409鈥?23. CrossRef
  20. Seeger, M. W., Kakade, S. M., & Foster, D. P. (2008). Information consistency of nonparametric Gaussian process methods. / IEEE Transactions on Information Theory, / 54(5), 2376鈥?382.
  21. Shanno, D. F. (1970). Conditioning of quasi-Newton methods for function minimization. / Mathematics of Computation, / 24, 647鈥?56. CrossRef
  22. Sollich, P., & Halees, A. (2002). Learning curves for Gaussian process regression: Approximations and bounds. / Neural Computation, / 14, 1393鈥?428. CrossRef
  23. Stein, M. L. (1999). / Interpolation of spatial data. Series in statistics. New York: Springer. CrossRef
  24. van der Vaart, A., & van Zanten, H. (2011). Information rates of nonparametric Gaussian process methods. / The Journal of Machine Learning Research, / 12, 2095鈥?119.
  25. Wackernagel, H. (2003). / Multivariate geostatistics. Berlin: Springer-Verlag. CrossRef
  26. Williams, C. K. I., & Vivarelli, F. (2000). Upper and lower bounds on the learning curve for Gaussian processes. / Machine Learning, / 40, 77鈥?02. CrossRef
  
• 刊物类别：Computer Science
• 刊物主题：Artificial Intelligence and Robotics
  Automation and Robotics
  Computing Methodologies
  Simulation and Modeling
  Language Translation and Linguistics
  
• 出版者：Springer Netherlands
• ISSN：1573-0565

文摘

This paper deals with the learning curve in a Gaussian process regression framework. The learning curve describes the generalization error of the Gaussian process used for the regression. The main result is the proof of a theorem giving the generalization error for a large class of correlation kernels and for any dimension when the number of observations is large. From this theorem, we can deduce the asymptotic behavior of the generalization error when the observation error is small. The presented proof generalizes previous ones that were limited to special kernels or to small dimensions (one or two). The theoretical results are applied to a nuclear safety problem.