Scaling Analysis of Delayed Rejection MCMC Methods

详细信息    查看全文

• 作者：Myl猫ne B茅dard (1)
  Randal Douc (2)
  Eric Moulines (3)
  
• 关键词：Random walk Metropolis ; Weak convergence ; Diffusion ; Correlated聽proposals ; Multiple proposals ; Primary 60F05 ; Secondary 65C40
• 刊名：Methodology and Computing in Applied Probability
• 出版年：2014
• 出版时间：December 2014
• 年：2014
• 卷：16
• 期：4
• 页码：811-838
• 全文大小：767 KB
• 参考文献：1. B茅dard M (2007) Weak convergence of Metropolis algorithms for non-i.i.d. target distributions. Ann Appl Probab 17:1222鈥?244 CrossRef
  2. B茅dard M, Rosenthal JS (2008) Optimal scaling of Metropolis algorithms: heading toward general target distributions. Can J Stat 36:483鈥?03 CrossRef
  3. B茅dard M, Douc R, Moulines E (2012) Scaling analysis of multiple-try MCMC methods. Stoch Process Their Appl 122:758鈥?86 CrossRef
  4. Beskos A, Roberts G, Stuart A (2009) Optimal scalings for local Metropolis鈥揌astings chains on nonproduct targets in high dimensions. Ann Appl Probab 19:863鈥?98 CrossRef
  5. Billingsley P (1999) Convergence of probability measures, 2nd edn. Wiley Series in Probability and Statistics: Probability and Statistics. Wiley, New York. A Wiley-Interscience Publication CrossRef
  6. Craiu RV, Lemieux C (2007) Acceleration of the multiple-try Metropolis algorithm using antithetic and stratified sampling. Stat Comput 17:109鈥?20 CrossRef
  7. Gelfand SB, Mitter S (1991) Weak convergence of Markov chain sampling methods and annealing algorithms to diffusions. J Optim Theory Appl 68:483鈥?98 CrossRef
  8. Green PJ, Mira A (2001) Delayed rejection in reversible jump Metropolis鈥揌astings. Biometrika 88:1035鈥?053 CrossRef
  9. Haario H, Laine M, Mira A, Saksman E (2006) DRAM: efficient adaptive MCMC. Stat Comput 16:339鈥?54 CrossRef
  10. Harkness MA, Green PJ (2000) Parallel chains, delayed rejection and reversible jump MCMC for object recognition. In: British machine vision conference
  11. Mattingly J, Pillai N, Stuart A (2012) Diffusion limits of random walk Metropolis in high dimensions. Ann Appl Probab 22:881鈥?30 CrossRef
  12. Mira A (2001a) On Metropolis鈥揌astings algorithms with delayed rejection. Metron LIX:231鈥?41
  13. Mira A (2001b) Ordering and improving the performance of Monte Carlo Markov chains. Stat Sci 16:340鈥?50 CrossRef
  14. Raggi D (2005) Adaptive MCMC for inference on affine stochastic volatility models with jumps. Econom J 8:235鈥?50 CrossRef
  15. Robert CP, Casella G (2004) Monte Carlo statistical methods, 2nd edn. Springer, New York CrossRef
  16. Roberts GO, Rosenthal JS (1998) Markov chain Monte Carlo: some practical implications of theoretical results. Can J Stat 26:5鈥?2 CrossRef
  17. Roberts GO, Rosenthal JS (2001) Optimal scaling for various Metropolis鈥揌astings algorithms. Stat Sci 16:351鈥?67 CrossRef
  18. Roberts GO, Gelman A, Gilks WR (1997) Weak convergence and optimal scaling of random walk Metropolis algorithms. Ann Appl Probab 7:110鈥?20 CrossRef
  19. Tierney LL, Mira AA (1999) Some adaptive Monte Carlo methods for Bayesian inference. Stat Med 18:2507鈥?515 CrossRef
  20. Trias M, Vecchio A, Veitch J (2009) Delayed rejection schemes for efficient Markov-Chain Monte-Carlo sampling of multimodal distributions. ArXiv e-prints
  21. Umst盲tter R, Meyer R, Dupuis R, Veitch J, Woan G, Christensen N (2004) Estimating the parameters of gravitational waves from neutron stars using an adaptive MCMC method. Class Quantum Gravity 21:1655鈥?675 CrossRef
  
• 作者单位：Myl猫ne B茅dard (1)
  Randal Douc (2)
  Eric Moulines (3)
  
  1. D茅partement de Math茅matiques et de Statistique, Universit茅 de Montr茅al, Montr茅al, QC, H3C聽3J7, Canada
  2. SAMOVAR, CNRS UMR 5157 - Institut T茅l茅com/T茅l茅com SudParis, 9 rue Charles Fourier, 91000, Evry, France
  3. LTCI, CNRS UMR 8151 - Institut T茅l茅com /T茅l茅com ParisTech, 46, rue Barrault, 75634, Paris Cedex 13, France
  
• ISSN：1573-7713

文摘

In this paper, we study the asymptotic efficiency of the delayed rejection strategy. In particular, the efficiency of the delayed rejection Metropolis鈥揌astings algorithm is compared to that of the regular Metropolis algorithm. To allow for a fair comparison, the study is carried under optimal mixing conditions for each of these algorithms. After introducing optimal scaling results for the delayed rejection (DR) algorithm, we outline the fact that the second proposal after the first rejection is discarded, with a probability tending to 1 as the dimension of the target density increases. To overcome this drawback, a modification of the delayed rejection algorithm is proposed, in which the direction of the different proposals is fixed once for all, and the Metropolis鈥揌astings accept-reject mechanism is used to select a proper scaling along the search direction. It is shown that this strategy significantly outperforms the original DR and Metropolis algorithms, especially when the dimension becomes large. We include numerical studies to validate these conclusions.