Adaptive Rejection Metropolis Simulated Annealing for Detecting Global Maximum Regions

详细信息    查看全文

• 作者：Huaiye Zhang ; Inyoung Kim
• 关键词：Adaptive rejection metropolis sampling ; Expectation Maximization algorithm ; Finite mixture models ; Simulated annealing
• 刊名：Methodology and Computing in Applied Probability
• 出版年：2016
• 出版时间：March 2016
• 年：2016
• 卷：18
• 期：1
• 页码：1-19
• 全文大小：2,647 KB
• 参考文献：Böhning D, Dietz E, Schlattmann P, Mendonca L, Kirchner U (1995) The zero-inflated Poisson model and the decayed, missing and filled teeth index in dental epidemiology. J R Stat Soc Ser A 162:194–209
  Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc Ser B 39:1–38MathSciNet MATH
  Duncan B (1999) Modeling charitable contributions of time and money. J Public Econ 77:213–242CrossRef
  Gilks WR, Wild P (1992) Adaptive rejection sampling for Gibbs sampling. Appl Stat 41:337–348CrossRef MATH
  Gilks WR, Best NG, Tan KKC (1995) Adaptive rejection Metropolis sampling. Appl Stat 44:455–472CrossRef MATH
  Hastings WK (1970) Monte Carlo sampling methods using Markove chains and their applications. Biometrika 57:97–109CrossRef MATH
  Kirkpatrick S, Gelatt CD, Vecchi MP (1983) Optimization by simulated annealing. Sci New Ser 220:671–680MathSciNet MATH
  McLachlan G, Peel D (2000) Finite mixture models. Wiley, New YorkCrossRef MATH
  Metropolis M, Rosenbluth AW, Rosenbluth MN, Teller AH, Teller E (1953) Equation of state calculations by fast computing machines. J Chem Phys 21:1087–1093CrossRef
  Park T, Park S (2004) An economic study on charitable giving of individuals in Korea: some new findings from 2002 survey data. Presented at the 6th ISTR conference, Toronto, Canada, 11–14 July 2004
  Ripley BD (1987) Stochastic simulation. Wiley, New YorkCrossRef MATH
  Smith VH, Kchoe MR, Creamer ME (1999) The private provision of public goods: altruism and voluntary giving. J Public Econ 77:213–242
  Swendsen RH, Wang JS (1986) Replica Monte Carlo simulation of spin glasses. Phys Rev Lett 57:2607–2609CrossRef MathSciNet
  van den Broek J (1995) A score test for zero inflated in a Poisson regression. Biometrics 51:738–743CrossRef MathSciNet MATH
  von Neumann J (1951) Various techniques used in connection with random digits: Monte Carlo methods. Natl Bur Stan AMS 12:36–38
  
• 作者单位：Huaiye Zhang (1)
  Inyoung Kim (1)
  
  1. Department of Statistics, Virginia Polytechnic Institute and State University, Blacksburg, VA, 24061, USA
  
• 刊物主题：Statistics, general; Life Sciences, general; Electrical Engineering; Economics general; Business/Management Science, general;
• 出版者：Springer US
• ISSN：1573-7713

文摘

A finite mixture model has been used to fit the data from heterogeneous populations to many applications. An Expectation Maximization (EM) algorithm is the most popular method to estimate parameters in a finite mixture model. A Bayesian approach is another method for fitting a mixture model. However, the EM algorithm often converges to the local maximum regions, and it is sensitive to the choice of starting points. In the Bayesian approach, the Markov Chain Monte Carlo (MCMC) sometimes converges to the local mode and is difficult to move to another mode. Hence, in this paper we propose a new method to improve the limitation of EM algorithm so that the EM can estimate the parameters at the global maximum region and to develop a more effective Bayesian approach so that the MCMC chain moves from one mode to another more easily in the mixture model. Our approach is developed by using both simulated annealing (SA) and adaptive rejection metropolis sampling (ARMS). Although SA is a well-known approach for detecting distinct modes, the limitation of SA is the difficulty in choosing sequences of proper proposal distributions for a target distribution. Since ARMS uses a piecewise linear envelope function for a proposal distribution, we incorporate ARMS into an SA approach so that we can start a more proper proposal distribution and detect separate modes. As a result, we can detect the maximum region and estimate parameters for this global region. We refer to this approach as ARMS annealing. By putting together ARMS annealing with the EM algorithm and with the Bayesian approach, respectively, we have proposed two approaches: an EM-ARMS annealing algorithm and a Bayesian-ARMS annealing approach. We compare our two approaches with traditional EM algorithm alone and Bayesian approach alone using simulation, showing that our two approaches are comparable to each other but perform better than EM algorithm alone and Bayesian approach alone. Our two approaches detect the global maximum region well and estimate the parameters in this region. We demonstrate the advantage of our approaches using an example of the mixture of two Poisson regression models. This mixture model is used to analyze a survey data on the number of charitable donations.