Bayesian structured variable selection in linear regression models

详细信息    查看全文

• 作者：Min Wang (1)
  Xiaoqian Sun (2)
  Tao Lu (3)
  
  1. Department of Mathematical Sciences ; Michigan Technological University ; Houghton ; MI ; 49931 ; USA
  2. Department of Mathematical Sciences ; Clemson University ; Clemson ; SC ; 29634 ; USA
  3. Department of Epidemiology and Biostatistics ; State University of New York ; Albany ; NY ; 12144 ; USA
  
• 关键词：Interactions ; Generalized singular $$g$$ g ; prior ; Beta ; prime prior ; Posterior probability ; Gibbs sampler ; Consistency
• 刊名：Computational Statistics
• 出版年：2015
• 出版时间：March 2015
• 年：2015
• 卷：30
• 期：1
• 页码：205-229
• 全文大小：399 KB
• 参考文献：1. Baragatti, M, Pommeret, D (2012) A study of variable selection using g-prior distribution with ridge parameter. Comput Stat Data Anal 56: pp. 1920-1934 CrossRef
  2. Barbieri, MM, Berger, JO (2004) Optimal predictive model selection. Ann Stat 32: pp. 870-897 3604000000238" target="_blank" title="It opens in new window">CrossRef
  3. Bartlett, M (1957) A comment on D.V. Lindley鈥檚 statistical paradox. Biometrika 44: pp. 533-534 3/biomet/44.3-4.533" target="_blank" title="It opens in new window">CrossRef
  4. Breiman, L, Friedman, JH (1985) Estimating optimal transformations for multiple regression and correlation. J Am Stat Assoc 80: pp. 580-619 CrossRef
  5. Brown, PJ, Vannucci, M, Fearn, T (1998) Multivariate Bayesian variable selection and prediction. J R Stat Soc Ser B 60: pp. 627-641 CrossRef
  6. Casella, G, Moreno, E (2006) Objective Bayesian variable selection. J Am Stat Assoc 101: pp. 157-167 CrossRef
  7. Chib, S (1995) Marginal likelihood from the Gibbs output. J Am Stat Assoc 90: pp. 1313-1321 35" target="_blank" title="It opens in new window">CrossRef
  8. Chipman, H (1996) Bayesian variable selection with related predictors. Can J Stat 24: pp. 17-36 307/3315687" target="_blank" title="It opens in new window">CrossRef
  9. Chipman, H, Hamada, M, Wu, C (1997) A Bayesian variable-selection approach for analyzing designed experiments with complex aliasing. Technometrics 39: pp. 372-381 CrossRef
  10. Cui, W, George, EI (2008) Empirical Bayes vs. fully Bayes variable selection. J Stat Plan Inference 138: pp. 888-900 CrossRef
  11. Farcomeni, A (2010) Bayesian constrained variable selection. Stat Sin 20: pp. 1043-1062
  12. Fern谩ndez, C, Ley, E, Steel, MFJ (2001) Benchmark priors for Bayesian model averaging. J Econom 100: pp. 381-427 304-4076(00)00076-2" target="_blank" title="It opens in new window">CrossRef
  13. Foster, DP, George, EI (1994) The risk inflation criterion for multiple regression. Ann Stat 22: pp. 1947-1975 325766" target="_blank" title="It opens in new window">CrossRef
  14. George, E, McCulloch, R (1993) Variable selection via Gibbs sampling. J Am Stat Assoc 22: pp. 881-889 3.10476353" target="_blank" title="It opens in new window">CrossRef
  15. George, E, McCulloch, R (1997) Approaches for Bayesian variable selection. Stat Sin 7: pp. 339-373
  16. George, EI, Foster, DP (2000) Calibration and empirical Bayes variable selection. Biometrika 87: pp. 731-747 3/biomet/87.4.731" target="_blank" title="It opens in new window">CrossRef
  17. Geweke J (1992) Evaluating the accuracy of sampling-based approaches to the calculation of posterior moments. Bayesian Statistics, 4 (Pe帽铆scola, 1991). Oxford University Press, New York, pp 169鈥?93
  18. Guo R, Speckman PL (2009) Bayes factor consistency in linear models. In: 2009 international workshop on objective bayes methodology, Philadelphia, June 5鈥?, 2009
  19. Kass, RE, Raftery, AE (1995) Bayes factors. J Am Stat Assoc 90: pp. 773-795 CrossRef
  20. Lamnisos, D, Griffin, JE, Steel, MFJ (2009) Transdimensional sampling algorithms for Bayesian variable selection in classification problems with many more variables than observations. J Comput Graph Stat 18: pp. 592-612 CrossRef
  21. Liang, F, Paulo, R, Molina, G, Clyde, MA, Berger, JO (2008) Mixtures of $$g$$ g priors for Bayesian variable selection. J Am Stat Assoc 103: pp. 410-423 337" target="_blank" title="It opens in new window">CrossRef
  22. Maruyama Y (2009) A Bayes factor with reasonable model selection consistency for ANOVA model. 329v1" class="a-plus-plus">arXiv:0906.4329v1 [stat.ME]
  23. Maruyama, Y, George, EI (2011) Fully Bayes factors with a generalized g-prior. Ann. Stat. 39: pp. 2740-2765 CrossRef
  24. Maruyama Y, Strawderman WE (2010) Robust Bayesian variable selection with sub-harmonic priors. 3" class="a-plus-plus">arXiv:1009.1926v3 [stat.ME]
  25. Nelder, J (1994) The statistics of linear models: back to basics. Stat Comput 4: pp. 221-234 CrossRef
  26. Panagiotelis, A, Smith, M (2008) Bayesian identification, selection and estimation of semiparametric functions in high-dimensional additive models. J Econom 143: pp. 291-316 3" target="_blank" title="It opens in new window">CrossRef
  27. Raftery, A, Madigan, D, Hoeting, J (1997) Bayesian model averaging for linear regression models. J Am Stat Assoc 92: pp. 179-191 3615" target="_blank" title="It opens in new window">CrossRef
  28. Raftery, AE, Lewis, SM (1992) One long run with diagnostics: implementation strategies for Markov chain Monte Carlo. Stat Sci 7: pp. 493-497 3" target="_blank" title="It opens in new window">CrossRef
  29. Smith, M, Kohn, R (1996) Nonparametric regression using Bayesian variable selection. J Econom 75: pp. 317-343 304-4076(95)01763-1" target="_blank" title="It opens in new window">CrossRef
  30. Song, X, Lu, Z (2011) Response to 鈥淐omments on 鈥楤ayesian variable selection for disease classification using gene expression data鈥?鈥? Bioinformatics 27: pp. 2169-2170 3/bioinformatics/btr334" target="_blank" title="It opens in new window">CrossRef
  31. Wang, M, Sun, X (2013) Bayes factor consistency for unbalanced ANOVA models. Stat A J Theor Appl Stat 47: pp. 1104-1115
  32. West, M (2003) Bayesian factor regression models in the 鈥渓arge p, small n鈥?paradigm. Bayesian Stat 7: pp. 723-732
  33. Yang, A, Song, X (2010) Bayesian variable selection for disease classification using gene expression data. Bioinformatics 26: pp. 215-222 3/bioinformatics/btp638" target="_blank" title="It opens in new window">CrossRef
  34. Yuan, M, Joseph, V, Zou, H (2009) Structured variable selection and estimation. Ann Appl Stat 3: pp. 1738-1757 CrossRef
  35. Zellner A (1986) On assessing prior distributions and Bayesian regression analysis with \(g\) -prior distributions. In: Goel PK, Zellner A (eds) Bayesian inference and decision techniques, Studies in Bayesian Econometrics and Statistics. North-Holland, Amsterdam, vol. 6, pp 233鈥?43
  
• 刊物类别：Mathematics and Statistics
• 刊物主题：Mathematics
  Statistics
  Statistics
  Probability and Statistics in Computer Science
  Probability Theory and Stochastic Processes
  Economic Theory
  
• 出版者：Physica Verlag, An Imprint of Springer-Verlag GmbH
• ISSN：1613-9658

文摘

In this paper we consider the Bayesian approach to the problem of variable selection in normal linear regression models with related predictors. We adopt a generalized singular \(g\) -prior distribution for the unknown model parameters and the beta-prime prior for the scaling factor \(g\) , which results in a closed-form expression of the marginal posterior distribution without integral representation. A special prior on the model space is then advocated to reflect and maintain the hierarchical or structural relationships among predictors. It is shown that under some nominal assumptions, the proposed approach is consistent in terms of model selection and prediction. Simulation studies show that our proposed approach has a good performance for structured variable selection in linear regression models. Finally, a real-data example is analyzed for illustrative purposes.