High dimensional robust M-estimation: asymptotic variance via approximate message passing

详细信息    查看全文

• 作者：David Donoho ; Andrea Montanari
• 关键词：Mathematics Subject Classification62F10 ; 62F12 ; 60F99
• 刊名：Probability Theory and Related Fields
• 出版年：2016
• 出版时间：December 2016
• 年：2016
• 卷：166
• 期：3-4
• 页码：935-969
• 全文大小：810 KB
• 刊物类别：Mathematics and Statistics
• 刊物主题：Mathematics
  Probability Theory and Stochastic Processes
  Mathematical and Computational Physics
  Quantitative Finance
  Mathematical Biology
  Statistics for Business, Economics, Mathematical Finance and Insurance
  Operation Research and Decision Theory
  
• 出版者：Springer Berlin / Heidelberg
• ISSN：1432-2064
• 卷排序：166

文摘

In a recent article, El Karoui et al. (Proc Natl Acad Sci 110(36):14557–14562, 2013) study the distribution of robust regression estimators in the regime in which the number of parameters p is of the same order as the number of samples n. Using numerical simulations and ‘highly plausible’ heuristic arguments, they unveil a striking new phenomenon. Namely, the regression coefficients contain an extra Gaussian noise component that is not explained by classical concepts such as the Fisher information matrix. We show here that that this phenomenon can be characterized rigorously using techniques that were developed by the authors for analyzing the Lasso estimator under high-dimensional asymptotics. We introduce an approximate message passing (AMP) algorithm to compute M-estimators and deploy state evolution to evaluate the operating characteristics of AMP and so also M-estimates. Our analysis clarifies that the ‘extra Gaussian noise’ encountered in this problem is fundamentally similar to phenomena already studied for regularized least squares in the setting \(n<p\).