Sparse estimators and the oracle property, or the return of Hodges’ estimator
详细信息 查看全文
- 作者：Hannes Leeb ; Benedikt M. Pö ; tscher
- 关键词：Oracle property ; Sparsity ; Penalized maximum likelihood ; Penalized least squares ; Hodges’ ; estimator ; SCAD ; Lasso ; Bridge estimator ; Hard-thresholding ; Maximal risk ; Maximal absolute bias ; Nonuniform limits
- 刊名：Journal of Econometrics
- 出版年：2008
- 出版时间：January 2008
- 年：2008
- 卷：142
- 期：1
- 页码：201-211
- 全文大小：231 K
文摘
We point out some pitfalls related to the concept of an oracle property as used in Fan and Li [2001. Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association 96, 1348–1360; 2002. Variable selection for Cox's proportional hazards model and frailty model. Annals of Statistics 30, 74–99; 2004. New estimation and model selection procedures for semiparametric modeling in longitudinal data analysis. Journal of the American Statistical Association 99, 710–723] which are reminiscent of the well-known pitfalls related to Hodges’ estimator. The oracle property is often a consequence of sparsity of an estimator. We show that any estimator satisfying a sparsity property has maximal risk that converges to the supremum of the loss function; in particular, the maximal risk diverges to infinity whenever the loss function is unbounded. For ease of presentation the result is set in the framework of a linear regression model, but generalizes far beyond that setting. In a Monte Carlo study we also assess the extent of the problem in finite samples for the smoothly clipped absolute deviation (SCAD) estimator introduced in Fan and Li [2001. Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association 96, 1348–1360]. We find that this estimator can perform rather poorly in finite samples and that its worst-case performance relative to maximum likelihood deteriorates with increasing sample size when the estimator is tuned to sparsity.