Practical inexact proximal quasi-Newton method with global complexity analysis
详细信息 查看全文
- 作者：Katya Scheinberg ; Xiaocheng Tang
- 关键词：Convex optimization ; Proximal Newton methods ; Convergence rates ; Coordinate descent ; Quasi ; Newton methods
- 刊名：Mathematical Programming
- 出版年：2016
- 出版时间：November 2016
- 年：2016
- 卷：160
- 期：1-2
- 页码：495-529
- 全文大小：814 KB
- 刊物类别：Mathematics and Statistics
- 刊物主题：Mathematics
Calculus of Variations and Optimal Control
Mathematics of Computing
Numerical Analysis
Combinatorics
Mathematical and Computational Physics
Mathematical Methods in Physics - 出版者：Springer Berlin / Heidelberg
- ISSN：1436-4646
- 卷排序：160
文摘
Recently several methods were proposed for sparse optimization which make careful use of second-order information (Hsieh et al. in Sparse inverse covariance matrix estimation using quadratic approximation. In: NIPS, 2011; Yuan et al. in An improved GLMNET for l1-regularized logistic regression and support vector machines. National Taiwan University, Taipei City, 2011; Olsen et al. in Newton-like methods for sparse inverse covariance estimation. In: NIPS, 2012; Byrd et al. in A family of second-order methods for convex l1-regularized optimization. Technical report, 2012) to improve local convergence rates. These methods construct a composite quadratic approximation using Hessian information, optimize this approximation using a first-order method, such as coordinate descent and employ a line search to ensure sufficient descent. Here we propose a general framework, which includes slightly modified versions of existing algorithms and also a new algorithm, which uses limited memory BFGS Hessian approximations, and provide a novel global convergence rate analysis, which covers methods that solve subproblems via coordinate descent.