Smooth strongly convex interpolation and exact worst-case performance of first-order methods

详细信息    查看全文

• 作者：Adrien B. Taylor ; Julien M. Hendrickx ; François Glineur
• 关键词：Smooth convex minimization ; Smooth convex interpolation ; First ; order methods ; Worst ; case analysis ; Rates of convergence ; Semidefinite programming
• 刊名：Mathematical Programming
• 出版年：2017
• 出版时间：January 2017
• 年：2017
• 卷：161
• 期：1-2
• 页码：307-345
• 全文大小：
• 刊物类别：Mathematics and Statistics
• 刊物主题：Calculus of Variations and Optimal Control; Optimization; Mathematics of Computing; Numerical Analysis; Combinatorics; Theoretical, Mathematical and Computational Physics; Mathematical Methods in Phys
• 出版者：Springer Berlin Heidelberg
• ISSN：1436-4646
• 卷排序：161

文摘

We show that the exact worst-case performance of fixed-step first-order methods for unconstrained optimization of smooth (possibly strongly) convex functions can be obtained by solving convex programs. Finding the worst-case performance of a black-box first-order method is formulated as an optimization problem over a set of smooth (strongly) convex functions and initial conditions. We develop closed-form necessary and sufficient conditions for smooth (strongly) convex interpolation, which provide a finite representation for those functions. This allows us to reformulate the worst-case performance estimation problem as an equivalent finite dimension-independent semidefinite optimization problem, whose exact solution can be recovered up to numerical precision. Optimal solutions to this performance estimation problem provide both worst-case performance bounds and explicit functions matching them, as our smooth (strongly) convex interpolation procedure is constructive. Our works build on those of Drori and Teboulle (Math Program 145(1–2):451–482, 2014) who introduced and solved relaxations of the performance estimation problem for smooth convex functions. We apply our approach to different fixed-step first-order methods with several performance criteria, including objective function accuracy and gradient norm. We conjecture several numerically supported worst-case bounds on the performance of the fixed-step gradient, fast gradient and optimized gradient methods, both in the smooth convex and the smooth strongly convex cases, and deduce tight estimates of the optimal step size for the gradient method.