几类谱共轭梯度方法理论及数值行为研究
|
摘要
谱共轭梯度算法的研究迄今虽已取得较丰富成果,但如何更合理选择谱系数和共轭系数,以保持谱方法和共轭梯度方法的计算优点,仍值得深入探讨。此外,合适的线搜索技术对算法效率和建立其收敛性理论亦至关重要。针对这些问题,本文对几类新的谱共轭梯度方法的理论及数值行为开展了较广泛和深入的研究,主要研究工作如下：
1、鉴于Armijo线搜索可能生成较粗糙步长,我们基于近似Wolfe条件,提出了一种修正的Armijo线搜索规则。结合新的线搜索规则,本文设计了一类求解无约束优化问题的具有充分下降性的谱PRP共轭梯度算法。数值实验表明该算法是有效的。
2、研究了一类新的非单调谱共轭梯度方法。一方面,该方法通过引入混合因子,将HS方法和PRP方法结合提出了共轭系数的新的选取方式。通过合适选取谱系数,我们证明了所得搜索方向不依赖于线搜索条件恒为充分下降方向。另一方面,该方法还修正了Zhang和Hager提出的非单调线搜索规则,在更弱的假设条件下证明了全局收敛性。数值试验说明了该方法的数值计算性能优良。
3、给出了一种新的求解无约束优化问题的修正谱FR共轭梯度法。由此方法产生的搜索方向是充分下降方向；通过引入一调节系数,调节谱系数和共轭系数,以保证新方法尽量兼备谱方法和共轭梯度法的优越性。结合Wolfe型线搜索,我们给出了算法的全局收敛的证明。数值实验证实了算法的有效性。
4、针对一系列具有充分下降的搜索方法,我们抽象了谱共轭梯度法的一般迭代格式,给出了谱系数和共轭系数的选择范围,以保证搜索方向是充分下降方向。对基于标准Armijo线搜索的任何其他线搜索条件,建立该一般迭代格式的全局收敛性理论。
Although there has existed a great contribution in the research of spectral conjugate gradient methods up to now, it is significant to further investigate how to choose a suitable spectral parameter and a conjugate parameter such that the obtained method has the advantages owned by the spectral method and the conjugate gradient method. On the other hand, a suitable line search technique is important for the numerical performance of the developed algorithm and the establishment of global convergence. For these issues, in this dissertation, we intend to conduct an extensive and deep research on some types of spectral conjugate gradient methods from the viewpoints of theory and numerical performance. The main contributions are as follows:
Since it is possible that the obtained step size by the standard Armijo line search is coarse, we first employ approximate conditions of Wolfe line search to improve the Armijo line search rule. On the basis of this new line search, a new spectral PRP conjugate gradient algorithm is developed for solving unconstrained optimization problems, where the search direction is sufficiently descent. Numerical experiments show that the developed algorithm is promising.
Then, a new nonmonotone spectral conjugate gradient method is proposed. By introducing a hybrid coefficient, the conjugacy parameter is determined based on the combination of PRP and HS methods. A spectral parameter is appropriately chosen such that each search direction is a sufficiently descent direction independent of the employed line search techniques. On the other hand, the nonomonotone line search technique proposed by Zhang and Hager is modified, and under more mild assumptions, the global convergence of the developed algorithm is proved. Numerical experiments are employed to demonstrate the efficiency of the algorithm.
Thirdly, a new modified spectral FR conjugate gradient method is presented for solving unconstrained optimization problems. The direction generated by the method produces sufficiently descent search direction. By introducing a control parameter to regulate the spectral coefficient and conjugate coefficient, the new method is assured to have the superiority of spectral method and conjugate gradient method. Global convergence results is established for the conjugate gradient methods with a Wolfe type line search. The numerical results show that the new method is efficient for general unconstrained optimization problems.
Finally, based on a series of methods with sufficiently descent direction, a generic iterative formula of spectral conjugate gradient method is summarized. To make sure that the search direction is sufficiently descent direction, the value range of spectral and conjugate parameters is presented. With any line search conditions based on standard Armijo line search, the global convergence theory is established for this generic iterative formula.
1、鉴于Armijo线搜索可能生成较粗糙步长,我们基于近似Wolfe条件,提出了一种修正的Armijo线搜索规则。结合新的线搜索规则,本文设计了一类求解无约束优化问题的具有充分下降性的谱PRP共轭梯度算法。数值实验表明该算法是有效的。
2、研究了一类新的非单调谱共轭梯度方法。一方面,该方法通过引入混合因子,将HS方法和PRP方法结合提出了共轭系数的新的选取方式。通过合适选取谱系数,我们证明了所得搜索方向不依赖于线搜索条件恒为充分下降方向。另一方面,该方法还修正了Zhang和Hager提出的非单调线搜索规则,在更弱的假设条件下证明了全局收敛性。数值试验说明了该方法的数值计算性能优良。
3、给出了一种新的求解无约束优化问题的修正谱FR共轭梯度法。由此方法产生的搜索方向是充分下降方向；通过引入一调节系数,调节谱系数和共轭系数,以保证新方法尽量兼备谱方法和共轭梯度法的优越性。结合Wolfe型线搜索,我们给出了算法的全局收敛的证明。数值实验证实了算法的有效性。
4、针对一系列具有充分下降的搜索方法,我们抽象了谱共轭梯度法的一般迭代格式,给出了谱系数和共轭系数的选择范围,以保证搜索方向是充分下降方向。对基于标准Armijo线搜索的任何其他线搜索条件,建立该一般迭代格式的全局收敛性理论。
Although there has existed a great contribution in the research of spectral conjugate gradient methods up to now, it is significant to further investigate how to choose a suitable spectral parameter and a conjugate parameter such that the obtained method has the advantages owned by the spectral method and the conjugate gradient method. On the other hand, a suitable line search technique is important for the numerical performance of the developed algorithm and the establishment of global convergence. For these issues, in this dissertation, we intend to conduct an extensive and deep research on some types of spectral conjugate gradient methods from the viewpoints of theory and numerical performance. The main contributions are as follows:
Since it is possible that the obtained step size by the standard Armijo line search is coarse, we first employ approximate conditions of Wolfe line search to improve the Armijo line search rule. On the basis of this new line search, a new spectral PRP conjugate gradient algorithm is developed for solving unconstrained optimization problems, where the search direction is sufficiently descent. Numerical experiments show that the developed algorithm is promising.
Then, a new nonmonotone spectral conjugate gradient method is proposed. By introducing a hybrid coefficient, the conjugacy parameter is determined based on the combination of PRP and HS methods. A spectral parameter is appropriately chosen such that each search direction is a sufficiently descent direction independent of the employed line search techniques. On the other hand, the nonomonotone line search technique proposed by Zhang and Hager is modified, and under more mild assumptions, the global convergence of the developed algorithm is proved. Numerical experiments are employed to demonstrate the efficiency of the algorithm.
Thirdly, a new modified spectral FR conjugate gradient method is presented for solving unconstrained optimization problems. The direction generated by the method produces sufficiently descent search direction. By introducing a control parameter to regulate the spectral coefficient and conjugate coefficient, the new method is assured to have the superiority of spectral method and conjugate gradient method. Global convergence results is established for the conjugate gradient methods with a Wolfe type line search. The numerical results show that the new method is efficient for general unconstrained optimization problems.
Finally, based on a series of methods with sufficiently descent direction, a generic iterative formula of spectral conjugate gradient method is summarized. To make sure that the search direction is sufficiently descent direction, the value range of spectral and conjugate parameters is presented. With any line search conditions based on standard Armijo line search, the global convergence theory is established for this generic iterative formula.
引文
[1]Yuan Y, Sun W., Theory and methods of optimization. Beijing:Science Press of China,1999.
[2]Raydan M., The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem, SIAM Journal of Optimization,1997,7: 26-33
[3]Schropp J., A note on minimization problems and multistep methods. Numeric Mathematic,1997,78:87-101.
[4]Schropp J., One-step and multistep procedures for constrained minimization problems. IMA Journal of Numerical Analysis,2000,20:135-152.
[5]Van Wyk D. J., Differential optimization techniques. Applied Mathematical Modelling,1984,8:419-424.
[6]Vrahatis M. N., Androulakis G.S., Lambrinos J. N., Magolas G.D., A class of gradient unconstrained minimization algorithms with adaptive stepsize, Journal of Computational and Applied Mathematics,2000,114:367-386
[7]Yuan G. L., Lu X. W., A new line search method with trust region for unconstrained optimization. Communications on Applied Nonlinear Analysis, 2008,15(1):35-49
[8]Raydan M., Svaiter B. F.,Relaxed steepest descent and Cauchy-Barzilai-Borwein method. Computational Optimization and Applications,2002,21:155-167
[9]Horst R., Pardalos P. M., Thoai A. V., Introduction to global optimization. Dordrecht:Kluwer,1995.
[10]Luenerger D. C., Linear and nonlinear programming (2nd ed.). Reading, MA: Addition Wesley,1989.
[11]Nocedal J., Wright S. J., Numerical optimization. Berlin Heidelberg, Springer New York,1999.
[12]赖炎连,贺国平,最优化方法,清华大学出版社,2008
[13]Dennis J.E., More J.J., A characterization of superlinear convergence and its application to quasi-Newton methods. Mathematics of Computational,1974,28: 549-560.
[14]Dennis J. E., Jr., More J. J., Quasi-Newton methods, motivation and theory. SIAM Review,1977,19:46-89.
[15]Fletcher R., Practical method of optimization, vol Ⅰ:Unconstrained optimization (2nd ed.), Wiley,New York,1987.
[16]Griewank A., Toint Ph. L., Local convergence analysis for partitioned quasi-Newton updates. Numeric Mathematic,1982,39:429-448.
[17]Byrd R., Nocedal J., A tool for the analysis of quasi-Newton methods with application to unconstrained minimization. SIAM Journal on Numerical Analysis,1989,26:727-739
[18]Dai Y., Convergence properties of the BFGS algorithm. SIAM Journal on Optimization,2003,13:693-701.
[19]Dennis J. E., Schnabel R. B., Numerical methods for unconstrained optimization and nonlinear equations. Englewood Cliffs, NJ:Prentice-Hall, Inc.,1983.
[20]Li D., Fukushima M., A modified BFGS method and its global convergence in nonconvex minimization. Journal of Computational and Applied Mathematics, 2001,129:15-35.
[21]Wei Z., Li G., Qi L., New quasi-Newton methods for unconstrained optimization problems. Applied Mathematics and Computation,2006,175:1156-1188
[22]Wei Z., Yu G., Yuan G, Lian Z., The superlinear convergence of a modified BFGS-type method for unconstrained optimization. Computational Optimization and Applications,2004,29:315-332
[23]Zhang J., Deng N., Chen L., New quasi-Newton equation and related methods for unconstrained optimization. Journal of Optimization Theory and Applications, 1997,102(1):147-167.
[24]孙文瑜,徐成贤,朱德通,最优化方法,高等教育出版社,2004.
[25]Hestenes M. R.,Stiefel E. L, Method of conjugate gradient for solving linear equations. Journal of Research of the National Bureau Standards,1952,49: 409-436
[26]Fletcher R., Reeves C., Function minimization bu conjugate gradients. Computer Journal,1964,7:149-154.
[27]Polak E., Ribiere G., Note sur la convergence de directions conjugees., Rev. Francaise informat Recherche Operatinelle,3e Annee,1969,16:35-43.
[28]Flecher R., Practical Methods of Optimization,Vol I:unconstrained optimization, New York:John Wiley and Sons,1987.
[29]Liu Y,Storey C. Efficient generalized conjugate gradient algorithms, part 1: theory. Journal of Optimization Theory and Application,1991,69:129-137
[30]Dai Y, Yuan Y., A nonlinear conjugate gradient with a strong global convergence properties, SI AM Journal of Optimization,1999,10:177-182.
[31]Zoutendijk G., Nonlinear programming, computational methods, Integer and Nonlinear Programming, North-Holland, Amsterdam,1970,37-86.
[32]Al-Baali M., Descent property and global convergence of the Fletcher-Revves method with inexact line search. IMA J. Numer. Annal.,1985,5:121-124.
[33]Liu G H., Han J. Y, ed. Global convergence of the Fletcher-Reeves method with inexact line search, Appl. Math. J. Chinese Univ. Ser. B,1995,10:75-82.
[34]Dai Y.H., Yuan Y.X., Convergence properties of the Fletcher-Reeves method, IMA J. Numer. Anal.,1996,16(2):155-164.
[35]Dai Y. H., Yuan Y. X. Convergence of the Fletcher-Reeves method under a generalized Wolfe search. Journal Computational Mathematics of Chinese Universities,1996,2:142-148.
[36]Liu G, Han J., Yin H., Global convergence of the Fletcher-Reeves method with inexact line search, Appl. Math. J. Chin. Univ. Ser. B,1995,10:75-82.
[37]Powell M. J. D. Restart procedures of the conjugate gradient method [123]. Math Program,1977,2:241-254.
[38]Polyak B. T.,The conjugate gradient method in extremem problems [123]. USSR Comp Math and Math Phys,1969,9:94-112.
[39]Powell M. J. D., Nonconvex minimization calculations and the conjugate gradient method,Lecture Notes in Mathematics, Berlin:Springer-Verlag,1984,1066: 122-141.
[40]Yuan Y. X., Analysis on the conjugate gradient method,Optimization Methods and Software,1993,2:19-29.
[41]Gilbert J. C., Nocedal J., Global convergence properties of conjugate gradient methods for optimization. SIAM J. Optim.,1992,2(1):21-42.
[42]Grippo L., Lucidi S., A globally convergent version of the Polak-Ribiere conjugate gradient method. Mathematical Programming,1997,78:375-391.
[43]Wang C., Chen Y, Du S., Further insight into the Shamanskii modification of Newton method, Appl. Math. Comput,2006,180:46-52.
[44]Zhang L., Zhou W. J., Li D. H., A descent modified Polak-Ribiere-Polyak conjugate gradient method and its global convergence,IMA J. Numer. Anal., 2006,26:629-640.
[45]徐成贤,陈志平等,近代优化方法,科学出版社,2002.
[46]戴或虹,袁亚湘.非线性共轭梯度法.上海：上海科学技术出版社,2000.
[47]Dai Y.H., Yuan Y., A nonlinear conjugate gradient method with a strong global convergence property. SIAM J. Optim.,2000,10:177-182.
[48]Dai Y.H., Yuan Y., Convergence properties of the conjugate descent method. Adv. Math.,1996,25:552-562.
[49]Dai Y.H., Some new properties of a nonlinear conjugate gradient method. Research report ICM-98-010, Insititute of Computational Mathematics and Scientific/Engineering Computing. Chinese Academy of Sciences, Beijing,1998.
[50]Hager W.W., Zhang H.,A survey of nonlinear conjugate gradient methods. Pacific J. Optim.,2006,2:35-58.
[51]Hu Y, Storey C.,Global convergence results for conjugate gradientmethods. J.Optim. Theory Appl.,1991,71:399-405.
[52]Liu D.C., Nocedal J.,On the limited memory BFGS method for large scale optimization methods, Math. Program.45,1989,503-528
[53]More J.J., Thuente D.J., Line search algorithms with guaranteed sufficient decrease. ACM Trans. Math. Softw.,1994,20:286-307.
[54]Nocedal J., Conjugate gradient methods and nonlinear optimization. In:Adams, L., Nazareth, J.L. (eds.) Linear and Nonlinear Conjugate Gradient-Related Methods., SIAM,Philadelphia,1996,9-23.
[55]Sun J., Zhang J., Convergence of conjugate gradient methods without line search. Ann. Oper. Res.,2001,103:161-173.
[56]Andrei N., Another nonlinear conjugate gradient algorithm for unconstrained optimization,Optimization Methods and Software,2009,24(1):89-104
[57]Andrei N., Acceleration of conjugate gradient algorithms for unconstrained optimization.Applied Mathematics and Computation,2009,213(2):361-369.
[58]Yuan G.L.,Modified nonlinear conjugate gradient methods with sufficient descent property for large-scale optimization problems,Optim. Lett.,2009,3:11-21.
[59]Andrei N., Open problems in nonlinear conjugate gradient algorithms for unconstrained optimization, Bulletin of the Malaysian Mathematical Sciences Society,2011,34(2):319-330
[60]Birgin E., Martinez J. M., A spectral conjugate gradient method for unconstrained opti-mization, Appl. Math. Optim.,2001,43:117-128.
[61]Andrei N. Scaled conjugate gradient algorithms for unconstrained optimization. Comput. Optim. Appl.,2007,38(3):401-416
[62]Andrei N., A Dai-Yuan conjugate gradient algorithm with sufficient descent and conjugacy conditions for unconstrained optimization, Applied Mathe. Letters, 2008,21(2):165-171.
[63]Andrei N., New accelerated conjugate gradient algorithm as a modification of Dai-Yuan's computational scheme for unconstrained optimization,J. of Comp. App. Math.,2010,234:3397-3410.
[64]Zhang L.,Zhou W.J., Li D. H., Global convergence of a modified FR conjugate gradient method with Armijo-type line search, Numer. Math.2006,104: 561-572.
[65]Du Shou-qiang, Chen Yuan-yuan, Global Convergence of a Modified Spectral FR ConjugateGradient Method Without Line Search,International Journal of Nonlinear Science,2007,4(1):151-155.
[66]Du Shou-qiang, Chen Yuan-yuan, Global Convergence of a Modified Spectral FR ConjugateGradient Method, App. Math. and Comp.,2008,202:766-770.
[67]Cheng W., A two-term PRP-based descent method, Numerical Functional Analysis and Optimization,2007,28:1217-1230.
[68]Wan Z., Yang Z.L., Wang Y.L., New spectral PRP conjugate gradient method for unconstrained optimization, Appl. Math. Letters,2011,24:16-22.
[69]Lu A.,Liu H.,Zheng X.,Cong W.,A variant spectral-type FR conjugate method and its global convergence,Applied Mathematics and Computation,2011,217: 5547-5552.
[70]Yu G.H., Huang J.H., Zhou Y, A descent spectral conjugate gradient method for impulse noise removal, Applied Mathematics Letters,2010,23:555-560.
[71]Yu G., Guan L., Chen W., Spectral conjugate gradient methods with sufficient descent property for large — scale unconstrained optimization, Optimization Methods and Software,2008,23:275-293
[72]宁亚楠,王希云,Arm ijo搜索下的谱共轭梯度法,太原科技大学学报,2011,32(2)：153-156
[73]马明娟,邓键,黄庆道,等.非精确条件下的谱共轭梯度算法,吉林大学学报,2009,47(2)：207-210.
[74]朱花,王希云,一种无约束优化问题的谱共轭梯度法,太原科技大学学报,2010,31(3)：245-248.
[75]李平芳,王希云,基于Armijo型线搜索下的谱共轭梯度法,太原科技大学学报,2011,32(6)：479-482.
[76]Livieris I., Sotiropoulos D., Pintelas P., On Descent Spectral CG Algorithms for Training Recurrent Neural Networks [C]. In Proceedings of 13th Panhellenic Conference on Informatics,2009,65-69.
[77]Wolfe P., Convergence conditions for ascent methods, SIAM Rev.,1969,11: 226-235.
[78]Wolfe P., Convergence conditions for ascent methods Ⅱ:some corrections, SIAM Rev.,1971,13:185-188.
[79]Leone R. D., Gaudioso M., Grippo L. Stopping criteria for line search methods without derivatives mathematical programming,1984,30:285-300.
[80]Grippo L., Lampariello F., Lucidi S., Anonmonotone line search technique for Newton's methods, SIAM J. Numer. Anal.,1986,23:707-716.
[81]Grippo L., Lampariello F., Lucidi S., A truncated Newton method with nonmonotone line search for unconstrained optimization, J. Optim. Theory Appl. 1989,60:410-419.
[82]Liu G., Han J., Sun D., Global convergence of BFGS algorithm with nonmonotone line search, Optimization,1995,34:147-159.
[83]E.R. Panier, A.L. Tits, Avoiding Maratos effect by means of nonmonotone line search constrained problems, SIAM J. Numer. Anal.,1991,28:1183-1195.
[84]Toint P.L., An assessment of nonmonotone line search technique for unconstrained optimization, SIAM J. on Scientific Comput.,1996,17:725-739.
[85]Toint P.L., A nonmonotone trust region algorithm for nonlinear programming subject to convex constraints, Math. Programming,1997,77:69-94.
[86]Birgin E. G, Martinez J. M., M. Raydan, Nonmonotone spectral projected gradient methods on convex sets, SIAM J. Optim.,2000,10:1196-1211.
[87]李董辉,童小娇,万中.数值最优化(第二版),北京：科学出版社,2005.
[88]Lucidi S., Rochetich F., and Roma M., Curvilinear stabilization techniques for truncated Newton methods in large-scale unconstrained optimization, SIAM J. Optim.,1998,8:916-939
[89]Zhou J. L. and Tits A. L., Nonmonotone line search for minimax problem, J. Optim. Theory Appl.,1993,76:455-476.
[90]Dai Y.H., On the nonmonotone line search, J. Optim. Theory Appl.,2001,112: 315-330.
[91]Zhang H. C., Hager, W. W., A nonmonotone line search technique and its application to unconstrained optimization. SIAM Journal of Optimization,2004, 14(4):1043-1056.
[92]万中,冯冬冬,一类非单调保守BFGS算法研究,计算数学,2011,33(4)：387-396.
[93]Sun W.Y., Han J.Y., Sun J., Global convergence of non-monotone descent methods for unconstrained optimization problems, J. Comput. Appl. Math.2002, 146:89-98.
[94]Deng N.Y., Xiao Y., Zhou F.J., Nonmonotonic trust-region algorithm, J. Optim. Theory Appl.,1993,26:259-285.
[95]Grippo L., Lampariello F., Lucid S.i, A class of nonomonotone stabilization methods in unconstrained optimization, Numer. Math.1991,59:779-805
[96]Han J., Liu G., Global convergence analysis of a new nonmotone BFGS algorithm on convex objective function, Comput. Optim. Appl.,1997,7:277-289
[97]Liu L., Yao S., We Z.i, The global and superlinear convergence of a new nonmonotone MBFGS algorithm on convex objective function, J. Comput. Appl. Math.,2007,220:422-438.
[98]万中,王旭,费云云,一类非单调三参数共轭梯度算法研究,湖南大学学报,2011,38(8)：71-75.
[99]Yuan Gonglin, Wei Zengxin, New line search methods for unconstrained optimization, Journal of the Korean Statistical Society,2009,38:29-39.
[100]Yu Zhensheng, Pu Dingguo, A new nonmonotone line search technique for unconstrained optimization, Journal of Computational and Applied Mathematics,2008,219:134-144.
[101]Yunhai Xiao, Huijuan Sun, Zhiguo Wang, A globally convergent BFGS method with nonmonotone line search for non-convex minimization, Journal of Computational and Applied Mathematics,2009,230:95-106
[102]Antoniou A., Lu W. S., Practical Optimization:Algorithms and Engineering Applica-tions, Springer, New York,2007.
[103]Feireisl E., Issard-Roch F., Petzeltova H., Long-time behaviour and convergence towards equilibria for a conserved phase field model. Partial differential equations and applications,Discrete Contin. Dyn. Syst.,2004,10: 239-252.
[104]Gunzburger M., Yang S. D.,Zhu W. X., Analysis and discretization of an optimal control problem for the forced Fisher equation, Discrete and Continuous Dynamical Systems-SeriesB,2007,8:569-587.
[105]Andrei N., Numerical comparison of conjugate gradient algorithms for unconstrained optimization, Studies in Informatics and Control,2007,16: 333-352.
[106]Huang Z., Li S., Guaranteed descent conjugate gradient methods with modified secant condition, J. Industrial and Management Optim.,2008,4:739-755.
[107]Li G, Guan L., Yu G., Global convergence of modified Polak-Ribiere-Polyak conjugate gradient methods with sufficient descent property, J. Industrial and Management Optim.,2008,4:565-579.
[108]Shi Z. J., Shen J.,Convergence of the Polak-Ribi-Polyak conjugate gradient method,Nonlinear Analysis,2007,66:1428--1441.
[109]Yu G. H., Guan L. T.,Wei Z. X., Globally convergent Polak-Ribire-Polyak conjugate gradient methods under a modified Wolfe line search,Applied Mathematics and computation,2009,215:3082-3090.
[110]Yuan G. L.,Lu X. W., Wei Z. X.,A conjugate gradient method with descent direction for unconstrained optimization,Journal of Computational and Applied Mathematics,2009,233(2):519--530.
[111]Zhang L.,Zhou W.J.,Li D. H., A descent modified Polak-Ribiere-Polyak conjugate gradient method and its global convergence,IMA J. Numer. Anal. 2006,26:629-640.
[112]Wan Zhong, Zheng XiaoDong, Fei YunYun,Deng SongHai, Modified Armijo-type line search strategy and its applications in Newton method for unconstrained optimization, International Journal of Applied Mathematics & Statistics,2012,29(5):116-125.
[113]Yuan G L., Lu X. W., Wei Z. X., A conjugate gradient method with descent direction for unconstrained optimization, Journal of Computational and Applied Mathematics,2009,233:519-530.
[114]Zhang L.,Zhou W. J., Li D. H., Global convergence of a modified Fletcher-Reeves conju-gate gradient method with Armijo-type line search, Numer. Math.,2006,104:561-572.
[115]Wan Z., Hu C. M., Yang Z. L.,A Spectral PRP Conjugate Gradient Methods for Nonconvex Optimization Problem Based on Modified Line Search. Discrete and Continuous Dynamical Systems:Series B,2011,16(4):1157-1169
[116]Andrei N. Accelerated scaled memoryless BFGS preconditioned conjugate gradient algorithm for unconstrained optimization. European Journal of Operational Research,2010,204(3):410-420
[117]Shi Z.J.,Shen J., Convergence of PRP method with new nonmonotone line search. Applied Mathematics and Computation,2006,181:423-431
[118]Shi Z. J.,Guo J. H., A new family of conjugate gradient methods. Journal of Computational and Applied Mathematics,2009,224:444-457
[119]Bongartz I, Conn A R, Gould N IM, Toint P L. CUTE:Constrained and unconstrained testing environments. ACM Trans Math Software,1995,21: 123-16
[120]More J. J., Garbow B. S., Hillstrom K. E. Testing unconstrained optimization software. ACM Transactions on Mathematical Software,1981,7(1)17-41
[121]Zhang Bo, Convergence of modified conjugate gradient methods without line search,2010 Second International Conference on Computational Intelligence and Natural Computing,2010,328-331.
[122]Dai Z.F., Tian B.S., Global convergence of some modified PRP nonlinear conjugate gradient methods,Optim. Lett.,2011,5(4):615-630.
[123]Jiang H.B., Deng S.H., Zheng X.D., Wan Z., Global convergence of a modified spectral conjugate gradient method, Journal of Applied Mathematics,2012,2012, Article ID 641276,13 pages,doi:10.1155/2012/641276
[124]王开荣,曹伟,王银河,Armijio型线搜索下的谱CD共轭梯度法,山东大学学报,2010,45(11)：104-108.
[2]Raydan M., The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem, SIAM Journal of Optimization,1997,7: 26-33
[3]Schropp J., A note on minimization problems and multistep methods. Numeric Mathematic,1997,78:87-101.
[4]Schropp J., One-step and multistep procedures for constrained minimization problems. IMA Journal of Numerical Analysis,2000,20:135-152.
[5]Van Wyk D. J., Differential optimization techniques. Applied Mathematical Modelling,1984,8:419-424.
[6]Vrahatis M. N., Androulakis G.S., Lambrinos J. N., Magolas G.D., A class of gradient unconstrained minimization algorithms with adaptive stepsize, Journal of Computational and Applied Mathematics,2000,114:367-386
[7]Yuan G. L., Lu X. W., A new line search method with trust region for unconstrained optimization. Communications on Applied Nonlinear Analysis, 2008,15(1):35-49
[8]Raydan M., Svaiter B. F.,Relaxed steepest descent and Cauchy-Barzilai-Borwein method. Computational Optimization and Applications,2002,21:155-167
[9]Horst R., Pardalos P. M., Thoai A. V., Introduction to global optimization. Dordrecht:Kluwer,1995.
[10]Luenerger D. C., Linear and nonlinear programming (2nd ed.). Reading, MA: Addition Wesley,1989.
[11]Nocedal J., Wright S. J., Numerical optimization. Berlin Heidelberg, Springer New York,1999.
[12]赖炎连,贺国平,最优化方法,清华大学出版社,2008
[13]Dennis J.E., More J.J., A characterization of superlinear convergence and its application to quasi-Newton methods. Mathematics of Computational,1974,28: 549-560.
[14]Dennis J. E., Jr., More J. J., Quasi-Newton methods, motivation and theory. SIAM Review,1977,19:46-89.
[15]Fletcher R., Practical method of optimization, vol Ⅰ:Unconstrained optimization (2nd ed.), Wiley,New York,1987.
[16]Griewank A., Toint Ph. L., Local convergence analysis for partitioned quasi-Newton updates. Numeric Mathematic,1982,39:429-448.
[17]Byrd R., Nocedal J., A tool for the analysis of quasi-Newton methods with application to unconstrained minimization. SIAM Journal on Numerical Analysis,1989,26:727-739
[18]Dai Y., Convergence properties of the BFGS algorithm. SIAM Journal on Optimization,2003,13:693-701.
[19]Dennis J. E., Schnabel R. B., Numerical methods for unconstrained optimization and nonlinear equations. Englewood Cliffs, NJ:Prentice-Hall, Inc.,1983.
[20]Li D., Fukushima M., A modified BFGS method and its global convergence in nonconvex minimization. Journal of Computational and Applied Mathematics, 2001,129:15-35.
[21]Wei Z., Li G., Qi L., New quasi-Newton methods for unconstrained optimization problems. Applied Mathematics and Computation,2006,175:1156-1188
[22]Wei Z., Yu G., Yuan G, Lian Z., The superlinear convergence of a modified BFGS-type method for unconstrained optimization. Computational Optimization and Applications,2004,29:315-332
[23]Zhang J., Deng N., Chen L., New quasi-Newton equation and related methods for unconstrained optimization. Journal of Optimization Theory and Applications, 1997,102(1):147-167.
[24]孙文瑜,徐成贤,朱德通,最优化方法,高等教育出版社,2004.
[25]Hestenes M. R.,Stiefel E. L, Method of conjugate gradient for solving linear equations. Journal of Research of the National Bureau Standards,1952,49: 409-436
[26]Fletcher R., Reeves C., Function minimization bu conjugate gradients. Computer Journal,1964,7:149-154.
[27]Polak E., Ribiere G., Note sur la convergence de directions conjugees., Rev. Francaise informat Recherche Operatinelle,3e Annee,1969,16:35-43.
[28]Flecher R., Practical Methods of Optimization,Vol I:unconstrained optimization, New York:John Wiley and Sons,1987.
[29]Liu Y,Storey C. Efficient generalized conjugate gradient algorithms, part 1: theory. Journal of Optimization Theory and Application,1991,69:129-137
[30]Dai Y, Yuan Y., A nonlinear conjugate gradient with a strong global convergence properties, SI AM Journal of Optimization,1999,10:177-182.
[31]Zoutendijk G., Nonlinear programming, computational methods, Integer and Nonlinear Programming, North-Holland, Amsterdam,1970,37-86.
[32]Al-Baali M., Descent property and global convergence of the Fletcher-Revves method with inexact line search. IMA J. Numer. Annal.,1985,5:121-124.
[33]Liu G H., Han J. Y, ed. Global convergence of the Fletcher-Reeves method with inexact line search, Appl. Math. J. Chinese Univ. Ser. B,1995,10:75-82.
[34]Dai Y.H., Yuan Y.X., Convergence properties of the Fletcher-Reeves method, IMA J. Numer. Anal.,1996,16(2):155-164.
[35]Dai Y. H., Yuan Y. X. Convergence of the Fletcher-Reeves method under a generalized Wolfe search. Journal Computational Mathematics of Chinese Universities,1996,2:142-148.
[36]Liu G, Han J., Yin H., Global convergence of the Fletcher-Reeves method with inexact line search, Appl. Math. J. Chin. Univ. Ser. B,1995,10:75-82.
[37]Powell M. J. D. Restart procedures of the conjugate gradient method [123]. Math Program,1977,2:241-254.
[38]Polyak B. T.,The conjugate gradient method in extremem problems [123]. USSR Comp Math and Math Phys,1969,9:94-112.
[39]Powell M. J. D., Nonconvex minimization calculations and the conjugate gradient method,Lecture Notes in Mathematics, Berlin:Springer-Verlag,1984,1066: 122-141.
[40]Yuan Y. X., Analysis on the conjugate gradient method,Optimization Methods and Software,1993,2:19-29.
[41]Gilbert J. C., Nocedal J., Global convergence properties of conjugate gradient methods for optimization. SIAM J. Optim.,1992,2(1):21-42.
[42]Grippo L., Lucidi S., A globally convergent version of the Polak-Ribiere conjugate gradient method. Mathematical Programming,1997,78:375-391.
[43]Wang C., Chen Y, Du S., Further insight into the Shamanskii modification of Newton method, Appl. Math. Comput,2006,180:46-52.
[44]Zhang L., Zhou W. J., Li D. H., A descent modified Polak-Ribiere-Polyak conjugate gradient method and its global convergence,IMA J. Numer. Anal., 2006,26:629-640.
[45]徐成贤,陈志平等,近代优化方法,科学出版社,2002.
[46]戴或虹,袁亚湘.非线性共轭梯度法.上海：上海科学技术出版社,2000.
[47]Dai Y.H., Yuan Y., A nonlinear conjugate gradient method with a strong global convergence property. SIAM J. Optim.,2000,10:177-182.
[48]Dai Y.H., Yuan Y., Convergence properties of the conjugate descent method. Adv. Math.,1996,25:552-562.
[49]Dai Y.H., Some new properties of a nonlinear conjugate gradient method. Research report ICM-98-010, Insititute of Computational Mathematics and Scientific/Engineering Computing. Chinese Academy of Sciences, Beijing,1998.
[50]Hager W.W., Zhang H.,A survey of nonlinear conjugate gradient methods. Pacific J. Optim.,2006,2:35-58.
[51]Hu Y, Storey C.,Global convergence results for conjugate gradientmethods. J.Optim. Theory Appl.,1991,71:399-405.
[52]Liu D.C., Nocedal J.,On the limited memory BFGS method for large scale optimization methods, Math. Program.45,1989,503-528
[53]More J.J., Thuente D.J., Line search algorithms with guaranteed sufficient decrease. ACM Trans. Math. Softw.,1994,20:286-307.
[54]Nocedal J., Conjugate gradient methods and nonlinear optimization. In:Adams, L., Nazareth, J.L. (eds.) Linear and Nonlinear Conjugate Gradient-Related Methods., SIAM,Philadelphia,1996,9-23.
[55]Sun J., Zhang J., Convergence of conjugate gradient methods without line search. Ann. Oper. Res.,2001,103:161-173.
[56]Andrei N., Another nonlinear conjugate gradient algorithm for unconstrained optimization,Optimization Methods and Software,2009,24(1):89-104
[57]Andrei N., Acceleration of conjugate gradient algorithms for unconstrained optimization.Applied Mathematics and Computation,2009,213(2):361-369.
[58]Yuan G.L.,Modified nonlinear conjugate gradient methods with sufficient descent property for large-scale optimization problems,Optim. Lett.,2009,3:11-21.
[59]Andrei N., Open problems in nonlinear conjugate gradient algorithms for unconstrained optimization, Bulletin of the Malaysian Mathematical Sciences Society,2011,34(2):319-330
[60]Birgin E., Martinez J. M., A spectral conjugate gradient method for unconstrained opti-mization, Appl. Math. Optim.,2001,43:117-128.
[61]Andrei N. Scaled conjugate gradient algorithms for unconstrained optimization. Comput. Optim. Appl.,2007,38(3):401-416
[62]Andrei N., A Dai-Yuan conjugate gradient algorithm with sufficient descent and conjugacy conditions for unconstrained optimization, Applied Mathe. Letters, 2008,21(2):165-171.
[63]Andrei N., New accelerated conjugate gradient algorithm as a modification of Dai-Yuan's computational scheme for unconstrained optimization,J. of Comp. App. Math.,2010,234:3397-3410.
[64]Zhang L.,Zhou W.J., Li D. H., Global convergence of a modified FR conjugate gradient method with Armijo-type line search, Numer. Math.2006,104: 561-572.
[65]Du Shou-qiang, Chen Yuan-yuan, Global Convergence of a Modified Spectral FR ConjugateGradient Method Without Line Search,International Journal of Nonlinear Science,2007,4(1):151-155.
[66]Du Shou-qiang, Chen Yuan-yuan, Global Convergence of a Modified Spectral FR ConjugateGradient Method, App. Math. and Comp.,2008,202:766-770.
[67]Cheng W., A two-term PRP-based descent method, Numerical Functional Analysis and Optimization,2007,28:1217-1230.
[68]Wan Z., Yang Z.L., Wang Y.L., New spectral PRP conjugate gradient method for unconstrained optimization, Appl. Math. Letters,2011,24:16-22.
[69]Lu A.,Liu H.,Zheng X.,Cong W.,A variant spectral-type FR conjugate method and its global convergence,Applied Mathematics and Computation,2011,217: 5547-5552.
[70]Yu G.H., Huang J.H., Zhou Y, A descent spectral conjugate gradient method for impulse noise removal, Applied Mathematics Letters,2010,23:555-560.
[71]Yu G., Guan L., Chen W., Spectral conjugate gradient methods with sufficient descent property for large — scale unconstrained optimization, Optimization Methods and Software,2008,23:275-293
[72]宁亚楠,王希云,Arm ijo搜索下的谱共轭梯度法,太原科技大学学报,2011,32(2)：153-156
[73]马明娟,邓键,黄庆道,等.非精确条件下的谱共轭梯度算法,吉林大学学报,2009,47(2)：207-210.
[74]朱花,王希云,一种无约束优化问题的谱共轭梯度法,太原科技大学学报,2010,31(3)：245-248.
[75]李平芳,王希云,基于Armijo型线搜索下的谱共轭梯度法,太原科技大学学报,2011,32(6)：479-482.
[76]Livieris I., Sotiropoulos D., Pintelas P., On Descent Spectral CG Algorithms for Training Recurrent Neural Networks [C]. In Proceedings of 13th Panhellenic Conference on Informatics,2009,65-69.
[77]Wolfe P., Convergence conditions for ascent methods, SIAM Rev.,1969,11: 226-235.
[78]Wolfe P., Convergence conditions for ascent methods Ⅱ:some corrections, SIAM Rev.,1971,13:185-188.
[79]Leone R. D., Gaudioso M., Grippo L. Stopping criteria for line search methods without derivatives mathematical programming,1984,30:285-300.
[80]Grippo L., Lampariello F., Lucidi S., Anonmonotone line search technique for Newton's methods, SIAM J. Numer. Anal.,1986,23:707-716.
[81]Grippo L., Lampariello F., Lucidi S., A truncated Newton method with nonmonotone line search for unconstrained optimization, J. Optim. Theory Appl. 1989,60:410-419.
[82]Liu G., Han J., Sun D., Global convergence of BFGS algorithm with nonmonotone line search, Optimization,1995,34:147-159.
[83]E.R. Panier, A.L. Tits, Avoiding Maratos effect by means of nonmonotone line search constrained problems, SIAM J. Numer. Anal.,1991,28:1183-1195.
[84]Toint P.L., An assessment of nonmonotone line search technique for unconstrained optimization, SIAM J. on Scientific Comput.,1996,17:725-739.
[85]Toint P.L., A nonmonotone trust region algorithm for nonlinear programming subject to convex constraints, Math. Programming,1997,77:69-94.
[86]Birgin E. G, Martinez J. M., M. Raydan, Nonmonotone spectral projected gradient methods on convex sets, SIAM J. Optim.,2000,10:1196-1211.
[87]李董辉,童小娇,万中.数值最优化(第二版),北京：科学出版社,2005.
[88]Lucidi S., Rochetich F., and Roma M., Curvilinear stabilization techniques for truncated Newton methods in large-scale unconstrained optimization, SIAM J. Optim.,1998,8:916-939
[89]Zhou J. L. and Tits A. L., Nonmonotone line search for minimax problem, J. Optim. Theory Appl.,1993,76:455-476.
[90]Dai Y.H., On the nonmonotone line search, J. Optim. Theory Appl.,2001,112: 315-330.
[91]Zhang H. C., Hager, W. W., A nonmonotone line search technique and its application to unconstrained optimization. SIAM Journal of Optimization,2004, 14(4):1043-1056.
[92]万中,冯冬冬,一类非单调保守BFGS算法研究,计算数学,2011,33(4)：387-396.
[93]Sun W.Y., Han J.Y., Sun J., Global convergence of non-monotone descent methods for unconstrained optimization problems, J. Comput. Appl. Math.2002, 146:89-98.
[94]Deng N.Y., Xiao Y., Zhou F.J., Nonmonotonic trust-region algorithm, J. Optim. Theory Appl.,1993,26:259-285.
[95]Grippo L., Lampariello F., Lucid S.i, A class of nonomonotone stabilization methods in unconstrained optimization, Numer. Math.1991,59:779-805
[96]Han J., Liu G., Global convergence analysis of a new nonmotone BFGS algorithm on convex objective function, Comput. Optim. Appl.,1997,7:277-289
[97]Liu L., Yao S., We Z.i, The global and superlinear convergence of a new nonmonotone MBFGS algorithm on convex objective function, J. Comput. Appl. Math.,2007,220:422-438.
[98]万中,王旭,费云云,一类非单调三参数共轭梯度算法研究,湖南大学学报,2011,38(8)：71-75.
[99]Yuan Gonglin, Wei Zengxin, New line search methods for unconstrained optimization, Journal of the Korean Statistical Society,2009,38:29-39.
[100]Yu Zhensheng, Pu Dingguo, A new nonmonotone line search technique for unconstrained optimization, Journal of Computational and Applied Mathematics,2008,219:134-144.
[101]Yunhai Xiao, Huijuan Sun, Zhiguo Wang, A globally convergent BFGS method with nonmonotone line search for non-convex minimization, Journal of Computational and Applied Mathematics,2009,230:95-106
[102]Antoniou A., Lu W. S., Practical Optimization:Algorithms and Engineering Applica-tions, Springer, New York,2007.
[103]Feireisl E., Issard-Roch F., Petzeltova H., Long-time behaviour and convergence towards equilibria for a conserved phase field model. Partial differential equations and applications,Discrete Contin. Dyn. Syst.,2004,10: 239-252.
[104]Gunzburger M., Yang S. D.,Zhu W. X., Analysis and discretization of an optimal control problem for the forced Fisher equation, Discrete and Continuous Dynamical Systems-SeriesB,2007,8:569-587.
[105]Andrei N., Numerical comparison of conjugate gradient algorithms for unconstrained optimization, Studies in Informatics and Control,2007,16: 333-352.
[106]Huang Z., Li S., Guaranteed descent conjugate gradient methods with modified secant condition, J. Industrial and Management Optim.,2008,4:739-755.
[107]Li G, Guan L., Yu G., Global convergence of modified Polak-Ribiere-Polyak conjugate gradient methods with sufficient descent property, J. Industrial and Management Optim.,2008,4:565-579.
[108]Shi Z. J., Shen J.,Convergence of the Polak-Ribi-Polyak conjugate gradient method,Nonlinear Analysis,2007,66:1428--1441.
[109]Yu G. H., Guan L. T.,Wei Z. X., Globally convergent Polak-Ribire-Polyak conjugate gradient methods under a modified Wolfe line search,Applied Mathematics and computation,2009,215:3082-3090.
[110]Yuan G. L.,Lu X. W., Wei Z. X.,A conjugate gradient method with descent direction for unconstrained optimization,Journal of Computational and Applied Mathematics,2009,233(2):519--530.
[111]Zhang L.,Zhou W.J.,Li D. H., A descent modified Polak-Ribiere-Polyak conjugate gradient method and its global convergence,IMA J. Numer. Anal. 2006,26:629-640.
[112]Wan Zhong, Zheng XiaoDong, Fei YunYun,Deng SongHai, Modified Armijo-type line search strategy and its applications in Newton method for unconstrained optimization, International Journal of Applied Mathematics & Statistics,2012,29(5):116-125.
[113]Yuan G L., Lu X. W., Wei Z. X., A conjugate gradient method with descent direction for unconstrained optimization, Journal of Computational and Applied Mathematics,2009,233:519-530.
[114]Zhang L.,Zhou W. J., Li D. H., Global convergence of a modified Fletcher-Reeves conju-gate gradient method with Armijo-type line search, Numer. Math.,2006,104:561-572.
[115]Wan Z., Hu C. M., Yang Z. L.,A Spectral PRP Conjugate Gradient Methods for Nonconvex Optimization Problem Based on Modified Line Search. Discrete and Continuous Dynamical Systems:Series B,2011,16(4):1157-1169
[116]Andrei N. Accelerated scaled memoryless BFGS preconditioned conjugate gradient algorithm for unconstrained optimization. European Journal of Operational Research,2010,204(3):410-420
[117]Shi Z.J.,Shen J., Convergence of PRP method with new nonmonotone line search. Applied Mathematics and Computation,2006,181:423-431
[118]Shi Z. J.,Guo J. H., A new family of conjugate gradient methods. Journal of Computational and Applied Mathematics,2009,224:444-457
[119]Bongartz I, Conn A R, Gould N IM, Toint P L. CUTE:Constrained and unconstrained testing environments. ACM Trans Math Software,1995,21: 123-16
[120]More J. J., Garbow B. S., Hillstrom K. E. Testing unconstrained optimization software. ACM Transactions on Mathematical Software,1981,7(1)17-41
[121]Zhang Bo, Convergence of modified conjugate gradient methods without line search,2010 Second International Conference on Computational Intelligence and Natural Computing,2010,328-331.
[122]Dai Z.F., Tian B.S., Global convergence of some modified PRP nonlinear conjugate gradient methods,Optim. Lett.,2011,5(4):615-630.
[123]Jiang H.B., Deng S.H., Zheng X.D., Wan Z., Global convergence of a modified spectral conjugate gradient method, Journal of Applied Mathematics,2012,2012, Article ID 641276,13 pages,doi:10.1155/2012/641276
[124]王开荣,曹伟,王银河,Armijio型线搜索下的谱CD共轭梯度法,山东大学学报,2010,45(11)：104-108.