AN EFFICIENT PRP-HRM HYBRID CONJUGATE GRADIENT METHOD FOR SOLVING UNCONSTRAINED OPTIMIZATION
Keywords:
Exact line search, Hybrid Conjugate gradient, sufficient descent condition, Global convergenceAbstract
The hybrid conjugate gradient methods are combinations of different conjugate gradient (CG) algorithms to give better performance. This paper develops a new hybrid method of conjugate gradient type, satisfies the sufficient descent condition under the exact line search conditions and becomes globally convergent. Preliminary numerical experiments are tested on a set of unconstrained optimization test problems. The results of comparisons show the computational efficiency of the developed hybrid method by solving selected large-scale benchmark test functions against some known algorithms in the sense of Dolan–More performance profile.
References
A. Y. Al-Bayati and S. M. Abbas, "A robust Spectral Three-Term Conjugate Gradient Algorithm for Solving Unconstrained Minimization," AL-Rafidain Journal of Computer Sciences and Mathematics, vol. 13, pp. 87-104, 2019.
M. R. Hestenes and E. Stiefel, "Methods of conjugate gradients for solving linear systems," Journal of Research of the National Bureau of Standards, vol. 49, pp. 409-436, 1952.
R. Fletcher and C. M. Reeves, "Function minimization by conjugate gradients," The Computer Journal, vol. 7, pp. 149-154, 1964.
B. T. Polyak, "The conjugate gradient method in extreme problems," USSR Computational Mathematics and Mathematical Physics, vol. 9, pp. 807-821, 1969.
R. Fletcher, Practical Methods of Optimization: Unconstrained Optimization vol. 1. New York: John Wiley & Sons, 1987.
Y. Liu and C. Storey, "Efficient generalized conjugate gradient algorithms, Part 1: Theory," Journal of Optimization Theory and Applications, vol. 69, pp. 129-137, 1991.
Y.-H. Dai and Y.-X. Yuan, "A nonlinear conjugate gradient method with a strong global convergence property," SIAM Journal on Optimization, vol. 10, pp. 177-182, 1999.
Z.-X. Wei, S. W. Yao, and L. Y. Liu, "The convergence properties of some new conjugate gradient methods," Applied Mathematics and Computation, vol. 183, pp. 1341-1350, Dec 15 2006.
M. Hamoda, M. Rivaie, and M. Mamat, "A new nonlinear conjugate gradient method with exact line search for unconstrained optimization," Journal of Humanities and Applied Science (JHAS), vol. 30, pp. 1-16, 2017.
D. Touati-Ahmed and C. Storey, "Efficient hybrid conjugate gradient techniques," Journal of optimization theory and applications, vol. 64, pp. 379-397, 1990.
M. Al-Baali, "Descent property and global convergence of the Fletcher-Reeves method with inexact line search," IMA Journal of Numerical Analysis, vol. 5, pp. 121-124, 1985.
Y.-H. Dai and Y.-X. Yuan, "Convergence properties of the Fletcher-Reeves method," Ima Journal of Numerical Analysis, vol. 16, pp. 155-164, Apr 1996.
Y.-Q. Zhang, H. Zheng, and C.-L. Zhang, "Global convergence of a modified PRP conjugate gradient method," in International Conference on Advances in Computational Modeling and Simulation, 2012, pp. 986-995.
L. Min, F. Heying, and L. Jianguo, "The global convergence of a descent PRP conjugate gradient method," Computational and Applied Mathematics, vol. 31, pp. 59–83, 2012.
Y. H. Dai and Y. Yuan, "Further studies on the Polak-Ribiere-Polyak method," Research report ICM-95-040, Institute of Computational Mathematics and Scientific/Engineering Computing, Chinese Academy of Sciences1995.
Z.-J. Shi and J. Shen, "Convergence of the Polak–Ribiére–Polyak conjugate gradient method," Nonlinear Analysis: Theory, Methods & Applications, vol. 66, pp. 1428-1441, 3/15/ 2007.
S. Babaie-Kafaki, "A quadratic hybridization of Polak–Ribière–Polyak and Fletcher–Reeves conjugate gradient methods," Journal of Optimization Theory and Applications, vol. 154, pp. 916-932, 2012.
P. Kaelo, P. Mtagulwa, and M. V. Thuto, "A globally convergent hybrid conjugate gradient method with strong Wolfe conditions for unconstrained optimization," Mathematical Sciences, November 05 2019.
M. J. D. Powell, "Nonconvex minimization calculations and the conjugate gradient method," in Numerical analysis: Lecture Notes in mathematics. vol. 1066, ed Berlin: Springer-Verlag, 1984, pp. 122-141.
W. Zhao, C. Wang, and Y. Gu, "On the convergence of s-dependent GFR conjugate gradient method for unconstrained optimization," Numerical Algorithms, August 18 2017.
A. V. Mandara, M. Mamat, M. Waziri, and M. A. Mohamed, "A class of conjugate gradient parameters and its global convergence for solving unconstrained optimization," Far East Journal of Mathematical Sciences (FJMS), vol. 106, pp. 43-58, 2018.
Y. Salih, M. Hamoda, Sukono, and M. Mamat, "The convergence properties of new hybrid conjugate gradient method," IOP Conference Series: Materials Science and Engineering, vol. 567, p. 012031, 2019/08/15 2019.
N. Shapiee, M. Rivaie, and M. Mamat, "A new classical conjugate gradient coefficient with exact line search," AIP Conference Proceedings, vol. 1739, p. 020082, 2016.
Y. Salih, M. Hamoda, and M. Rivaie, "New hybrid conjugate gradient method with global convergence properties for unconstrained optimization," Malaysian Journal of computing and applied mathematics (MyJCAM), vol. 1, pp. 29-38, 2018.
N. S. Mohamed, M. Mamat, M. Rivaie, and S. M. Shaharudin, "A comparison on classical-hybrid conjugate gradient method under exact line search," International Journal of Advances in Intelligent Informatics, vol. 5, p. 10, 2019-07-31 2019.
N. ‘Aini, M. Rivaie, and M. Mamat, "A modified conjugate gradient coefficient with inexact line search for unconstrained optimization," AIP Conference Proceedings, vol. 1787, p. 080019, 2016.
D. Touati-Ahmed, "A study of hybrid conjugate gradient methods," PhD Dissertation, Loughborough University, 1989.
P. Kaelo, S. Narayanan, and M. Thuto, "A modified quadratic hybridization of Polak-Ribiere-Polyak and Fletcher-Reeves conjugate gradient method for unconstrained optimization problems," An International Journal of Optimization and Control: Theories & Applications (IJOCTA), vol. 7, pp. 177-185, 2017.
Y. F. Hu and C. Storey, "Global convergence result for conjugate-gradient methods," Journal of Optimization Theory and Applications, vol. 71, pp. 399-405, Nov 1991.
J. C. Gilbert and J. Nocedal, "Global convergence properties of conjugate gradient methods for optimization," SIAM journal on optimization, vol. 2, pp. 21-42, 1992.
Y.-H. Dai and Y.-X. Yuan, "An efficient hybrid conjugate gradient method for unconstrained optimization," Annals of Operations Research, vol. 103, pp. 33-47, 2001.
N. Andrei, "A hybrid conjugate gradient algorithm for unconstrained optimization as a convex combination of Hestenes – Stiefel and Dai - Yuan," Studies in Informatics and Control, vol. 17, pp. 55-70, 2008.
M. A. H. Ibrahim, M. Mamat, P. L. Ghazali, and Z. Salleh, "The scaling of hybrid method in solving unconstrained optimization method," Far East Journal of Mathematical Sciences (FJMS), vol. 99, pp. 983-991, 2016.
I. Abdullahi and R. Ahmad, "Global convergence analysis of a new hybrid conjugate gradient method for unconstrained optimization problems," Malaysian Journal of Fundamental and Applied Sciences, vol. 13, pp. 40-48, 2017.
N. H. A. Ghani, N. S. Mohamed, N. Zull, S. Shoid, M. Rivaie, and M. Mamat, "Performance comparison of a new hybrid conjugate gradient method under exact and inexact line searches," Journal of Physics: Conference Series, vol. 890, p. 012106, 2017.
W. F. H. W. Osman, M. A. H. Ibrahim, and M. Mamat, "Hybrid DFP-CG method for solving unconstrained optimization problems," Journal of Physics: Conference Series, vol. 890, p. 012033, 2017.
J. Sabi'u and M. Y. Waziri, "Effective modified hybrid conjugate gradient method for large-scale symmetric nonlinear equations," Applications & Applied Mathematics, vol. 12, pp. 1036-1056, 2017.
M. Hamoda, "Modification of Polak-Ribiere-Polyak (PRP) conjugate gradient coefficient for unconstrained optimization problems," Doctor of Philosophy, Department of Computational & Applied Mathematics, Universiti Sultan Zainal Abidin, non-published, 2016.
G. Zoutendijk, "Nonlinear programming, computational methods," in Integer and nonlinear programming, ed North-Holland, Amsterdam, 1970, pp. 37-86.
N. Andrei, "An unconstrained optimization test functions collection," Advanced Modeling and Optimization, vol. 10, pp. 147-161, 2008.
M. Molga and C. Smutnicki. (2005). Test functions for optimization needs. Available: http://www.zsd.ict.pwr.wroc.pl/files/docs/functions.pdf
S. K. Mishra, "Some new test functions for global optimization and performance of repulsive particle swarm method," Munich Personal RePEc Archive, Apr 13 2007.
K. E. Hillstrom, "A simulation test approach to the evaluation of nonlinear optimization algorithms," ACM Transactions on Mathematical Software, vol. 3, pp. 305-315, 1977.
E. D. Dolan and J. J. More, "Benchmarking optimization software with performance profiles," Mathematical Programming, vol. 91, pp. 201-213, 2002
Downloads
Published
Issue
Section
License
Copyright (c) 2019 Journal of Basic Sciences
This work is licensed under a Creative Commons Attribution 4.0 International License.
