1 Overview
This assignment aims to experiment with several basic optimization methods discussed in the course content. The objective is to analyze and compare the performance of these methods on the given test functions, such as the strength and weakness of each method, the impact of parameters of line search or initial conditions on the optimal solution, etc. You will be evaluated on the depth of your analysis of the algorithm behaviour.
2 Test functions
The following objective functions represent different types of optimization test functions: quadratic, large-dimensional, etc. Test the performance of the methods on these functions.
1. ), where n = 50. The optimal solution is x∗ = 0.
2. , where n = 10. The optimal solution is x∗ = [1,...,1]T
3. . The optimal solution
3 Task
Your task is to implement the following optimization algorithms with backtracking line search:
• Steepest descent method
• Newtons method
• Quasi-Newton method (DFP or BFGS)
• Conjugate gradient method (Polak-Ribiere or Fletcher-Reeves )
Analyze each method on each of the given test functions. In your analysis, relate the behaviour of the methods to the characteristics of the functions.
