Implementation of gradient descent from scratch along with newton method.
Problem statement:
Consider two functions: f1(x, y) = (x − 2)^2 + (y − 3)^2 and f2(x, y) = (1 − (y − 3))^2 + 20((x + 3) − (y − 3)^2)^2 Starting with (x, y) = (0, 0) run the gradient descent algorithm for each function. Run for T iterations, and report the function value at the end of each step.
• First, run with a fixed learning rate of λ = 0.5
• Second, run with any variant of gradient descent you want. Try to get the smallest function value after T steps.
For f1 you are allowed only T = 10 steps. For f2 you are allowed T = 100 steps.
