Your programs must be written in Python. All code must be able to compile and run for full credit. Comment all code following proper coding conventions. Remember, if we can’t read it, we can’t grade it! (For more information on python coding standards, refer to: https://www.python.org/dev/peps/pep-0008/)
1. Consider the Neural Network below.
All weights are initialized to the values shown (and there are no biases for simplicity). Consider the data point x = [1,1]T with desired output vector d = [1,0]T Complete one iteration of backpropagation by hand assuming a learning rate of η = 0.1. What would all the weight values be after the one backpropagation iteration? Show your work. Use the following activation function:
2. Derive the update equation for output layer neurons if the activation function usedis the hyperbolic tangent, ϕ(v) = tanh(v) (instead of the activation function used in the notes). Show your work.
3. Derive the update equation for output layer neurons if the activation function used isthe softmax function, , where O is the number of output neurons and
vi is from the ith neuron. Note: In this case the output is multi-dimensional (i.e., di ∈RO) and there will be a specific di for each output neuron. Show your work.
