1) Design a neural network. (20%)
a. Using a figure to illustrate your neural network.
b. Specify functions, e.g. ReLU in the neurons
c. Specify an objective function, e.g. L2 norm to train the network.
2) Derive gradients for all the network parameters. Please write down each step clearly. (30%)
3) Apply the results in (1) and (2) and derive the training equations based on gradient descent. (20%)
4) Implement the network and train it based on the equations you derive. You are not allowed to call the build-in functions in your programme tools in the training, except for basic mathematical functions and matrix operations. More clearly, you are not allowed to use the optimization functions and automatic differentiation functions in your toolboxes. You can use public datasets to train your network. You also can generate your own datasets, for example using Gaussian distributions. In your report, you need to provide a figure of training curve. (30%)
