CSCE5013-Assignment 1 Solved

Problem 1  

 We want to build an MLP that composes the decision boundary shown in the figure below.  The output of the MLP must 1 in the yellow regions and 0 elsewhere

  

 

The MLP we design is suboptimal and has the structure and weights shown in the figure below, where each perceptron computes the function.   

                                                           𝑦 =       $1                                                         𝑖𝑓                          (. 𝑤𝑒𝑖𝑔ℎ𝑡.. 𝑖𝑛𝑝𝑢𝑡. + 𝑏𝑖𝑎𝑠. ≥ 0

0              𝑒𝑙𝑠𝑒

 

 

 

Problem 2
 

We are given the relationship  

 

                                                                  𝑦   = 𝑓(𝐱, 𝑔(𝐱, 𝑐))𝑔(𝐱, 𝑓(𝐱, 𝑑)) 

Where x is a vector. 𝑓(.) and 𝑔(.) are both scalar functions which take a vector and a scalar as inputs. What is 𝒅𝒚? 

𝒅𝒙

 Note that the relationship between x, f(.) and g(.) is presented as following figure and we use dx for full derivate and ¶x for partial derivative  

 

  

 

 

 

Problem 3
Manually design MLP network to perform the XOR Gate with the truth table and its plot on 2D as follows:

  

 

The goal of the neural network is to classify the input patterns according to the above truth table.

As mentioned in the class, the network is designed in similar following architecture  

  

 

 

 

Where:  inputs = ([[0,0],[0,1],[1,0],[1,1]]) expected_output ([[0],[1],[1],[0]])

 

Start with uniform random initialization for parameters wij . Perform forward and backward pass in the following case:

1.      Activation function is (a) Sigmod (b) ReLu, (c)Tanh 

2.      Divergence is defined as (a) L2_norm, (b) cross entropy 

3.      Train the network for 2 iterations (3 forward pass and 2 backward pass) 

Please report the parameters and actual output from the MLP in each iteration  

 

 

 

Problem 4  
Building a MLP with one hidden layer to perform classification task with the following description:  

+ Training data (X, Y): 

Training data contains N1 = 10,000 points in 2-dimentional space and are followed by the uniform radius between 0 and 2 and its label is 1 is it is inside the yellow circle, otherwise it is 0 + Validation data (X, Y): 

Validation data contains N2 = 2,000 points in 2-dimentional space and are followed by the uniform radius between 0 and 2 and its label is 1 is it is inside the yellow circle, otherwise it is 0 

+ Testing data (X, Y): 

Testing data contains N2 = 2,000 points in 2-dimentional space and are followed by the uniform radius between 0 and 2 and its label is 1 is it is inside the yellow circle, otherwise it is 0 

 

  

Assume that we use CrossEntropyLoss (nn. CrossEntropyLoss) and  

GradientDescent(torch.optim.SGD) with lr = 0.01

 

Report the training loss, validation loss, testing accuracy (in number and visualized by figure) in the following case

+ Train the MLP with 10 iterations  

+ Train the MLP with 100 iterations

+ Train the MLP with 1000 iterations

 

Note: use import matplotlib.pyplot to plot figures