COMP5630 Assignment #3- Perceptrons Solution

Assignment #3 Perceptrons
Submission Instructions
Tasks
1 Linear Separators [50 pts]
Recall that a linear separator in 2 dimensions is given a function of the form:
y = ϕ(w1x1 + w2x2 + b)
where xi ∈{0,1}, and transfer function:
(
0 if, z ≤ 0
ϕ(z) =
1 otherwise

Figure 1: Linear Separator for question 1.
a. [5 pts] Complete the following truth table for the AND operation.
x1 x2 AND
0 0
0 1
1 0
1 1
Table 1: Logical AND operation
b. [5 pts] Provide values of w1, w2 and b to use the linear separator for logical AND.
c. [5 pts] Complete the following truth table 2 for the OR operation.
x1 x2 OR
0 0
0 1
1 0
1 1
Table 2: Logical OR operation
d. [5 pts] Provide values of w1, w2 and b to use the linear separator for logical OR.
e. [5 pts] Complete the following truth table 3 for the XOR operation.
f. [10 pts] Prove that the linear separator depicted in Figure 1 cannot be used to create logical XOR.
x1 x2 XOR
0 0
0 1
1 0
1 1
Table 3: Logical XOR operation
g. [15 pts] Suppose, we want to implement a classifier that takes two input values, where each value is either 0, 1 or 2, and outputs a 1 if at least one of the two inputs has value 2; otherwise it outputs a 0. Can this function be learned by a Perceptron? If so, construct a Perceptron that does it; if not, why not.
2 Programming Assignment: Perceptrons [50 pts]
In this assignment, we are going to see if we can teach a computer to distinguish living things from non-living things. More precisely, you will implement the perceptron algorithm to detect whether or not an image contains an animal or not.
2.1 Problem Statement
2.2 The template package
The template package contains everything you need to get started on your assignment. Specifically, it contains
• reader.py: This file is responsible for reading in the data set. It returns the training images, labels for these images, development data, and labels for the development data.
• main.py: This is the main file that starts the program, and computes the accuracy, precision, recall, and F1-score using your implementation of the classifers.
• classify.py: This is the file where you will be doing all of your work.
• data.zip The dataset. When you uncompress this file, you’ll find a binary object that our reader code will unpack for you.
To understand more about how to run the MP, run python3 main.py -h in your terminal. Add your code to classify.py. Do not modify the code provided in the other files.
2.3 The dataset
The dataset consists of 10000 32 * 32 colored images total. We have split this data set for you into 2500 development examples and 7500 training examples. The images have their RGB values scaled to range 0-1. This is a subset of the CIFAR-10 dataset, provided by Alex Krizhevsky.
The reader will supply the set of test images as one giant numpy array. The development data is in the same format. So your code must read the dimensions from the input numpy array.
In the dataset, each image is 32 * 32 and has three (RGB) color channels, yielding 32 * 32 * 3 = 3072 features.
2.4 Perceptron Model
The perceptron model is a linear function that tries to separate data into two or more classes. It does this by learning a set of weight coefficients wi and then adding a bias b. Suppose you have features x1, ... , xn then this can be expressed in the following fashion:

You will use the perceptron learning algorithm to find good weight parameters wi and b such that sign fw,b(x) > 0 when there is an animal in the image and sign fw,b(x) ≤ 0 when there is a no animal in the image.
Your function classifyPerceptron() will take as input the training data, training labels, development data, learning rate, and maximum number of iterations. It should return a list of labels for the development data. Do not hardcode values for tuning parameters inside your classifier functions.
2.5 Training and Development
• Training: To train the perceptron you are going to need to implement the perceptron learning algorithm on the training set. Each pixel of the image is a feature in this case. Be sure to initialize weights and biases to zero.
Note: Use a constant learning rate (no decay) and do not shuffle the training data.
• Development: After you have trained your perceptron classifier, you will have your model decide whether or not each image in the development set contains animals. In order to do this take the sign of the function fw,b(x). If it is negative or zero then classify as 0. If it is positive then classify as 1.
Use only the training set to learn the weights.
2.6 Accuracy Reporting
Using the main.py script, generate accuracy, f1, precision and recall scores for your perceptron algorithm on the development set. Submit your code (only classify.py) and report accuracy, f1, precision and recall scores in your PDF submission.
2.7 Using Numpy
It is much easier to write fast code by using numpy operations. Your data is provided as a numpy array. Use numpy operations as much as possible, until right at the end when you choose a label for each image and produce the output list of labels.
NumPy Tips:
• Running computations on arrays tend to be faster when run using NumPy arrays. If you are having issues with timing out, then consider using a NumPy implementation
• Linear algebra operators (dot product, vector norm, matrix multiplication) are immensely simplified when using NumPy. Consider looking up methods to perform some of these operations if they are needed in your solution.