DATA51100-Assignment 3 Solved

Introduction
Machine learning is an area of computer science whose aim is to create programs which improve their performance with experience. There are many applications for this, including: face recognition, recommendation systems, defect detection, robot navigation, and game playing. For this assignment, you will implement a simple machine learning algorithm called Nearest Neighbor which learns by remembering training examples. It then classifies test examples by choosing the class of the “closest” training example. The notion of “closeness” differs depending on applications. You will need to use the Nearest Neighbor algorithm to learn and classify types of Iris plants based on their sepal and petal length and width. There are three Iris types you will need to classify:
                                           

Iris Setosa                                              Iris Versicolour                                                     Iris Virginica



The learning will be done by remembering training examples stored in a comma-separated file. The training examples include different measurements which collectively are called features or attributes, and a class label for different instances. These are:

1.  sepal length in cm

2.  sepal width in cm

3.  petal length in cm

4.  petal width in cm    5. class:  

      -- Iris Setosa

      -- Iris Versicolour

      -- Iris Virginica

 

To see how well the program “learned”, you will then load a file containing testing examples, which will include the same type of information, but for different instances. For each test instance, you will apply the Nearest Neighbor algorithm to classify the instance. This algorithm works by choosing a class label of the “closest” training example, where “closest” means shortest distance. The distance is computed using the following formula:

 

𝑑𝑖𝑠𝑡(𝑥, 𝑦) = √(𝑠𝑙𝑥 − 𝑠𝑙𝑦)2 + (𝑠𝑤𝑥 − 𝑠𝑤𝑦)2 + (𝑝𝑙𝑥 − 𝑝𝑙𝑦)2 + (𝑝𝑤𝑥 − 𝑝𝑤𝑦)2

where 𝑥, 𝑦 are two instances (i.e. a training or a testing example), 𝑠𝑙𝑥, 𝑠𝑙𝑦 are their sepal lengths, 𝑠𝑤𝑥, 𝑠𝑤𝑦 are their sepal widths, 𝑝𝑙𝑥, 𝑝𝑙𝑦 are their petal lengths, and 𝑝𝑤𝑥, 𝑝𝑤𝑦 are their petal widths.

 

After you finish classifying each testing instance, you will then need to compare it to the “true” label that is specified for each example and compute the accuracy. Accuracy is measured as the number of correctly classified instances divided by the number of total testing instances.

Requirements
You are to create a program in Python 3 that performs the following:

1.       Loads and parses the training and testing dataset files into separate NumPy ndarrays. Given what you know, the easiest way to do this is to create four separate arrays:

•       2D array of floats for storing training example attribute values

•       2D array of floats for storing testing example attribute values

•       1D array of strings for storing training example class labels

•       1D array of strings for storing testing example class labels

You can assume there are exactly 4 attribute values in the training and testing examples.

2.       Classifies each testing example. You also need to output the true and predicted class label to the screen and save it into a new 1D array of strings. This is done by first computing the distance value for each pair of training and testing examples (their attribute values). Then, for each test example, find the training example with the closest distance. You can do all that easily with NumPy’s vectorized functions - you shouldn’t use loops for this.

3.       Computes the accuracy. Go through the array of class labels for testing examples and compare the label stored in the array created in step (2). Count how many matches you get. Output the number of matches, divided by the number of testing examples as a percentage.