COSC21110 - Assignment 2 - Solved

        Learning Outcomes
This assessment relates to the following learning outcomes of the course.

•     Demonstrate advanced knowledge of data mining concepts and techniques.

•     Apply the techniques of clustering, classification, association finding, feature selection and visualisation on real world data

•     Apply data mining software and toolkits in a range of applications

•     Set up a data mining process for an application, including data preparation, modelling and evaluation

3           Assignment Details
3.1           Part 1: Classification with Neural Networks (15 marks)
This part involves predicting the Class attribute in the following file:

chronic-kidney-disease-2019.arff in the directory:

/KDrive/SEH/SCSIT/Students/Courses/COSC2111/DataMining/data/arff/UCI/

The main goal is to achieve the lowest classification error with the lowest amount of overfitting.

For the neural network training runs build a table with the following headings:

Run
Archi-
Param
Train
Train
Epochs
Test
Test
No
tecture-
eters
MSE
Error
 
MSE
Error
1
ii-hh-oo
lr=.2
0.5
30%
500
0.6
40%
1.    Describe the data encoding that is required for this task. How many outputs and how many inputs will there be?

2.    Develop a script to generate the necessary training, validation and test files. You might want to normalize the numeric attributes with Weka beforehand. Include your data preparation script as an appendix (not part of the page count).

3.    Determine the “analyze” strategy that you will use.

4.    Using Javanns carry out 5 train and rest runs for a network with 10 hidden nodes. Comment on the variation in the training runs and the degree of overfitting.

5.    Experiment with different numbers of hidden nodes. What seems to be the right number of hidden nodes for this problem?

6.    For 10 hidden nodes, explore different values of the learning rate. What do you conclude?

7.    [Optional] Change the learning function to backprop-momentum. Explore different combinations of learning rate and momentum. What do you conclude?

8.    Perform a run with 10 hidden nodes and no validation data. Stop training when the MSE is no longer changing. Get the classification error on the training and test data. Comment on the degree of overfitting.

9.    Compare the classification accuracy of the neural classifiers with the classification accuracy of Weka J48 and MultilayerPerceptron.

Report Length Up to two pages.

3.2           Part 2: Numeric Prediction with Neural Networks (10 marks)
This part involves predicting the Age in the following file:

chronic-kidney-disease-2019.arff in the directory:

/KDrive/SEH/SCSIT/Students/Courses/COSC2111/DataMining/data/arff/UCI/

The main goal is to achieve the lowest mean absolute error with the lowest amount of overfitting.

The task is to predict the value of the Age variable. Build a similar table of runs to the one in the previous question.

1.    Describe the data encoding that is required for this task. How many outputs and how many inputs will there be? What scaling or normalization is required?

2.    Modify your script from part 1 to generate the necessary training, validation and test files. You can use Weka to normalize all of the numeric attributes except for the class, ie Age attribute. You will need to write a suitable program to scale the age to the range [0,1] and another one to reverse scale the neural net outputs to get the mean absolute error. Include your data preparation script as an appendix (not part of the page count).

3.    Using Javanns carry out 5 train and test runs for a network with 5 hidden nodes. Comment on the variation in the training runs and the degree of overfitting.

[Hint: When you are comparing the predictive accuracy of different models you don’t have to reverse scale the output.]

4.    Experiment with different numbers of hidden nodes. What seems to be the right number of hidden nodes for this problem?

5.    For 5 hidden nodes, explore different values of the learning rate. What do you conclude?

6.    [Optional] Change the learning function to backprop-momentum. Explore different combinations of learning rate and momentum. What do you conclude?

7.    Perform a run with 5 hidden nodes and no validation data. Stop training when the MSE is no longer changing. Get the error on the training and test data. Comment on the degree of overfitting.

8.    Compare the mean absolute error of the neural classifiers with the mean absolute error of Weka M5P and MultiLayerPerceptron.

Report Length Up to one page.

3.3           Part 3: Data Mining (25 marks)
This part of the assignment is concerned with the files portugal-students.arff portugal-students.txt which are in the directory:

/KDrive/SEH/SCSIT/Students/Courses/COSC2111/DataMining/data/arff

The file portugal-students.arff contains data about students in 2 high schools in Portugal. The file portugal-students.txt contains a description of the data.

Your task is to analyse this data with appropriate classification, clustering, association finding, attribute selection and visualisation techniques selected from the Weka menus and identify any “golden nuggets” in the data. If you don’t use any of the above techniques, you need to say why.

Submit: Up to two pages that describe the strategy you adopted, your methodology, the runs you performed, any “golden nuggets” you found and your conclusions.