CSCI964- Assignment 1 Solved

Your main task in this assignment is to implement selectable ordering of the training data and make the net more powerful by enabling the mlp to be configured as a 2, 3 or 4 layer mlp with a specified number of neurons in each layer and to use the mlp to classify all the given data. You are to also provide a test function so that the mlp can learn training data and be tested with different test data. For this assignment you can write and compile your code on PC or UNIX platforms. To complete this assignment it is recommended you follow the steps below. 

 

Step 1:  To improve mlp training, implement an option for providing selectable ordering of the training data. The “Ordering” parameter should be added to the definition header of the training data files after ObjErr. e.g.: 

. . .  

Mtm1: 1.2 

Mtm2: 0.4 

ObjErr: 0.005 Ordering: 1 . . .  

“Ordering” determines which training pattern is selected each training iteration (epoch). The options are:  

 

0   Fixed  Always uses the same given order of training patterns.  

1   Random           Make completely different (random) order each iteration (i.e. new permutation).  

2   Random Swap               Random permutation at start. After each epoch, two random patterns are exchanged. 3, 4 . .  Random N for i=0; i<(N-1); i++ 

                                                             Select a random pattern.  

                              If it was wrongly classified last time use it    if no wrong pattern was selected choose use the first one.  

 

Tip: Try using an index array to rearrange the selection order, or to select the next pattern. 

 

Step 2:  Implement 3 and 4 layer back propagation neural network procedures by modifying the code in mlp.cpp. To do this rename the TrainNet() function to TrainNet2() and make modified copies of this function (named: TrainNet3() and TrainNet4()) with 3 and 4 layers respectively. Then incorporate a switch statement into your code so that the appropriate TrainNet() function is invoked according to the data specs. Test your completed code on the provided data and ensure that the mlp configuration in the data files (ie the number of layers, number of neurons, etc) is being complied with.   

 

                

Step 3:  Write a TestNet() function that tests the mlp’s performance by using the mlp trained with the training data to classify the test data in the data files and report the error rate. A typical run of you mlp should look like: 

 

       NetArch: IP:8 H1:5 OP:1  

       Params:  LrnRate: 0.6  Mtm1: 1.2  Mtm2: 0.4 

 

       Training mlp for 1000 iterations: 

          #       MinErr      AveErr      MaxErr     %Correct 

          1:     0.000006    0.077862    0.713725    19.0373 

          2:     0.000000    0.072893    0.673643    17.1607 

          3:     0.000018    0.072357    0.670814    16.9976 

          4:     0.000002    0.071879    0.669441    16.9976 

          5:     0.000012    0.071394    0.668451    16.8072 

          6:     0.000005    0.071004    0.667836    17.0247 

          7:     0.000003    0.070734    0.667509    17.2151 

          8:     0.000047    0.070535    0.667491    17.4055 

          . . . . 

       1000:     0.000126    0.001256    0.008060     5.1607  

       Testing mlp: 

                  MinErr      AveErr      MaxErr     %Correct 

                 0.001126    0.008256    0.015060    10.1607 

 

       End of program. 

 

Step 4:  Your task here is to devise various back-propagation neural network of minimum size, (in terms of the number of neurons in the mlp), that can correctly classify the Two Spiral Problem (data1.txt) and the other data sets associated with Problems 2 and 3 (see below). Note: the data for problems 2 and 3 may need extra work, like normalizing the data, dividing it into training and test data and adding the mlp header information.  You should experiment with various mlps with variations in the numbers of layers, number of neurons in each layer, parameters (eg learning rate, momentum) and the number of training iterations. If you are unable to classify all the data correctly then your final mlp configuration should be a best case compromise between size and performance. For each data set you should write a report comprised of the following information: 

 

1)      A brief description of the problem.  

2)      A progress report of the various mlps (at least 3 mlps) that you experimented with including any parameter changes you made and the results that were achieved. Try running each mlp a few times to determine if the problem has local minimums. If so state this in the report and see if this can be avoided by increasing the momentum. You should also indicate the approximate architecture that you think is most suited for the problem and how long and the number of iterations this takes to learn the data. (eg “ . . . the best mlp for the problem appears to be a network with a large number of hidden layers with few nodes in each layer. . . ”).  

Use graphs if you think this is helpful in understanding the performance differences of the various mlps. 

3)      Report on the final mlp that you consider is either the one that classifies all the data correctly or is a best case compromise between size and performance. Provide a description of the final mlp architecture together with a drawing of the mlp. Also provide a graph showing the iterations vs error rate. (Note: for binary classification problems, the error rate should indicate the percentage of patterns correctly classified on the training and test data.)  

4)      Write a brief summary of the final mlp’s performance. Was the final mlp able to learn all the training data         and classify the test data correctly? If not why not? Are there local minimums with this mlp?