Neural-Project 1 Solved

Project 1: (Adaline, Back Propagation and Some Transfer Learning) 

The details of the Homework: 

  x, y <= 100.  The data is all data points <x, y> where   x is of the form m/100 where m is an integer between -10000 and +10000  and y is of the form  n/100 with n an integer between 10000 and +10000.  Suppose that all data points with y >1  have the value 1; all other points have the value -1.

Now  suppose you do not know this; but you are given a random sample of data of size 1000 together with its value (e.g. the point  <601/100,  802/100> has value 1; while the point  <8000/100, 70/100> has the value -1.

(Part A and Part B are due by April 29)

Part A.    Implement the Adaline learning algorithm and show how it generalizes to develop a decision that works on all the set.  That is, you randomly select a training data set which is a random sample of 1000 data points of all the data points.    You use the training data set as input to the Adaline algorithm.     

Now randomly select a “test” set of 1000 data points and see how well your trained Adaline performs.    Pick a second random test set and see if the results change.

What can you conclude about your results?   How  

    Does the accuracy of the result depend on the training set?  Present tables and possibly a picture indicating your results.   Discuss the impact on the size of training set and choice of test set.

Part B:   Now change the problem so that points such that <x.y> has value 1 only if

4  <= x^2 + y^2 <= 9

What are the best results you obtain using an Adaline?   Does the quality of the results change if you use more data?  Present tables and perhaps a figure.

Part C and Part D are due by May 13  

Part C:    Now try the same with a back-propagation algorithm instead of the adaline.

You will have to define the architecture (i.e number of neurons and number of levels)   You may either implement the algorithm or use a package.  BUT YOU WILL NEED TO LOOK INSIDE the results of each neuron separately for Part D

Show a geometric diagram in terms of the inputs of the training set for the output of each neuron separately in the neural network as well as for the output neuron.   Present tables of results both for training and testing.

Part D:   Now use the trained neurons from the next to last level of Part C as input and only an Adaline for the output.    (That is, you will give the adaline the output of the neurons from Part 3 in the level below the output,  and train only the Adaline.)

Describe how accurate the Adaline can be.  Give diagrams.

Draw whatever conclusions you think are appropriate from your results and report them.