ML-Homework 3 Solved

1. Random Data Generator    a. Univariate gaussian data generator

  Expectation value or mean:

Variance:  

Output: A data point from

Generating values from normal distribution

You have to handcraft your geneartor based on one of the approaches given in the hyperlink.

You can use uniform distribution function (Numpy)

 

2.  Sequential Estimator  
Sequential estimate the mean and variance

 Data is given from the univariate gaussian data generator (1.a). Input:      as in (1.a) Function:

Call (1.a) to get a new data point from  

                     Use sequential estimation to find the current estimates to     and

Repeat steps above until the estimates converge.

          Output: Print the new data point and the current estimiates of          and  in each iteration.

Notes

 You should derive the recursive function of mean and variance based on the sequential esitmation.

Hint: Online algorithm

          Sample input & output (     for reference only     )

1 Data point source function: N(3.0, 5.0)

2

3     Add data point: 3.234685454257290  

4     Mean = 3.408993960833291   Variance = 0.030383455464755956

5     Add data point: 0.519242879651157   

6     Mean = 2.445743600439247   Variance = 1.875958150575018

7     Add data point: 1.347113997201991   

8     Mean = 2.171086199629932   Variance = 1.633278676389248

9     Add data point: 8.979491998496083   

10   Mean = 3.532767359403163   Variance = 8.723325264636875

11   Add data point: 3.603448448693051   

12   Mean = 3.544547540951477   Variance = 7.270131583917285

13   Add data point: 4.127197937610908   

14   Mean = 3.627783311902824   Variance = 6.273110519038578

15   Add data point: 4.992735798186870   

16   Mean = 3.798402372688330   Variance = 5.692747751482052

17

18 ...

19

20    Add data point: 4.233592159021013 

21    Mean = 2.961576104513964   Variance = 5.045715437349161

22    Add data point: 3.529990930040463   

23    Mean = 2.961883688294010   Variance = 5.043159812425648

24    Add data point: 1.125210345431449   

25    Mean = 2.960890354955524   Variance = 5.042255747918937

3.  Baysian Linear regression  
 Input

The precision (i.e., b) for initial prior  

All other required inputs for the polynomial basis linear model geneartor (1.b)

Function

Call (1.b) to generate one data point

Update the prior, and calculate the parameters of predictive distribution Repeat steps above until the posterior probability converges.

Output

Print the new data point and the current paramters for posterior and predictive distribution.

 After probability converged, do the visualization

Ground truth function (from linear model generator)

Final predict result

At the time that have seen 10 data points

At the time that have seen 50 data points

Except ground truth, you have to draw those data points which you have seen before

Draw a black line to represent the mean of function at each point

 Draw two red lines to represent the variance of function at each point

In other words, distance between red line and mean is ONE variance