In this homework, you will implement an expectation-maximization (EM) clustering algorithm in Matlab, Python, or R. Here are the steps you need to follow:
Generate random data points from five bivariate Gaussian densities with the following parameters:
𝜇! = #++22..55( ,
Σ! = #+−00..86
−0.6( ,
+0.8
𝑁! = 50
𝜇" = #−+22..55( ,
Σ" = #++00..86
+0.6( ,
+0.8
𝑁" = 50
𝜇# = #−−22..55( ,
Σ# = #+−00..86
−0.6( ,
+0.8
𝑁# = 50
𝜇$ = #+−22..55( ,
Σ$ = #++00..86
+0.6( ,
+0.8
𝑁$ = 50
𝜇% = #++00..00( ,
Σ% = #++10..60
+0.0( ,
+1.6
𝑁% = 100
Your data points should be similar to the following figure.
To initialize your EM algorithm from a good initial solution, run 𝑘-means clustering algorithm with 𝑘 = 5 just for two iterations.
After running 𝑘-means clustering algorithm for two iterations, take centroids as the initial values for the mean vectors in your EM algorithm. Using the data points covered by each center, estimate the initial covariance matrices and prior probabilities in your EM algorithm.
After the initialization step, run your EM algorithm for 100 iterations. Report the mean vectors your EM algorithm finds. Your results should be similar to the following matrix.
[,1] [,2]
## [1,] -2.0441920 -2.69776844
## [2,] 2.6622246 -2.30911081
## [3,] 2.4887435 2.67687075
## [4,] -2.6759195 2.44658904
## [5,] 0.1553517 0.05773829
Draw the clustering result obtained by your EM algorithm by coloring each cluster with a different color. You should also draw the original Gaussian densities you use to generate data points and the Gaussian densities your EM algorithm finds with dashed and solid lines, respectively. Draw these Gaussian densities where their values are equal to 0.05. Your figure should be similar to the following figure.
