Note: Please do not use built-in functions for the expectation maximization algorithm..
1. Derive the expression for the optimal decorrelating linear transform for a set of observations X ∈ Rd×N where each row is assumed to be zero-mean. (5)
2. Derive the expressions for the partial derivates of the log likelihood function of a Gaussian Mixture Model (GMM) with respect to each of its parameters. Set these derivatives to zero and find the expressions for the “locally optimal” parameters in terms of the posterior probabilities and the observations. (10)
3. Implement the expectation maximization (EM) algorithm for estimating the parameters of a Gaussian Mixture Model(GMM). The GMM density is given by K p(x|θ) = ∑πkN(x; µk, Σk),
k=1
where x ∈ Rd, N(x; µ, Σ) is the multivariate Gaussian distribution with mean vector µ and covariance matrix Σ. The parameter set θ = [π1, . . . , πK, µ1, . . . µK, Σ1, . . . , ΣK]. Your program must accept as inputs the observation matrix X (of size d × N), and the mixture size K as inputs, and output the estimated parameter set θˆ. Generate X on your own and experiment by varying your choices of θ. (20)
1
