In class we studied the linear regression model y = Xθ? + , under the homoskedastic assumption ∼ N(0,σ2I). In this homework you will derive the same results for the slightly more general heteroskedastic model where ∼ N(0,Σ?). Each subproblem is worth 10 points.
Problem 2.1. Derive an expression for the coefficient vector θ that minimizes the mean squared error, i.e.,
argmin .
θ
Problem 2.2. Derive an expression for the maximum likelihood estimator (MLE) of θ?, i.e.,
argmax P(y,X|θ,Σ?)
θ
Problem 2.3. What is the distribution of the MLE of θ??
Problem 2.4. Given a new sample with feature vector x, what is the MLE of the response, ˆy?
Problem 2.5. Given a new sample with feature vector x, what is the distribution of the MLE ˆy? Problem 2.6. Derive an expression for the MLE of Σ?, i.e.,
argmax
Σ
Problem 2.7. Consider the following vector y, containing information about glucose level of four individuals, and the following data matrix X containing information about height and weight of the corresponding individuals:
110
y = 140180, X.
190
Given these data
(a) What are your maximum likelihood estimates of Σ and θ??
(b) Given a new sample with feature vector x = [175 170]T, what is the maximum likelihood estimate of its response yˆ?
(c) Derive a 95% confidence interval for yˆ.
(d) Would you conclude that height is a significant feature for this model? Why?
(e) Would you conclude that weight is a significant feature for this model? Why?
2-1
