Problem 1
The linear regression model can be written in matrix notation as y = Xβ + . Create the table shown below and describe what each term (y,X,β,) represents (interpretation), specify the dimension of each term (size), indicate whether we model the term as random or non-random, and whether the term is observed or unobserved.
Term
Size
Interpretation
Random?
Observable?
y β
X
Problem 2
Assuming a simple linear regression model, derive the ordinary least squares estimators of β0 and β1. Do not use matrix notation in deriving your solution.
Problem 3
Let H = X(XTX)−1XT is the hat matrix. Prove that I−H is a projection matrix (Symmetric + Idempotent).
Problem 3
While proving that βˆ1 is an unbiased estimator of β1, we represented the OLS estimate as βˆ1 = PkiYi, where ki = PX(Xi−i−XX)2. Use it with the properties of ki, that we already have proved, to derive the variance of βˆ1. (Hint: in linear regression framework we assume V ar(i) = σ2 and Cov(i,j) = 0 when i 6= j)
Problem 4
P(yi−yˆi)2
Under simple linear regression model, the Mean Squared Error (MSE) is defined as n−2 where n is the number of observations. MSE is an unbiased estimator of σ2, where σ2 is the variance of i. What is an unbiased estimator of the variance of βˆ1?
Problem 5
The square root of the variance of an estimator is the standard error (SE). You can derive the SE (βˆ) from problem 4. According to theory,
1
βˆ1 − β1
∼ tn−2
SE(βˆ1)
where tn−2 represents a Student’s t distribution with n − 2 degrees of freedom. Find an expression for 95%
confidence interval of β1.
Problem 6
If βˆ1 = 2, SE(βˆ1) = 0.02, and n = 50 calculate 95% confidence interval for β1.
Problem 7
Based on the confidence interval on problem 6, perform the hypothesis test,
H0 : β1 = 0 Vs. H1 : β1 6= 0
Problem 8
Use the simu_hw1.txt data and fit a multiple linear regression model with response as the response variable and pred1, pred2, and pred3 as predictors. Write down the equation of the fitted line (fitted model).
