1. Bayesian Inference for Univariate Normal: Consider an IID sample for a Normal variable
with mean µ and precision λ, and consider a Normal-Gamma prior given by
) Gamma(λ|a0,b0).
Calculate the posterior for µ and λ and show that it is also decomposable as a Normal-Gamma distribution, with new parameters aN,bN,βN,µN. In particular explicitly calculate and show the parameters of the posterior when given in this form.
Notes: (1) The text on page 101 of the textbook is relevant for this question but the notation there can be confusing. It is therefore recommended to use the notation given here. (2) This is a special case of the Bayesian linear regression result for (w,β) derived in class but in this assignment you are asked to develop the result from first principles via completing the square.
2. Calculate one iteration of the Newton-Raphson method for minimizing the function f(x) = where x = (x1,x2)T. Use x = (1,1)T as the initial value.
3. Solve problem 4.18 (page 223) in the textbook.
4. In this question we develop the properties for the scaled-variants of exponential family distributions that were used in the derivation of GLM.
(i) Show that if p1(x) is a normalized probability density function then ) is also a normalized probability density function.
(ii) Now consider the scaled variant of the one dimensional exponential family distribution . Show that the expectation parameter θ = E[x|η,s] satisfies
(iii) Show that the variance satisfies
5. Consider a Poisson likelihood function, Poisson( , with prior, Gamma(λ|a,b), on λ. Develop a Gaussian approximation to p(λ|x,a,b) using the Laplace approximation. First develop the formula in general and then apply it to the case a = 1, b = 1, x = 3. For this case, plot the approximation and the true function for 0 ≤ λ ≤ 10.
