Reading: Appendices A.1 – A.5, Notes, Chapter 2.1–2.7
1.1 (10 pts) Let x be a real-valued random variable.
(a) Prove that the variance of x = 2 = E[(x µ)2] = E[x2] µ2.
(b) Let x be a real-valued random vector. Prove that the covariance matrix of x = ⌃ = E[xxT] µµT.
1.2 (10 pts) Suppose two equally probable one-dimensional densities are of the form p(x|!i) / e |x ai|/bi for i = 1,2 and b > 0.
(a) Write an analytic expression for each density, that is, normalize eachfunction for arbitrary ai, and positive bi.
(b) Calculate the likelihood ratio p(x|!1)/p(x|!2) as a function of your four variables.
(c) Plot a graph (using MATLAB) of the likelihood ratio for the casea1 = 0, b1 = 1, a2 = 1 and b2 = 2. Make sure the plots are correctly labeled (axis, titles, legend, etc) and that the fonts are legible when printed.
1.3 (10 pts) Consider a two-class problem, with classes c1 and c2 where P(c1) = P(c2) = 0.5. There is a one-dimensional feature variable x. Assume that the x data for class one is uniformly distributed between a and b, and the x data for class two is uniformly distributed between r and t. Assume that a < r < b < t. Derive a general expression for the Bayes error rate for this problem. (Hint: a sketch may help you think about the solution.)
1.4 (12 pts) Consider a two-class, one-dimensional problem where P(!1) = P(!2) and p(x|!i) ⇠ N(µi, i2). Let µ1 = 0, 12 = 1, µ2 = µ, and 22 = 2.
(a) Derive a general expression for the location of the Bayes optimaldecision boundary as a function of µ and 2.
(b) With µ = 1 and 2 = 2, make two plots using MATLAB: one for the class conditional pdfs p(x|!i) and one for the posterior probabilities p(!i|x) with the location of the optimal decision regions. Make sure the plots are correctly labeled (axis, titles, legend, etc) and that the fonts are legible when printed.
(c) Estimate the Bayes error rate pe.
(d) Comment on the case where µ = 0, and 2 is much greater than 1. Describe a practical example of a pattern classification problem where such a situation might arise.
1
