Question 1. Let . Find using the definition of the derivative (i.e. taking the limit of difference quotients).
Question 2. Compute the gradients of , and xAx w.r.t the input vector x and and matrix X.
Question 3. Recall the variance of X is Var(X) = E[(X − E[X])2].
1. Let X be a random variable with finite mean. Show Var(X) = E[X2] − E[X]2.
2. Let X and Z be random variables on the same probability space. Show that Var(X) = EZ[Var(X|Z)]+ VarZ(E[X|Z]). (Hint : E[X] = EY [E[X|Y ]].)
Question 4. Recall the density function of the uniform distribution on [a,b] for a < b is equal to for x ∈ [a,b] and 0 elsewhere.
1. Use the density function to compute the mean and variance of a uniform distribution on [a,b].
2. For integer n 0, derive a formula to compute the moment E[Xn] for X uniformly distributed between a and b.
Question 5. Let X ∈ X be a random variable with density function fX, and g : X → Y be continuously differentiable, where X and Y are subsets of R. Let Y := g(X), which is continuously distributed with density function fY .
1. Show that if g is monotonic, .
2. Let fX(x) = 1x∈[0,1](x) and . Find a monotonic mapping g that translates fX and fY .
Question 6. Let Q and P be univariate normal distributions with mean and variance µ,σ2 and m,s2, respectively. Derive the entropy H(Q), the cross-entropy H(Q,P), and the KL divergence DKL(Q||P).
