Starting from:
$34.99
CS7015 Solution
Instructions:
• This assignment is meant to help you grok certain concepts we will use in the course.
Please don’t copy solutions from any sources.
• Avoid verbosity.
• The assignment needs to be written in latex using the attached tex file. The solution for each question should be written in the solution block in space already provided in the tex file. Handwritten assignments will not be accepted.
1. Partial Derivatives
(a) Find the derivative of g(ρ) with respect to ρ where g(ρ) is given by,
(You can consider ˆρ as constant)
Solution:
(b) Consider the following computation ,
by definition : and
The value L is given by,
L = −y log(f(x))
Here, x and y are constants and w and b are parameters that can be modified. In other words, L is a function of w and b.
Derive the partial derivatives, and .
Solution:
2. Chain Rule:
(a) Consider the evaluation of E as given below,
E = h(u,v,w,x) = 2 ∗ f(au + bv) − g(cw + dx))
Represented as graph:
Here u,v,w,x are inputs (constants) and a,b,c,d are parameters (variables). f and g are the activation functions (with z as input) defined as below:
f(z) = sigm(z) g(z) = tanh(z)
Note that here E is a function of parameters a,b,c,d. Compute the partial derivatives of E with respect to the parameters a, b, c and d i.e. ∂E∂a , ∂E∂b , ∂E∂c and .
Solution:
(b) Assume that z = f(x,y), where x = uv and . We use fx to denote the partial derivative . Using the chain rule, express and in terms of u, v, fx and fy.
Solution:
(c) Given the change of variables as mentioned in the previous part: x = uv and , calculate the Jacobian of this transformation.
Solution:
(d) Calculate the Jacobian of the transformation for rectangular coordinates; i.e., the Jacobian of x = rsinθ, y = rcosθ, z = z, (hint: using the relevant partial derivatives) Solution:
3. Visit Taylor Series The first order derivative of a function f is defined by the following limit,
(1)
On observing the above definition we see that the derivative of a function is the ratio of change in the function value to the change in the function input, when we change the input by a small quantity (infinitesimally small). A first degree approximation based on eq. 1 would be the following.
(2)
Consider f(x) = ln(x + 5).
(a) Estimate the value of f(1),f(1.1) and f(2.5) using the above formula.
Solution:
(b) Compare these estimates to the actual values of function f(1),f(1.1) and f(2.5). Explain the discrepancy as we increase the value.
Solution:
(c) Can we get a better estimate of f(1),f(1.1) and f(2.5) ? How?
Solution:
(d) Consider g(x) = a + bex + c ∗ cos(x). Find a, b, c ∈ R such that g approximates f at x = 0. (i.e. by matching the (i) direct values (ii) first derivative and (iii) second derivative at x = 0).
Solution:
4. Differentiation of function of multiple variables
s1 = tan(w1x) s2 = sec(w2x + s1) s3 = sin(w3s1 + w4s2) s4 = cos(w5s3 + w6) y = tanh(w7s4 + w8)
An alternative representation of the function y is given in the figure below.
Compute the derivatives and (show all the steps).
Solution:
5. Differentiation of vectors/matrices Consider vectors u,b ∈Rd, and matrix A ∈Rn×n.
The derivative of a scalar f w.r.t. a vector u is a vector by itself, given by
( Hint: The derivative of a scalar f w.r.t. a matrix X, is a matrix whose (i,j) component is , where Xij is the (i,j) component of the matrix X.)
(a) Derive the expression for the derivative: ∇uTAu + bTu.
Solution:
(b) Compare your results with derivatives for the scalar equivalents of the above expressions au2 + bu.
Solution:
(c) Derive the Hessian: given that f = uTAu + bTu
Solution:
(a) One way to encode this sequence is to use fixed length code with each code word long enough to encode ten different symbols. How many bits would be needed for this 33-character phrase using such a fixed-length code?
Solution:
(b) What are the minimum number of bits needed (theoretically) to encode the entire phrase , assuming that each character is independent of the surrounding character? Hint: We can calculate the average information (in other words, bits needed) of a symbol using entropy information.
Solution:
7. Plotting Functions for Great Good
(a) Consider the variable x and functions h11(x), h12(x) and h21(x) such that
The above set of functions are summarized in the graph below.
Plot the following functions: h11(x), h12(x) and h21(x) for x ∈ (−1,1)
Solution:
(b) Now consider the variables x1,x2 and the functions h11(x1,x2),h12(x1,x2),h13(x1,x2),h14(x1,x2), h21(x1,x2),h22(x1,x2),h31(x1,x2) and f(x1,x2) such that
The above set of functions are summarized in the graph below.
Plot the following functions: h11(x1,x2),h12(x1,x2),h13(x1,x2),h14(x1,x2),h21(x1,x2), h22(x1,x2),h31(x1,x2) and f(x1,x2) for x1 ∈ (−5,5) and x2 ∈ (−5,5)
Solution:
• This assignment is meant to help you grok certain concepts we will use in the course.
Please don’t copy solutions from any sources.
• Avoid verbosity.
• The assignment needs to be written in latex using the attached tex file. The solution for each question should be written in the solution block in space already provided in the tex file. Handwritten assignments will not be accepted.
1. Partial Derivatives
(a) Find the derivative of g(ρ) with respect to ρ where g(ρ) is given by,
(You can consider ˆρ as constant)
Solution:
(b) Consider the following computation ,
by definition : and
The value L is given by,
L = −y log(f(x))
Here, x and y are constants and w and b are parameters that can be modified. In other words, L is a function of w and b.
Derive the partial derivatives, and .
Solution:
2. Chain Rule:
(a) Consider the evaluation of E as given below,
E = h(u,v,w,x) = 2 ∗ f(au + bv) − g(cw + dx))
Represented as graph:
Here u,v,w,x are inputs (constants) and a,b,c,d are parameters (variables). f and g are the activation functions (with z as input) defined as below:
f(z) = sigm(z) g(z) = tanh(z)
Note that here E is a function of parameters a,b,c,d. Compute the partial derivatives of E with respect to the parameters a, b, c and d i.e. ∂E∂a , ∂E∂b , ∂E∂c and .
Solution:
(b) Assume that z = f(x,y), where x = uv and . We use fx to denote the partial derivative . Using the chain rule, express and in terms of u, v, fx and fy.
Solution:
(c) Given the change of variables as mentioned in the previous part: x = uv and , calculate the Jacobian of this transformation.
Solution:
(d) Calculate the Jacobian of the transformation for rectangular coordinates; i.e., the Jacobian of x = rsinθ, y = rcosθ, z = z, (hint: using the relevant partial derivatives) Solution:
3. Visit Taylor Series The first order derivative of a function f is defined by the following limit,
(1)
On observing the above definition we see that the derivative of a function is the ratio of change in the function value to the change in the function input, when we change the input by a small quantity (infinitesimally small). A first degree approximation based on eq. 1 would be the following.
(2)
Consider f(x) = ln(x + 5).
(a) Estimate the value of f(1),f(1.1) and f(2.5) using the above formula.
Solution:
(b) Compare these estimates to the actual values of function f(1),f(1.1) and f(2.5). Explain the discrepancy as we increase the value.
Solution:
(c) Can we get a better estimate of f(1),f(1.1) and f(2.5) ? How?
Solution:
(d) Consider g(x) = a + bex + c ∗ cos(x). Find a, b, c ∈ R such that g approximates f at x = 0. (i.e. by matching the (i) direct values (ii) first derivative and (iii) second derivative at x = 0).
Solution:
4. Differentiation of function of multiple variables
s1 = tan(w1x) s2 = sec(w2x + s1) s3 = sin(w3s1 + w4s2) s4 = cos(w5s3 + w6) y = tanh(w7s4 + w8)
An alternative representation of the function y is given in the figure below.
Compute the derivatives and (show all the steps).
Solution:
5. Differentiation of vectors/matrices Consider vectors u,b ∈Rd, and matrix A ∈Rn×n.
The derivative of a scalar f w.r.t. a vector u is a vector by itself, given by
( Hint: The derivative of a scalar f w.r.t. a matrix X, is a matrix whose (i,j) component is , where Xij is the (i,j) component of the matrix X.)
(a) Derive the expression for the derivative: ∇uTAu + bTu.
Solution:
(b) Compare your results with derivatives for the scalar equivalents of the above expressions au2 + bu.
Solution:
(c) Derive the Hessian: given that f = uTAu + bTu
Solution:
(a) One way to encode this sequence is to use fixed length code with each code word long enough to encode ten different symbols. How many bits would be needed for this 33-character phrase using such a fixed-length code?
Solution:
(b) What are the minimum number of bits needed (theoretically) to encode the entire phrase , assuming that each character is independent of the surrounding character? Hint: We can calculate the average information (in other words, bits needed) of a symbol using entropy information.
Solution:
7. Plotting Functions for Great Good
(a) Consider the variable x and functions h11(x), h12(x) and h21(x) such that
The above set of functions are summarized in the graph below.
Plot the following functions: h11(x), h12(x) and h21(x) for x ∈ (−1,1)
Solution:
(b) Now consider the variables x1,x2 and the functions h11(x1,x2),h12(x1,x2),h13(x1,x2),h14(x1,x2), h21(x1,x2),h22(x1,x2),h31(x1,x2) and f(x1,x2) such that
The above set of functions are summarized in the graph below.
Plot the following functions: h11(x1,x2),h12(x1,x2),h13(x1,x2),h14(x1,x2),h21(x1,x2), h22(x1,x2),h31(x1,x2) and f(x1,x2) for x1 ∈ (−5,5) and x2 ∈ (−5,5)
Solution:
1 file (743.4KB)
