COMP9417 Homework 0 Solution

Homework 0: Revision



Introduction The goal of this homework is to review some important mathematical concepts that are used regularly in machine learning, and which are assumed knowledge for the course. If you find yourself strug-
gling significantly with any aspects of this homework, please reach out to course staff so that we can betterhelp you prepare for the course. Please also note that we have posted some helpful resources under the

What to Submit
• A single PDF file which contains solutions to each question. For each question, provide your solutionin the form of text and requested plots. For any question in which you use code, provide a copy of
your code at the bottom of the relevant section.

• You are free to format your work in any way you think is appropriate. This can include using Lor taking pictures of handwritten work, or writing your solutions up using a tablet. Please ensureATEX,
that your work is neat, and start each question on a new page.
When and Where to Submit

• Submissions must be through Moodle - email submissions will be ignored.
Question 1. (Calculus Review) (a) Consider the function
f(x,y) = a1x2y2 + a4xy + a5x + a7
compute all first and second order derivatives of f with respect to x and y. (b) Consider the function
f(x,y) = a1x2y2 + a2x2y + a3xy2 + a4xy + a5x + a6y + a7

compute all first and second order derivatives of f with respect to x and y.
(c) Consider the logistic sigmoid:
(x) = 1 + 1e x

show that

1


(d) Consider the following functions:


• yy34 = 4= xx++x 11 x
Using the second derivative test, find all local maximum and minimum points.

Question 2. (Probability Review)
(a) products from Outlet I alone, 10% buy from both I and II, and 40% buy from neither. LetA manufacturing company has two retail outlets. It is known that 20% of potential customers buyA denote
the event that a potential customer, randomly chosen, buys from outel I, and B the event that the customer buys from outlet II. Compute the following probabilities:
tl P(A), P(B), P(A [ B), P(A¯B¯)

(b) Let X,Y be two discrete random variables, with joint probability mass functionto.zo.to40.3 P(X = x,Y = y)
displayed in the table below:
y
1 2 3
1 1/6 1/12 1/12
x 2 1/6 0 1/6
3 0 r 0
Compute the following quantities:
(i) r
(ii) P(X = 2,Y = 3)
(iii) P(X = 3) and P(X = 3|Y = 2)
(iv) E[X], E[Y ] and E[XY ]
(v) E[X2], E[Y 2]
(vi) Cov(X,Y )
(vii) Var(X) and Var(Y )
(viii) Corr(X,Y ) 2 2
(ix) E[X + Y ], E[X + Y ], Var(X + Y ) and Var(X + Y ).
Question 3. (Linear Algebra Review) (a) Write down the dimensions of the following objects:



Page 2
(b) Consider the following objects:

Compute the following:
(i) AB and BA (ii) AC and CA
(iii) AD and DA
(iv) DC and CD and DTC (v) Bu and uB
((viivi)) AuAv and vA
(viii) Av + Bv
(c) Consider the following objects:

.
46 4 35 3 415 4 2 5
Compute the following:
(i) kuk1,kuk2,kuk222,kuk1 (ii) kvk1,kvk2,kvk2,kvk1
(d) ((Consider the following vectors iniiiiv)) kkvAv+kw2,kk1A,k(vv +ww)kk21,kv + wk1R2

hx,yi = x · y = xTy.
Then compute the angle between the vectors and plot.
(e) Dot products are extremely important in machine learning, explain what it means for a dot product
to be zero, positive or negative.
(f) Consider the 2 ⇥ 2 matrix:
1 3
A = 4 1
Compute the inverse of A.
(g) Consider the 2 ⇥ 2 matrix

Compute its inverse A 1.
(h) Let X be a matrix (of any dimension), show that XTX is always symmetric.

Page 3





























b ⻔ r 1 - 合⼀台⼀⾔⼀⽅⼀个冷
ii ⽐如ㄚ 3 ⼆点灿⼆号
iii Plx 3 r 5 P X31ㄚ 2 ⾔⾔ ⼆号
Elx 这⼗六⼗六⼋⼋⼗这坨 2⼗号 3 2
iv
E Y ⼆个㘭州六⼗台 2 占㘭 3 告

END lxlxitlxzxitlx 3XEt

zxlxitz NO ⼗ 213恬⼗

3Xlxot 3112X513ㄨ3 0 岩

ME 们这⼗六⼗六⼋作这㘭⼼兮 5
E们⼆个㘭州六⼗㫃122 占㘭 32 号

以⽠炒 E XD EM El红岩 - 2ㄨ吾⼆台

vii VarX EM tn ⼆号- 22 ⾔
VarY E ⽕ - 欧丫⼆千⼀逼忙是
以炒
viii lorry 1 - 4 - jÉ- y 0.134
EM Y EM HE D 21䛒答
ix
EM YI E XHEMI 2⼗朵娑
Var HY VarMt VarY - 2⼮炒三弘瑟 - 2 炲毙
Var HYIVarMtvarlYJ zcux li


Question3
a ThedimensionofAis 3 xjbisbxl ATis5113
b DA is 3Band Bis 2ㄨ2 hencebothABandBAcannotbe computed ii AC d21 1414 叫䥷 别

e
It can show the similarity of the items, The sign of the dot product determines the angle 0 means 900， positive means less than 900， negative means larger than 900