CSE514 Homework 2 Solution

Q1 (6pts): A friend informs you that a casino is using loaded dice, such that:

1112 𝑖𝑓 𝑘 ∈ {1, 2, 3}
𝑙𝑜𝑎𝑑𝑒𝑑! = 𝑝𝑟𝑜𝑏 𝑖𝑓 𝑘 ∈ {4, 5, 6}
0 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒

Q1a: What is the entropy of a roll at this casino? Please use log base 2








Q1b: Imagine your friend is right, but you choose to give the casino the benefit of the doubt and assume fair dice. What’s the KL divergence of fair dice (Q) from the true
distribution (P)? i.e. calculate 𝐷"#(𝑙𝑜𝑎𝑑𝑒𝑑 𝑑𝑖𝑐𝑒||𝑓𝑎𝑖𝑟 𝑑𝑖𝑐𝑒)
Please use log base 2








Q1c: Imagine you choose to believe your friend, but it turned out the casino has since switched back to fair dice. What’s the KL divergence of the loaded dice (Q) from the true distribution (P)? i.e. calculate 𝐷"#(𝑓𝑎𝑖𝑟 𝑑𝑖𝑐𝑒||𝑙𝑜𝑎𝑑𝑒𝑑 𝑑𝑖𝑐𝑒)
Please use log base 2








Q2 (5pts): Given this ANN structure:

And the following parameter/function definitions:

W = [-15, -3, -2, 4, 1, 10] B = [4, 1, -0.5]

𝑓$(𝑥) = 𝑓%(𝑥) = max (0.1𝑥, 𝑥) 𝑓&(𝑥) = 𝑥%

What are the intermediate and/or output values for the following data points?

Q2a: Data point: a = 0.5, c = 0.5

𝑎′$ value: 𝑎′% value:


𝑐′$ value:


Q2b: Data point: a = 1, c = 0

𝑐′% value: 𝑓$ value:


𝑓% value:


Q2c: Data point: a = 0, c = 1

𝑥′$ value: 𝑥′% value:


𝑓& value:


Q3 (4pts): Given a test data point:
Height = 200
Weight = 200

And the training dataset in the table below, use kNN classification with k=1, k=3, and k=5 to label the test data point. Break ties by increasing k by 1.
Show your work by filling in the table and writing in the model’s class label predictions.

Class Height Weight Manhattan Distance from test sample
1 105 114
1 92 169
1 87 140
2 111 109
2 79 44
2 92 55
3 265 331
3 330 284
3 185 309


Model predictions for:


k = 1 _________ k = 3 _________ k = 5 _________