Exercise 1
1. Write the factorization of the joint distribution 𝑃(𝐴, 𝐵, 𝐶, 𝐷, 𝐸, 𝐹) corresponding to the Bayesian network:
2. Indicate whether the following statements on conditional indepence are True or False and motivate your answer. a. A ⊥⊥ B
b. A ⊥⊥ F
c. A ⊥⊥ C | {B,E}
d. F ⊥⊥ D | B
e. B ⊥⊥ D | C
Exercise 2
Topic models are statistical models that learn the distribution of the abstract topics occurring in a collection of documents. In this context, documents are collections of topics and topics are collections of words.
We consider a dictionary of 𝑁 words, an ordered collection of 𝐷 documents and an ordered collection of 𝑇 possible topics appearing in each document. We build our model using the indexes representing each quantity in its collection (e.g. 𝑖 is the 𝑖-th word in the dictionary). For simplicity, we assume that all documents contain the same number of words. Each document can contain multiple topics; specifically, we associate a topic to each word appearing in the document.
We make the following assumptions:
𝐷 is the total number of documents
𝑇 is the total number of topics
𝑊 < 𝑁 is the number of words per document
For the 𝑗-th word and the 𝑖-th document, we sample a topic 𝑡𝑖,𝑗 from a Categorical distribution on 𝜃𝑖
The distribution 𝜃𝑖 of topics in the 𝑖-th document is a Dirichlet distribution with concentration parameters 𝛼 (the length of 𝛼 is 𝑇)
The 𝑗-th word in the 𝑖-th document, namely 𝑤𝑖,𝑗, is sampled from a Categorical distribution on 𝜑𝑡𝑖,𝑗
The distribution 𝜑𝑘 of words per topic 𝑘 is a Dirichlet distribution with concentration parameters 𝛽 (the length of 𝛽 is 𝑁)
1. Write the generative process of the following graphical model. Be aware of plate notation!
2. Set 𝑇 = 5, 𝐷 = 10, 𝑊 = 50, 𝑁 = 100 and use pyro to implement this graphical model as a function of the hyperparameters model(alpha, beta) , that outputs theta, phi, t, w . 3. Evaluate your function on the hyperparameters
alpha = torch.tensor([0.5, 0.3, 0.1, 0.4, 0.2])
beta = dist.Gamma(1./T, 1.).sample((W,)) and print the shape of the output tensors theta, phi, t, w .
