EECS491-Assignment 1 Solved

Q1. Basic probability 
In the proofs below you should use general probability distributions (as opposed to specific examples) and the
basic laws of probability. Be concise and clear. The proof should be in terms of mathematical facts of probability
theory.
1.1. Prove 
1.2. Prove 
Q2. Independence 
Again these proofs should use general probability distributions and the basic laws of probability. Note that the proof
should be in terms of mathematical facts. It should not be an argument that depends on real-world knowledge. The
example should use common real-world knowledge and interpretation should convey the ideas of the proof.
2.1 Prove that independence is not transitive, i.e. . Define a joint probability distribution
 for which the previous expression holds and provide an example with an interpretation. 
2.2 Prove that conditional independence does not imply marginal independence, i.e. . Again
provide an example that illustrates the statement. 
Q3. Inspector Clouseau re-revisited 
3.1 Write a program to evaluate in Example 1.3 in Barber. Write your code and choose your data
representations so that it is easy to use it to solve the remaining questions. Show that it correctly computes the
value in the example. 
3.2 Define a different distribution for . Your new distribution should result in the outcome that
is either or , i.e. reasonably strong evidence. Use the original values of and from the
example. Provide (invent) a reasonble justification for the value of each entry in . 
3.3 Derive the equation for . 
3.4 Calculate it's value for both the original and the one you defined yourself. Is it possible to provide a
summary of the main factors that contributed to the value? Why/Why not? Explain. 
Q4. Biased views 
4.1 Write a program that calculates the posterior distribution of the (probability of heads) from the Binomial
distribution given heads out of trials. Feel to use a package where the necessary distributions are defined as
primitives.
4.2 Imagine three different views on the coin bias:
"I believe strongly that the coin is biased to either mostly heads or mostly tails."
"I believe strongly that the coin is unbiased".
"I don't know anything about the bias of the coin."
Define and plot prior distributions that expresses each of these beliefs. Provide a brief explanation. 
4.3 Perform Bernoulli trials where one of these views is correct. Show how the posterior distribution of changes
for each view for =0, 1, 2, 5, 10, and 100. Each view should have its own plot, but with the curves of the posterior
after different numbers of trials overlayed. 
4.4 Is it possible that each view will always arrive at an accurate estimate of ? How might you determine which
view is most consistent with the data after trials?
Q5. Inference using the Poisson distribution 
Suppose you observe for 3 seconds and detect a series of events that occur at the following times (in seconds):
0.53, 0.65, 0.91, 1.19, 1.30, 1.33, 1.90, 2.01, 2.48.
5.1 Model the rate at which the events are produced using a Poisson distribution where is the number of events
observed per unit time (1 second). Show the likelihood equation and plot it for three different values of : less,
about equal, and greater than what you estimate (intuitively) from the data. 
5.2 Derive the posterior distribution of assuming a Gamma prior (usually defined with parameters and ). The
posterior should have the form where is the total duration of the observation period and is the
number of events observed within that period. 
5.3 Show that the Gamma distribution is a conjugate prior for the Poisson distribution, i.e. it is also a Gamma
distribution, but defined by parameters and that are functions of the prior and likelihood parameters. 
5.4 Plot the posterior distribution for the data above at times = 0, 0.5, and 1.5. Overlay the curves on a single plot.
Comment how it is possible for your beliefs to change even though no new events have been observed. 
Q6. Probability Distribution Example 
In this problem you will illustrate a probability distribution in a settings of your choosing. It can be discrete or
continuous. This is meant to be a simpler version of the letter seqeunce example shown in class (so don't use that).
Your example should use two random variables that each have at least three distinct values (if it is discrete), i.e.
don't use binary variables. The variables should not be independent, in other words, the setting you are modeling
should have structure, and ideally structure that is interesting and interpretable in some way. Your example should
include the following:
a decription of the setting
an illustration of the joint probability and how it captures the structure
an illustration of a conditional probability
an illustration of marginal probability
Note that "illustration" here means to explain with tables or figures that convey the ideas of the mathematical
operations. The motivation behind this exercise is to help you develop a better understanding of how joint
probability distributions model probabilistic structure in a simplified setting, so try to choose something you are
very familiar with. If find this is getting too long, you can continue it as part of the exploration, but there you will
also need to add and inference problem.
Exploration 
In these problems, you are meant to do creative exploration. Define and explore:
E.1 A discrete inference problem 
E.2 A continuous inference problem 
This is meant to be open-ended; you should not feel the need to write a book chapter; but neither should you just
change the numbers in one of the problems above. After doing the readings and problems above, you should pick a
concept you want to understand better or an simple modeling idea you want to try out. You can also start to explore
ideas for your project. The general idea is for you to teach yourself (and potentially a classate) about a concept
from the assignments and readings or solidify your understanding of required technical background.
You can use the readings and other sources for inspiration, but here are a few ideas:
An inference problem using categorical data
A disease for which there are two different tests
A two-dimensional continuous inference problem
The idea of a conjugate prior