EECS491-Assignment 2 Solved

Q1. Conditional Independence 
Consider the following Bayes net.
 Activating environment at `~/Dropbox/teaching/PM/PM-notebooks/Project.toml`
1.1.  Show that is independent of given no other infomration, i.e.
1.2. Prove or disprove the following using basic probability (i.e. not using d-separation)
Q2. Conditional Independence and Causality )
Consider the following model
Show that this causal relationship suggested by the arrows does not necessarily hold, because the identical
distribution can be represented by a model defined by different conditional distributions. What conditional
independence assumption does this model make?
Q3. Model Complexity, Free Parameters, and Simplifying Assumptions )
3.1.  Consider a general joint probability distribution with variables each of which can have
values. What is the expression for the joint distribution in terms of conditional probabilities?
3.2.  What is the total number of free-paramters requried to specify this model? (Note: the term "free
parameter" means a parameter that is unconstrained. For example a Beroulli distribution to describe a coin flip
has one free parameter to describe, say, the probability of heads; the probability of tails must be ,
because the probability is constrained to sum to one.) Provide both the exact expression and a simpler one in
"big-O" notation.
3.3. Now suppose that the complexity of the model is constrained, so that each variable depends on (at
most) other variables and is conditionally independent of the rest, i.e. a Bayes net. Each node has parents
and there are root nodes. How many parameters are required to define this model?
3.4.  Let us make one more simplifying assumption, which is that in addition to depending on only
variables, the conditional probability is described by a noisy-OR function (K=2, see Q3). What is the expression
for the number of parameters in this case?
Q4. Models of Conditional Probability 
In Bayesian networks (or directed acyclic graphical models), the joint probability distribution is factored into the
product of conditional probability distributions
As we used the previous problem, a simplifying assumption for the conditional probability is noisy-OR model
where is an index over the parents of . Note that the exponent is either 0 or 1 so the term is either 1 or
 depending on the state of the parent .
4.1  Show that the noisy-OR function can be interpreted as a "soft" (i.e. probabilistic) form of the logical
OR function, i.e. the function gives whenever at least one of the parents is 1.
4.2  What is the interpretation of ? Provide a clear explanation.
Another choice for the conditional probability is a sigmoid function
where is the logistic sigmoid function.
4.3 Contrast the noisy-OR function and the sigmoid mathematically. Is one more general than the other?
Can each compute unique functions?
4.4 Think of two examples, one for the noisy-OR and one for the sigmoid, that contrast the way these
functions model the conditional dependencies. Explain how each is appropriately modeled by one function but
not the other.
Q5. Car Troubles 
(Adpted from Barber Exercise 3.6) Your friend has car trouble. The probability of the car starting is described by
the model below, with the probabilities givien in Barber 3.6.
5.1  Calculate the , the probability of the fuel tank being empty given that the car
does not start. Do this "by hand", i.e in manner similar to the Inference section in Barber 3.1.1. Use the
probabilities given in the exercise. Show your work.
5.2  Implement this network using a toolbox for probabilistic models (e.g. pgmpy or BayesNets.jl ).
Use this to verify that your derivation and calculations are correct for the previous problem.
5.3 ( Suppose you have loaned this car to a friend. They call call you and announce, "the car won't start".
Illustrate your diagnostic and inference process by using the model to show how your beliefs change as you ask
questions. Your friend can only tell you the states of and (and you already know ). Use two different
scenarios, i.e. two differnt reasons why the car won't start. For each scenario, your answer should discuss your
choice of each question you pose to the network, and how it allows you to uncover the true cause the problem.
Exploration 
Like in the first assignment, in this exercise, you have more lattiude and are meant to do creative exploration. Like
before you don't need to write a book chapter, but the intention is for you to go beyond what's been covered
above.
Implement a belief network of your own choosing or design. It should be more complex that the examples above.
It should be discrete (we will cover continous models later). Use the model to illustrate deductive inference
problems.