Naïve Bayes Model
For simplicity, we only consider binary features
The generation model is
Here: A Bernoulli distribution parametrized by 𝜋 means that It is a special case of categorical distributions in that only two cases are considered.Pr[𝑋 = 1] = π Pr[𝑋 = 0] = 1 − π and .
Such a model can be used to represent a document in text classification. For example, the target indicates Spam or NotSpam. The feature indicates if a word in the vocabulary occurs in the document.
Problem 1. Please show that the parameters of naïve Bayes decompose, i.e., the probability factorizes (for the same reason as Gaussian mixture models).
Problem 2. Write out the MLE for naïve Bayes (which is simply counting).
Hint: No proof is needed for the second part, because the MLE for categorical distribution has been clear in the Gaussian mixture models.
