1. Consider a problem of fitting a logistic regression model to data.
(a) Write a function that can fit logistic regression to data using two optimization methods:
• Gradient descent/steepest ascent method
• Newton-Raphson/Fisher scoring/IRLS method
Your function should return MLEs of regression coefficients, their corresponding asymptotic confidence intervals, and a vector of the log-likelihoods “visited” by the chosen optimization option.
(b) Apply your function to the heart-disease data: https://hastie.su.domains/ ElemStatLearn/datasets/SAheart.data (description is here: https://hastie. su.domains/ElemStatLearn/datasets/SAheart.info.txt).
(c) Compare convergence of the two optimization options by plotting iterations vs.log-likelihood values and benchmark run times of the two optimization algorithms.
2. Implement the EM algorithm For the ABO blood type example discussed in class andapply it to the the data
n = (nA,nAB,nB,nO) = (6,4,55,35).
Package your implementation into a function that takes data and initial values of the allele frequency probabilities pA, pB, and pO as inputs. Document and add this function to your class package on github.
3. Consider the occasional dishonest casino hidden Markov model (HMM) with hiddenstates 1=fair die, 2=loaded die, transition probabilities
P ,
emission probabilities
E ,
and initial distribution
(a) Simulate (x1:100,y1:100) from this HMM and plot the hidden and observed states as time series.
1
(b) Implement the forward and backward and algorithms for the occasional dishonestcasino example. Run these algorithms on the observed states generated in part (a), compute marginal probabilities of hidden states at each time and plot them together with the true simulated states.
(c) Pretend you don’t know the initial distribution, transition probabilities, and emission probabilities corresponding to the loaded die. In other words, you only know the first row of the emission probability matrix. Implement the Baum-Welch algorithm to estimate all unknown parameters and apply it to data from part (a).
Report estimated parameters.
2
