Statistical-Inferencing Homework 1 Solution

Question 1
Suppose that X is a discrete random variable with

where 0 ≤ θ ≤ 1 is a parameter. The following 10 independent observations were taken from such a distribution: (3,0,2,1,3,2,1,0,2,1).
1. What is the maximum likelihood estimate of θ?
Hint: To find the MLE of θ, we first define the likelihood function:
lik(θ) = f (x1,...,xn | θ) = f (x1 | θ)···f (xn | θ)
= P (X = x1 | θ)···P (X = xn | θ)
Now, obtain the logarithm, l(θ) = ln(lik(θ)), and maximize w.r.t. θ.
2. Prove that
and
3. What is an approximate standard error of the maximum likelihood estimate? Hint: The approximate variance of error in a maximum likelihood estimator is given as (we yet have to cover this material in the class)

The approximate standard error is simply the square root of Var(θ˜).
Question 2
Suppose that X follows a geometric distribution,
P(X = k) = p(1 − p)k−1
and assume an i.i.d. sample of size n.
1. Find the method of moments estimate of p.
2. Find the mle of p.
3. Show that
.
where l(p) = ln(lik(p)).
4. Find the asymptotic variance of the mle.
5. In an ecological study of the feeding behavior of birds, the number of hops between flights was counted for several birds. For the following data, fit a geometric distribution:

Hint: The geometric event X = k or X = x, where k (or x) is a positive integer, means that in k consecutive trials, first k − 1 are failures each with probability (1 − p), and kth trial is a success with probability p. To find the MLE of p, define the likelihood function:
lik(p) = f (x1,...,xn | p) = f (x1 | p)···f (xn | p)
= P (X = x1 | p)···P (X = xn | p)
Substituting the definition of the probability distribution function of X yields
lik( .
Question 3
Consider an i.i.d. sample of random variables with Laplacian density function

1. Find the method of moments estimate of σ.
2. Find the maximum likelihood estimate of σ.
Question 4
Suppose that X1,X2,...,X25 are i.i.d. , where µ = 0 and σ = 10.
Plot the sampling distributions of X¯ and ˆσ2. [This will be discussed in the class.]
Question 5
The following 16 numbers came from normal random number generator on a computer:
5.3299 4.2537 3.1502 3.7032 1.6070 6.3923 3.1181
6.5941 3.5281 4.7433 0.1077 1.5977 5.4920 1.7220
4.1547 2.2799
1. What would you guess the mean and variance ( µ and σ2 ) of the generating normal distribution were?
2. Give 90%,95%, and 99% confidence intervals for µ and σ2.
3. Give 90%,95%, and 99% confidence intervals for σ.
4. How much larger a sample do you think you would need to halve the length of the confidence interval for µ ?
[This will be discussed in the class.]
Question 6
The Weibull distribution is defined in terms of CDF as follows:
F(x) = 1 − e−(x/α)β, x ≥ 0, α > 0, β > 0
This distribution is sometimes fit to lifetimes.
Its density function is simply the derivative of the above cumulative distribution function, so

Let’s find the MLE for α and β (actually, we’ll see that they can’t be found in the closed form, but can only be obtained numerically using some expressions that we’ll find).
Let n be the sample size, and let X1,...,Xn be independent identically distributed Weibull random variables with the parameters α and β, both positive. To find the MLE of α and β, we first define the likelihood function: lik(α,β) = f (x1,...,xn | α,β) = f (x1 | α,β)···f (xn | α,β)
Substituting the definition of the density function of X yields
lik(
.
It’s easier to work with the natural logarithm of the given expression, so we
define
l(α,β) = ln(lik(
Partial derivative of l with respect to α is

Setting ) = 0, we have that

Therefore, the MLE for α is
(1)
where βˆ is the MLE for β, which is yet to be found. Partial derivative of l with respect to β is

Setting ∂β∂ l(α,β) = 0, and after a lot of cancelling and dreadful algebra, we can obtain the following nonlinear equation in terms of β:

So, the MLE for β should satisfy the following condition:
(2)
So, βˆ clearly has to be found numerically.
1. Generate Weibull distributed random numbers for given values of α and β (using some existing tool).
2. Obtain the histogram of generated random numbers; normalize it and compare with true expression of PDF (they should be very close to each other).
3. Write a (recursive or iterative) computer code to compute the estimate the values of α and β using expressions (1) and (2), respectively, for the data generated in step 1 (above). Note: For large enough data, the estimated and true values must be very close to each other.