Mathematics 376 – Probability and Statistics II Solutions – Problem Set 6 March 26, 2004

A)

1) First we substitute μ = \overline{Y} into the equation \frac{∂ ln(L)}{∂σ^{2}} = 0 and add \frac{n}{2σ^{2}} to both sides to get

\frac{n}{2σ^{2}} = \frac{∑_{i=1}^{n}(y_{i} − \overline{Y})^{2}/2}{(σ^{2})^{2}}

Multiplying by (σ^{2})^{2} and dividing by n we get the maximum likelihood estimator:

σ^{2} = σ̂^{2}_{ML} = \frac{1}{n} ∑_{i=1}^{n}(y_{i} − \overline{Y})^{2}

Note that this is the biased estimator for the variance. Our usual formula

S^{2} = \frac{1}{n − 1} ∑_{i=1}^{n}(y_{i} − \overline{Y})^{2},

gives an unbiased estimator, so

E(σ̂^{2}_{ML}) = \frac{n − 1}{n}σ^{2}

2) For the second derivative test for functions of two variables, we need all the second order partial derivatives of ln(L):

\begin{aligned}\frac{∂^{2} ln(L)}{∂μ^{2}} & = \frac{−n}{σ^{2}} \\ \frac{∂^{2} ln(L)}{∂σ^{2}∂μ} & = \frac{− ∑_{i=1}^{n}(y_{i} − μ)}{(σ^{2})^{2}} \\ \frac{∂^{2} ln(L)}{∂(σ^{2})^{2}} & = \frac{n}{2(σ^{2})^{2}} − \frac{∑_{i=1}^{n}(y_{i} − μ)^{2}}{(σ^{2})^{3}} \end{aligned}

Now we substitute the coordinates of the critical point: μ = \overline{Y}, and σ^{2} = \frac{1}{n} ∑_{i=1}^{n}(y_{i} − \overline{Y})^{2} to obtain:

\begin{aligned}\frac{∂^{2} ln(L)}{∂μ^{2}} & = \frac{−n^{2}}{∑(y_{i} − \overline{Y})^{2}} \\ \frac{∂^{2} ln(L)}{∂σ^{2}∂μ} & = 0 \\ \frac{∂^{2} ln(L)}{∂(σ^{2})^{2}} & = \frac{−n^{3}}{2(∑(y_{i} − \overline{Y})^{2})^{2}} \end{aligned}

Then, at the critical point, the Hessian determinant

H = \frac{∂^{2} ln(L)}{∂μ^{2}} \frac{∂^{2} ln(L)}{∂(σ^{2})^{2}} − (\frac{∂^{2} ln(L)}{∂σ^{2}∂μ})^{2} = \frac{n^{5}}{2(∑(y_{i} − \overline{Y})^{2})^{3}} > 0

and \frac{∂^{2} ln(L)}{∂μ^{2}} = \frac{−n^{2}}{∑(y_{i}−\overline{Y})^{2}} < 0 so the critical point is a (local) maximum point of ln(L).

9.78/First, we consider the likelihood function for the two sets of sample values, thinking of μ_{1} and μ_{2} as known: Because the X and Y are independent,

L = L_{X}L_{Y} = (\frac{1}{\sqrt{2πσ^{2}}} )^{m+n} e^{−(∑_{i=1}^{m}(x_{i}−μ_{1})^{2}+∑_{j=1}^{n}(y_{j}−μ_{2})^{2})\frac{1}{2σ^{2}}}

Hence

ln(L) = \frac{−(m + n)}{2} ln(2πσ^{2}) − \frac{(∑(x_{i} − μ_{1})^{2} + ∑(y_{j} − μ_{2})^{2})}{2σ^{2}}

To maximize, we differentiate with respect to σ^{2} and set equal to zero:

0 = \frac{d}{dσ^{2}} ln(L) = \frac{−(m + n)}{2σ^{2}} + \frac{(∑(x_{i} − μ_{1})^{2} + ∑(y_{j} − μ_{2})^{2})}{2(σ^{2})^{2}}

Hence

σ^{2} = \frac{1}{m + n} (∑(x_{i} − μ_{1})^{2} + ∑(y_{j} − μ_{2})^{2})

gives the maximum likelihood estimator under the assumption that μ_{1} and μ_{2} are known. Since they are not actually known for the purposes of this problem, we use the invariance property to replace them by \overline{X} and \overline{Y} (the maximum likelihood estimators of μ_{1} and μ_{2}):

σ̂^{2}_{ML} = \frac{1}{m + n} (∑_{i=1}^{m}(x_{i} − \overline{X})^{2} + ∑_{j=1}^{n}(y_{j} − \overline{Y})^{2})

9.79/Each of the three proportions is a binomial random variable (when we lump together the other two). Hence the likelihood function will have the form

L(y_{i}|p_{i}) = p_{i}^{y_{i}}(1 − p_{i})^{n−y_{i}} ⇒ ln(L) = y_{i} ln(p_{i}) + (n − y_{i}) ln(1 − p_{i})

Taking the derivative with respect to p_{i}, setting equal to zero and taking one term to the other side of the equation gives

\frac{y_{i}}{p_{i}} = \frac{n − y_{i}}{1 − p_{i}}

so

(1 − p_{i})y_{i} = p_{i}(n − y_{i}) ⇒ p_{i} = \frac{y_{i}}{n}

gives the maximum likelihood estimator:

p̂_{i}_{ML} = \frac{y_{i}}{n}

Now with the data given in the problem,

p̂_{1} − p̂_{2} = .30 − .38 ± 2σ_{_{p_{1}}−_{p_{2}}}

The standard error comes from taking the square root of

V (p̂_{1} − p̂_{2}) = V (p̂_{1}) + V (p̂_{2}) − 2Cov(p̂_{1}, p̂_{2})

But, here since both samples are being drawn from the same pool of 100 people, they are not independent and the covariance term must be computed using the result of Theorem 5.13 in the text:

= \frac{(.30)(.70)}{100} + \frac{(.32)(.68)}{100} − 2 (\frac{−(.30)(.38)}{100})

Then taking the square root of this we get:

σ_{_{p_{1}}−_{p_{2}}} = \frac{1}{10}\sqrt{.21 + .2356 + .228} = .08207

and the 2-standard deviation bound on the error of estimation is

−.08 ± .1641

9.92/a) We have, integrating by parts:

E(Y) = ∫_{θ}^{∞} ye^{−(y−θ)} dθ
= −ye^{−(y−θ)}|_{θ}^{∞} + ∫_{θ}^{∞} e^{−(y−θ)} dθ
= θ + 1

The method of moments estimator is obtained by setting this equal to the sample mean \overline{Y} and solving for θ:

θ̂_{MM} = \overline{Y} − 1

b) If all y_{i} ≥ θ, the likelihood function is

L(y_{1}, …, y_{n}) = e^{−(y_{1}−θ)} ⋯ e^{−(y_{n}−θ)}
= e^{− ∑_{i=1}^{n} y_{i}+nθ}
⇒ ln(L) = − ∑_{i=1}^{n} y_{i} + nθ

(L is zero otherwise, and ln(L) is undefined if any y_{i} < θ.) Here \frac{d}{dθ} ln(L) = n > 0. So, there are no critical points. However, ln(L) is increasing as a function of θ, so to maximize it, we want to make θ as large as possible subject to the constraint that y_{i} ≥ θ for all i. This says the maximum likelihood estimator for θ will be

θ̂_{ML} = Y_{(1)},

the sample minimum.

10.4/The possible outcomes of the test, the decision in each case and the associated probabilities in case H_{0} is true (p = .05) are given in the following table (Y means an error was found, N means no error was found):

Sheet 1 Sheet 2 Sheet3 Decision  P
N  N  Reject  (.95)^{2}
Y  N  Y  Accept
Y  N  N  Reject (.05)(.95)^{2}
N  Y  Y  Accept
N  Y  N  Reject (.05)(.95)^{2}
Y  Y  Y  Accept
Y  Y  N  Reject (.05)^{2}(.95)

a) A Type I error occurs if we reject the null hypothesis, but it is actually true. For this test, that happens if the last sheet we inspect has no error (but H_{0} is true). b) The level of the test is a truly horrendous

α = (.95)^{2} + 2(.05)(.95)^{2} + (.05)^{2}(.95) = .99513

(i.e. the chance of making a Type I error is over 99.5%). We sum the terms above since these are mutually exclusive cases. c) A Type II error occurs if we do not reject the null hypothesis but it is false. If the actual proportion of sheets with no errors is p, this occurs if we are in the “Accept” cases. d) The probability of a Type II error is

β = 2p^{2}(1 − p) + p^{3}

10.7/ a) Let S_{ave} be the average savings. Then H_{0} is S_{ave} = 900, and H_{a} is S_{ave} ≠ 900. That is, we use a two-tailed Z-test with test statistic

Z = \frac{S_{ave} − 900}{σ/\sqrt{35}}

b) For a level α = .01 test, we use the the rejection region:

RR = {z : |z| > z_{.005}} = {z : |z| > 2.575}

(value of z_{.005} from the standard normal table). c) The observed value of the test statistic (using the approximation σ = 50) is

z = \frac{885 − 900}{50/\sqrt{35}} = −1.77

d) Since this is not in the rejection region, we cannot reject the null hypothesis in this case. That is, there is not sufficient evidence at the α = .01 level to say that the average savings differs from $900.

10.9/We use a lower-tail Z-test with α = .05 so the test statistic is

z = \frac{v − 130}{2.1/\sqrt{40}}

and

RR = {z : z < −z_{.05}} = {z : z < −1.645}

With the given data:

z = \frac{128.6 − 130}{2/\sqrt{40}} = −4.2164

This is far into the rejection region, so there is sufficient evidence to say that the voltage is < 130 at this level of significance.

10.23/Changing the notation of the problem a bit for suggestiveness, let p_{R} be the proportion of Republicans strongly favoring the death penalty, and let p_{D} be the corresponding proportion of Democrats. We test the null hypothesis H_{0} : p_{R} = p_{D} against H_{a} : p_{R} > p_{D} with an upper-tail Z-test. The test statistic must be set up under the assumption that H_{0} is true. If p_{R} = p_{D}, the best estimate we have for that proportion from the given data is p = \frac{46+34}{200+200} = \frac{80}{400} = .2. For a test with α = .05, we want

z = \frac{p̂_{R} − p̂_{D} − 0}{\sqrt{\frac{(.2)(.8)}{200} + \frac{(.2)(.8)}{200}}} = \frac{p̂_{R} − p̂_{D}}{.04}

and the rejection region will be

RR = {z : z > z_{.05}} = {z : z > 1.645}

The value of the test statistic is

z = \frac{.23 − .17}{.04} = 1.5

This is not in the rejection region, so there is not sufficient evidence to support the conclusion that p_{R} > p_{D} at the α = .05 level.