\magnification=\magstep1
\def\Xbar{\overline{X}}
\def\Ybar{\overline{Y}}
\centerline{Mathematics 376 -- Probability and Statistics II}
\centerline{Solutions -- Problem Set 6}
\centerline{March 26, 2004}
\bigskip
\noindent
A) 
\item{1)} First we substitute $\mu = \Ybar$ into the equation
${\partial \ln(L)\over \partial\sigma^2} = 0$ and add ${n\over 2\sigma^2}$
to both sides to get
$${n\over 2\sigma^2} = {\sum_{i=1}^n(y_i - \Ybar)^2/2\over (\sigma^2)^2}$$
Multiplying by $(\sigma^2)^2$ and dividing by $n$ we get the maximum
likelihood estimator:
$$\sigma^2 = \widehat{\sigma^2}_{ML} = {1\over n}\sum_{i=1}^n(y_i - \Ybar)^2$$
Note that this is the {\it biased estimator} for the variance.  Our
usual formula 
$$S^2 = {1\over n-1}\sum_{i=1}^n(y_i - \Ybar)^2,$$
gives an unbiased estimator, so
$$E(\widehat{\sigma^2}_{ML}) = {n-1\over n}\sigma^2$$
\bigskip
\item{2)}  For the second derivative test for functions of two
variables, we need all the second order partial derivatives
of $\ln(L)$:
$$\eqalign{
{\partial^2 \ln(L)\over \partial \mu^2} &= {-n\over \sigma^2}\cr
{\partial^2 \ln(L)\over \partial \sigma^2 \partial \mu} &= {-\sum_{i=1}^n (y_i - \mu)\over (\sigma^2)^2}\cr
{\partial^2 \ln(L)\over \partial (\sigma^2)^2} &= {n\over 2(\sigma^2)^2} 
- {\sum_{i=1}^n (y_i - \mu)^2\over (\sigma^2)^3}\cr}$$
Now we substitute the coordinates of the critical point:
$\mu = \Ybar$, and $\sigma^2 = {1\over n}\sum_{i=1}^n (y_i - \Ybar)^2$ to obtain:
$$\eqalign{
{\partial^2 \ln(L)\over \partial \mu^2} &= {-n^2\over \sum (y_i - \Ybar)^2}\cr
{\partial^2 \ln(L)\over \partial \sigma^2 \partial \mu} &= 0\cr
{\partial^2 \ln(L)\over \partial (\sigma^2)^2} &= {-n^3\over 2(\sum (y_i - \Ybar)^2)^2}\cr}$$
Then, at the critical point, the Hessian determinant
$$H ={\partial^2 \ln(L)\over \partial \mu^2} {\partial^2 \ln(L)\over \partial (\sigma^2)^2}
-\left({\partial^2 \ln(L)\over \partial \sigma^2 \partial \mu}\right)^2 = 
{n^5 \over 2 (\sum (y_i - \Ybar)^2)^3} > 0$$
and ${\partial^2 \ln(L)\over \partial \mu^2} = {-n^2\over \sum (y_i - \Ybar)^2} < 0$
so the critical point is a (local) maximum point of $\ln(L)$.
\bigskip
\noindent
9.78/First, we consider the likelihood function for the two sets of sample
values, thinking of $\mu_1$ and $\mu_2$ as known: Because the $X$ and $Y$
are independent,
$$L = L_XL_Y = \left({1\over \sqrt{2\pi\sigma^2}}\right)^{m+n} 
e^{-\left(\sum_{i=1}^m (x_i - \mu_1)^2+\sum_{j=1}^n (y_j - \mu_2)^2\right){1\over 2\sigma^2}}$$
Hence 
$$\ln(L) = {-(m+n)\over 2}\ln(2\pi\sigma^2) - 
{\left(\sum (x_i - \mu_1)^2+\sum(y_j - \mu_2)^2\right) \over 2\sigma^2}$$
To maximize, we differentiate with respect to $\sigma^2$ and set equal
to zero:
$$0 = {d\over d\sigma^2}\ln(L) = {-(m+n)\over 2\sigma^2} + 
{\left(\sum (x_i - \mu_1)^2+\sum(y_j - \mu_2)^2\right) \over 2(\sigma^2)^2}$$
Hence
$$\sigma^2 = {1\over m+n} \left(\sum (x_i - \mu_1)^2+\sum(y_j - \mu_2)^2\right)$$
gives the maximum likelihood estimator under the assumption that
$\mu_1$ and $\mu_2$ are known.  Since they are not actually known
for the purposes of this problem, we use the {\it invariance property}
to replace them by $\Xbar$ and $\Ybar$ (the maximum likelihood estimators
of $\mu_1$ and $\mu_2$):
$$\widehat{\sigma^2}_{ML} = {1\over m+n} \left(\sum_{i=1}^m (x_i - \Xbar)^2+\sum_{j=1}^n(y_j - \Ybar)^2\right)$$
\bigskip
\noindent
9.79/Each of the three proportions is a binomial random variable
(when we lump together the other two).  Hence the likelihood
function will have the form
$$L(y_i|p_i) = p_i^{y_i}(1 - p_i)^{n-y_i} \Rightarrow
\ln(L) = y_i \ln(p_i) + (n-y_i)\ln(1 - p_i)$$
Taking the derivative with respect to $p_i$, setting
equal to zero and taking one term to the other 
side of the equation gives
$${y_i \over p_i} = {n - y_i \over 1 - p_i}$$
so
$$(1 - p_i)y_i = p_i(n-y_i) \Rightarrow p_i = {y_i\over n}$$
gives the maximum likelihood estimator:
$$\widehat{p_i}_{ML} = {y_i\over n}$$
Now with the data given in the problem,
$$\widehat{p_1} - \widehat{p_2} = .30 - .38 \pm 2\sigma_{\widehat{p_1}-\widehat{p_2}}$$
The standard error comes from taking the square root of 
$$V(\widehat{p_1}-\widehat{p_2}) = V(\widehat{p_1}) 
+ V(\widehat{p_2}) - 2{\rm Cov}(\widehat{p_1},\widehat{p_2})$$
But, here {\it since both samples are being drawn from the 
same pool of 100 people}, they are {\it not} independent
and the covariance term must be computed using the 
result of Theorem 5.13 in the text: 
$$= {(.30)(.70)\over 100} + {(.32)(.68)\over 100} - 2\left({-(.30)(.38)\over 100}\right)$$
Then taking the square root of this we get:
$$\sigma_{\widehat{p_1}-\widehat{p_2}} = {1\over 10}\sqrt{.21+.2356+.228} = .08207$$
and the 2-standard deviation bound on the error of estimation is
$$-.08 \pm .1641$$
\bigskip
\noindent
9.92/a)  We have, integrating by parts:
$$\eqalign{
E(Y) &= \int_\theta^\infty y e^{-(y-\theta)}\ d\theta\cr
&= -ye^{-(y-\theta)}|_\theta^\infty + \int_\theta^\infty e^{-(y-\theta)}\ d\theta\cr
&=\theta + 1\cr}$$
The method of moments estimator is obtained by setting this equal
to the sample mean $\Ybar$ and solving for $\theta$:
$$\widehat{\theta}_{MM} = \Ybar - 1$$
\bigskip
\noindent
b) If all $y_i \ge \theta$, the likelihood function is 
$$\eqalign{
L(y_1,\ldots,y_n) &= e^{-(y_1 -\theta)} \cdots e^{-(y_n-\theta)}\cr
&=e^{-\sum_{i=1}^n y_i + n\theta}\cr
\Rightarrow\ \ln(L) &= -\sum_{i=1}^n y_i + n\theta\cr}$$
($L$ is zero otherwise, and $\ln(L)$ is undefined
if any $y_i < \theta$.)
Here ${d\over d\theta}\ln(L) = n > 0$.  So, there are 
no critical points.  However, $\ln(L)$ is increasing
as a function of $\theta$, so to maximize it,
we want to make $\theta$ as large as possible
subject to the constraint that $y_i \ge \theta$
for all $i$.  This says the maximum likelihood
estimator for $\theta$ will be
$$\widehat{\theta}_{ML} = Y_{(1)},$$
the {\it sample minimum}.
\bigskip
\noindent
10.4/The possible outcomes of the test, the decision in 
each case and the associated probabilities in case
$H_0$ is true ($p = .05$) are given in the following table
($Y$ means an error was found, $N$ means no error was
found):
$$\matrix{{\rm Sheet\ 1} & {\rm Sheet\ 2} & {\rm Sheet 3} & {\rm Decision} & P\cr
                N        &         N      &               & {\rm Reject}   & (.95)^2\cr
                Y        &         N      &      Y        & {\rm Accept}   & \cr
                Y        &         N      &      N        & {\rm Reject}   & (.05)(.95)^2\cr
                N        &         Y      &      Y        & {\rm Accept}   & \cr
                N        &         Y      &      N        & {\rm Reject}   & (.05)(.95)^2\cr
                Y        &         Y      &      Y        & {\rm Accept}   & \cr
                Y        &         Y      &      N        & {\rm Reject}   & (.05)^2(.95)\cr}$$
a)  A Type I error occurs if we reject the null hypothesis, but it is
actually true.  For this test, that happens if the last sheet we
inspect has no error (but $H_0$ is true).  b)  The level of the 
test is a truly horrendous
$$\alpha = (.95)^2 + 2(.05)(.95)^2 + (.05)^2(.95) = .99513$$
(i.e. the chance of making a Type I error is over $99.5\%$). 
We sum the terms above since these are mutually exclusive
cases. c)
A Type II error occurs if we do not reject the null hypothesis
but it is false.  If the actual proportion of sheets
with no errors is $p$, this occurs if we are in the 
``Accept'' cases.  d)  The probability of a Type II
error is 
$$\beta = 2p^2(1-p) + p^3$$
\bigskip
\noindent
10.7/ a)  Let $S_{ave}$ be the average savings.  Then
$H_0$ is $S_{ave} = 900$, and $H_a$ is $S_{ave} \ne 900$.
That is, we use a two-tailed Z-test with test statistic
$$Z = {S_{ave} - 900\over \sigma/\sqrt{35}}$$
b) For a level $\alpha = .01$ test, we use the
the rejection region:
$$RR = \{z : |z| > z_{.005}\} = \{z : |z| > 2.575\}$$
(value of $z_{.005}$ from the standard normal table).
c)  The observed value of the test statistic
(using the approximation $\sigma = 50$) is 
$$z = {885 - 900 \over 50/\sqrt{35}} = -1.77$$
d)  Since this is {\it not} in the rejection 
region, we cannot reject the null hypothesis
in this case.  That is, there is not sufficient
evidence at the $\alpha = .01$ level to say that
the average savings differs from \$900.
\bigskip
\noindent
10.9/We use a lower-tail $Z$-test with $\alpha=.05$
so the test statistic is 
$$z = {v - 130\over 2.1/\sqrt{40}}$$
and 
$$RR = \{ z : z < -z_{.05}\} = \{ z : z < -1.645\}$$
With the given data:
$$z = {128.6 - 130\over 2/\sqrt{40}} = -4.2164$$
This is far into the rejection region, so there
is sufficient evidence to say that the voltage
is $< 130$ at this level of significance.
\bigskip
\noindent
10.23/Changing the notation of the problem a bit
for suggestiveness,
let $p_R$ be the proportion of Republicans
strongly favoring the death penalty, and let 
$p_D$ be the corresponding
proportion of Democrats. We test the null hypothesis
$H_0 : p_R = p_D$ against $H_a : p_R > p_D$
with an upper-tail $Z$-test.
The test statistic must be set up under the 
assumption that $H_0$ is {\it true}.  If $p_R = p_D$,
the best estimate we have for that proportion
from the given data is $p = {46 + 34\over 200 + 200} = {80\over 400} = .2$.
For a test with $\alpha = .05$, we want
$$z = {\widehat{p_R} - \widehat{p_D} - 0
\over 
\sqrt{ {(.2)(.8)\over 200} + {(.2)(.8)\over 200} }}
= {\widehat{p_R} - \widehat{p_D} 
\over .04}$$
and the rejection region will be
$$RR = \{z : z > z_{.05}\} = \{z : z > 1.645\}$$
The value of the test statistic is
$$z = {.23 - .17\over .04} = 1.5$$
This is not in the rejection region, so there {\it is
not sufficient evidence} to support the conclusion
that $p_R > p_D$ at the $\alpha = .05$ level. 
       

\bye

