\chapter{Hypothesis Testing and Statistical Inference}

\specialsection{Introduction}
\index{statistics!inferential|(}
\index{statistical inference|(}
For the purposes of this introductory discussion, imagine
first that you are a scientist.  You have collected some data
from an experiment and you believe that it shows that a certain
pattern exists in the real world situation you are studying.  For
instance, you might think you have shown that a particular
non-till farming method reduces soil erosion as compared
with conventional farming methods. \emph{How
do you demonstrate that the measurements you have made
provide evidence for claiming that that pattern actually
exists?  Your arguments should be in a form that can be 
checked and evaluated by other scientists, and your goal
is essentially to convince those others that your claims
are true, using some accepted mathematical tools.}

Similarly, now suppose you are a public opinion pollster, and you
think you have found some interesting pattern in the answers to a
question on a survey you have designed.
For instance, this pattern could be
that a particular political candidate seems to be leading in
the run-up to an election.\footnote{As we said in Chapter 11, this 
might be a situation you are 
sick of hearing about.  But it is an important application with significant
practical implications.}  Again, \emph{how do you demonstrate that 
this pattern is there, in 
a way that is verifiable and that will convince others of the correctness
of your claims?}

In both cases, you would need to be able to show convincingly that whatever
you saw in the data you collected was not simply the result of some naturally 
occurring variation or some random glitch in the measurement process.  In both 
cases, the issue is whether the same pattern observed in a {\it sample} (that is, 
\index{sample}
the measurements made) is also true of the whole population from which the samples 
were drawn (that is, all possible instances of using the farming method, or all the 
people in the population from which the respondents of the questionnaire were chosen).
\vskip 10pt
\noindent
{\it Key Idea:}  The typical methods used by most scientists 
would include performing a statistical analysis on
the data aimed at showing that {\it the observed results would be extremely
rare if the claimed pattern did not actually hold true}.
\vskip 10pt
\noindent
If that can be done successfully, the fact that
the pattern in the measurements made was observed can be due to one of
two possible reasons:  Either it is 
\begin{itemize}
\item due to the fact
that the pattern is {\it truly there} in the whole population,  or else, 
\item  due to some truly
``bad luck'' in how your experiment turned out.
\end{itemize}
\noindent
In the second case, you ``hit the jackpot,'' but in a
negative way.  You observed a very rare event that does not agree with the
true state of things. 

This probably sounds rather
convoluted at first, but it is the logic behind the {\it hypothesis tests}
we will discuss, and it is the most important idea behind this part
of our course.

\specialsection{The Logic of Hypothesis Tests}
\index{hypothesis testing|(}
\index{statistics!hypothesis testing|(}
A fundamental assumption underlies all of the methods we will describe
in this chapter, namely that whatever measurements we are analyzing 
come from a \emph{random sample} of the population of all possible 
\index{sample}
measurements of the type we are considering.  If that condition is
violated, these methods can yield misleading results.  However, 
techniques used in the design of experiments to ensure reasonable
compliance with the requirement of randomness are somewhat beyond the 
scope of our discussion and we will not concentrate on that side
of the equation. 
 
\subsection*{Null and Alternative Hypotheses}
\index{null hypothesis|(}
\index{alternative hypothesis|(}
\index{hypothesis!null|(}
\index{hypothesis!alternative|(}
Assuming we have collected data that yields something like 
a truly random sample from the population of possible measurements,
the way the testing process is described typically involves two
{\it competing explanations} for the results of the experiment:
\begin{itemize}
\item A {\it ``null hypothesis''} (often denoted $H_0$ if
we need a symbolic abbreviation) that says essentially ``there's nothing
there -- the results were just the result of naturally occurring variation
or some random glitch in the measurement process.'' Of course, the
exact statement is usually more precise than this -- it usually involves
a more specific statement about the thing(s) you are measuring.
\item An {\it alternative hypothesis} (denoted $H_a$) that
is is some assertion that a claimed pattern is really there.  This is
also usually stated in more precise form(!)
\end{itemize}

The statistical null and alternative hypotheses
can be, and usually are, different from the scientific hypotheses that 
were made in designing the experiment that produced the data.  These statistical 
hypotheses are proposed explanations for the data that was observed.
The goal is to decide whether the weight of the evidence contained in
the measurements supports $H_a$, or whether that evidence could
reasonably be explained by $H_0$.  From the point of view of a researcher,
of course, it is typically the alternative hypothesis $H_a$ that
is the ``preferred'' alternative.  Not being able to rule out the
null hypothesis is usually taken as a negative outcome, since in that
case we are saying the results we saw could just be due to chance.

Here are two examples that should make the distinction between the
null and alternative hypotheses clearer.

\begin{example}  
\label{lightbulbs}
A random sample of $n = 50$ compact fluorescent
light bulbs was chosen from the production line of the manufacturer
and the mean weight of mercury per bulb
was measured to be $5.3$ mg, with an SD of $.5$ mg.
Assume the mercury amounts in all such bulbs are normally distributed.
Looking at the $5.3$, it might be tempting to say:  ``The average amount of
mercury in these bulbs is $> 5$ mg.''  But does the data support that?
Might the particular sample chosen have just contained especially 
mercury-heavy bulbs, not typical of the whole population?
In this case the null hypothesis could be $H_0:$ the actual population
average mercury level $\mu$ satisfies $\mu \le 5$ mg per bulb.  And the alternative hypothesis
could be: $H_a:$ the actual population average mercury level is $\mu > 5$ mg per bulb.
$\triangle$
\end{example}

\begin{example}
\label{starlinkcorn}
\index{StarLink corn}
{\it StarLink} (a registered trademark) corn is a genetically engineered
variety that was approved as a source of feed for animals, but never
approved for human consumption.  A study by the USDA shows that
99 out of 1100 samples of corn taken from US (human) corn-based
food products were contaminated by traces of StarLink corn.  At the same time
the corresponding agency of the Mexican government does a similar study and
finds that 100 out of 1200 samples of corn-based food products taken from Mexican manufacturers
contain traces of StarLink.  Note that the contaminated proportions are $\widehat{p_{US}} = 99/1100 = .09$,
while $\widehat{p_M} = 100/1200 \doteq .083$ is smaller.
Does the data support the conclusion that StarLink contamination of human food products
is different in the US than in Mexico?  Here we could have $H_0: p_{US} = p_{M}$ (the actual
proportions in the two countries are just the same and the difference observed in the
samples is due to chance).  The corresponding alternative hypothesis might be $H_a: p_{US} \ne p_M$. It
would also be possible to use
$H_a: p_{US} > p_M$ if the goal is to decide if the evidence shows that the proportion in the
US is actually greater.  $\triangle$
\end{example}

The conclusions from statistical hypothesis tests are usually phrased in very cautious language.
One usually says something like ``there is enough evidence to reject the null hypothesis''
rather than saying ``the alternative hypothesis is definitely true.''  Similarly a negative
result might be described by saying ``there is not enough evidence to reject the null hypothesis''
rather than saying ``the alternative hypothesis is definitely false.''  The point here, of
course, is that the results from one experiment, no matter how strong or weak, can only
provide limited evidence one way or the other.  In addition, the scientific hypotheses 
underlying the experiment or the statistical
forms $H_0$ and $H_a$ are always subject to revision if further evidence indicates that
previous thinking was incorrect.

\subsection*{Type I and Type II Errors, $\alpha$ and $\beta$}
\index{Type I error|(}
\index{Type II error|(}
\index{probability!Type I error ($\alpha$)|(}
\index{probability!Type II error ($\beta$)|(}
A statistical test aimed at choosing between the two hypotheses $H_0,H_a$ can
``go wrong'' in two different ways:
\begin{itemize}
\item \emph{Type I Error:}  We could reject $H_0$ (and implicitly assert the evidence
favors accepting $H_a$) when $H_0$ is actually true.
\item \emph{Type II Error:}  We could fail to reject $H_0$ when it is
actually false.
\end{itemize}
Both types of errors are of concern, but the Type I error is usually considered to be, if
anything, more serious.  That is because making an incorrect conclusion
when an apparent pattern is due only to chance variation can throw off subsequent
research, can lead to inappropriate recommendations for real-world action, and can have other
undesirable consequences.
Making a Type II error, on the other hand, means essentially that we {\it missed}
a pattern that is there (perhaps by being too cautious in assessing the data).
A common view would be that since there is always the chance of {\it catching} the pattern
with another experiment later, Type II errors are, in a sense, easier to correct.
If there were some urgency or time pressure involved in the real-world situation under study, a Type II
error could also lead to serious real-world consequences, though.  For instance, consider a study of
the effectiveness of a vaccine for an infectious 
disease (informally: $H_0: $ no benefit from the vaccine
and $H_a:$ there is a benefit).  If the testing was being done while an epidemic was in progress,
then a Type II error leading to a decision not to use the vaccine
might lead to loss of lives that could have been saved by using the vaccine.
In real life, in fact, people are often willing to try unproven medical treatments 
in situations where they have nothing to lose, or in which any chance of a positive outcome, 
even a small one, outweighs other risks.

In any case, it is important to realize that most statistical tests are set up 
{\it to make the chance of a Type I error small}.  This is exactly the 
{\it Key Idea} stated at the beginning of this chapter, restated in a more precise form.
The chance of making a Type I error is usually denoted by the Greek letter 
$\alpha$ (``alpha''), and typical values for $\alpha$ used in the design of 
statistical hypothesis tests are $.05$, or $.01$, or perhaps even smaller values.  
Saying $\alpha = .05$, for instance, says that we want to set up
our test so that if $H_0$ is actually true, then roughly $95$ times out of $100$  the results
of the test will correctly indicate that $H_0$ {\it should not} be rejected.  Or equivalently,
Type I errors would happen roughly only $5$ out of $100$ times when $H_0$ is true.
In other words, as we said before, {\it results indicating that we should reject $H_0$ would be extremely rare if the claimed pattern did not actually hold true 
(that is if $H_0$ is actually true) }.

The chance of making a Type II error is denoted by another Greek letter, $\beta$ (``beta'').
In a sense, the real quantity of interest here is $1 - \beta$, the chance of making
a correct conclusion and rejecting $H_0$ when $H_0$ is false.  This is called the
{\it power} of the test, and ideally we would like the power to be as close to 1
\index{power (of a statistical test of hypotheses)}
as possible.   But it is important to realize that the power, and equivalently,
the value of $\beta$, is usually somewhat {\it harder to control} than $\alpha$ 
because they will typically depend on characteristics of the population from which the
measurements are being taken that may not be known exactly.  For instance, if
$H_0$ is the hypothesis that a population mean is equal to some particular number $\mu_0$,
then $H_0$ being false means the population mean is something different from $\mu_0$.
The value of $\beta$ will usually depend on the exact value of that population mean.
Thus, the power of the test is actually {\it a function of $\mu = $ true
population mean}, not just a constant.
Statisticians have developed techniques for studying $\beta$ and the power of tests as
well, but they are somewhat beyond the scope of our treatment of this subject.  
Without going into the details, we can just say that in order to obtain
a test with a given desired power, we would typically have to be able
to choose a sample size $n$ that was sufficiently large.  

Since making 
measurements in carrying out experiments will usually incur real-world costs 
in one fashion or another, this might not always be feasible.  For example, a polling 
organization might not have the time, the manpower, or the access to contact information
to carry out a phone survey of a random sample of $n = 5000$ likely voters 
within a few days before an election in order
in a test of voter preferences, even though having
$n$ that large might be necessary to to achieve a small $\beta$ value.


\subsection*{Test Statistics, Rejection Regions}
Here is the overall plan for a typical statistical test:
\begin{itemize}
\item Using assumptions about the distribution
of the possible measurement values from the population, some
\index{test statistic}
\index{rejection region}
{\it test statistic} that has a known probability distribution under the
assumption that the null hypothesis $H_0$ is {\it true} is identified.
\item Some desired Type I error probability $\alpha$ is selected.
(Often $\alpha = .05$ is used as a standard choice; smaller values might be
used too.  Values $\alpha > .05$ would almost never be used in practice since
there is something of a consensus that Type I error probabilities that large
are unacceptable.)
\item  Using that probability distribution, a {\it rejection region}
is identified.  This is a range $R$ of possible values of the test statistic with the
property that, under the assumption $H_0$ is true, the chance that the test statistic
lies in $R$ is $\alpha$.
\item  Then starting from the observed measurements, we compute
the value of the test statistic, determine whether it lies in the
rejection region or not.  
\item If, so we say ``there is evidence indicating $H_0$ should
be rejected'' or something similar; if not we say ``there is not enough evidence to
reject $H_0$.''
\end{itemize}

This probably seems somewhat abstract without a specific example to think
about.  Hence we will proceed immediately to first example tests, the ``$z$-tests
for means and proportions.''
\index{null hypothesis|)}
\index{alternative hypothesis|)}
\index{hypothesis!null|)}
\index{hypothesis!alternative|)}
\index{Type I error|)}
\index{Type II error|)}
\index{probability!Type I error ($\alpha$)|)}
\index{probability!Type II error ($\beta$)|)}

\specialsection{Tests for Means and Proportions}
\subsection*{A $t$-test for a mean (``large sample case'')}
\index{hypothesis testing!large sample test on a mean}
Suppose we have made a relatively large number $n \ge 30$ of numerical measurements
of a particular characteristic from randomly chosen individuals in some
fixed population (think something like lengths of individual adult fish from
some particular species).  Call those measurements $Y_1, \ldots, Y_n$.  Then
it is reasonable to expect that the sample mean
$$\overline{Y} = {Y_1 + \cdots + Y_n \over n}$$
should give a good estimate of the population average $\mu$. 
\index{estimator}
\index{Central Limit Theorem}
From the Central
Limit Theorem, the intuition gained in our Chapter Projects, and the discussion from
Chapter 11, recall that we expect that the values of $\overline{Y}$ from different
samples should be normally distributed, and that the conversion to the
standard normal should go by this $z$-score formula:
\index{z@$z$-scores}
$$ {\overline{Y} - \mu\over \sigma/\sqrt{n}},$$
where $\sigma$ is the {\it population} SD.  Of course we don't know the
exact value of $\sigma$ in practice, so we would usually estimate that using
$S = $ the sample SD.  Moreover, we don't know the value of $\mu$ either.

Let's consider the following situation.  Suppose we have a particular ``candidate''
value $\mu = \mu_0$ in mind as part of a null hypothesis-- $H_0: \mu = \mu_0$.  Moreover,
suppose the data seems to indicate, say, that $\mu > \mu_0$.  That will be the
alternative hypothesis $H_a$.
\index{null hypothesis}
\index{alternative hypothesis}
\index{hypothesis!null}
\index{hypothesis!alternative}

Under the assumption that $H_0$ is true, we have seen in Chapter 11 that the test statistic
$$T = {\overline{Y} - \mu_0 \over S/\sqrt{n}}$$
\index{test statistic}
\index{t@$t$ distribution}
\index{percentage points of a distribution}
\index{t@$t$ table}
has a $t(n-1)$-distribution.  For common values of $\alpha$ the percentage
points of the $t$-distributions from Figure~\ref{fig:PctPtsT} show
how to set up a rejection region.  See Example~\ref{Ztest} below for an
example.    For $n > 30$, though, the 
$t(n-1)$ distribution is very close to the standard normal.  So
we can use the standard normal table rather than
the $t$-table to find the associated percentage point.  For instance,
from the standard normal table, the chance that
$T > 1.65$ would be about $.05$.  This would gives the rejection region for a test with
$\alpha = .05$ -- we reject $H_0$ whenever $T > 1.645$ and we do not reject it otherwise.

\begin{example}
\label{Ztest}
Now let's illustrate how this would be applied.  Suppose we had measured data
that looked like this ($n = 30$ measurements):
\begin{align*}
&5.2, 5.3, 5.9, 8.8, 8.9, 7.1, 5.9, 6.3, 7.2, 5.0,\\
&5.3, 4.9, 3.2, 5.8, 6.2, 6.0, 7.2, 7.2, 4.2, 5.1,\\
&6.8, 7.8, 6.2, 5.2, 5.5, 4.3, 4.7, 4.7, 6.7, 6.3
\end{align*}
We have $\overline{Y} = 5.96$ and $S = 1.30$ (rounding to two decimal places).
For our example test, let's take:
\index{null hypothesis}
\index{alternative hypothesis}
\medskip
\begin{itemize}
\item $H_0: \mu = \mu_0 = 5.75$ and
\item $H_a: \mu > \mu_0 = 5.75$.
\end{itemize}
\noindent
Then, we compute:
$$T = {\overline{Y} - \mu_0\over S/\sqrt{n}} = {5.96 - 5.75\over 1.30/\sqrt{30}} = .8677$$
From the $t$-table, we find $t_{.05}(29) \doteq 1.6973$.  We still use the $t$-table here
because with $n = 30$ our test statistic has a $t(29)$-distribution.   
Since $.8677 < 1.697$, we {\it cannot reject} $H_0$ on the basis of the evidence
in this data.

The numbers above are a hypothetical dataset generated from a normal distribution with
true mean $\mu = 6.4$ and true $\sigma = 1.2$(!)  As you should have noticed in the Chapter 
Project on sampling, there is a lot of potential variability in sampling or measuring.  In this
case, the sample mean $\overline{Y}$ is especially small relative to the actual $\mu$,
and the sample SD $S$ is larger than the actual $\sigma$.  Both of these contributed
to a test statistic $T$ that was ``smaller than expected.''
This is a case where we are making a Type II error!  Although we won't discuss
how this would be derived, the power of this test with $\mu = 6.4$ would be about 
$1 - \beta = .83$--that is, the probability of a Type II error would be about 
$\beta = 1 - .83 = .17$.  The
above dataset ``beat the odds'' in a way, but that sort of thing would happen roughly
$17\%$ of the time. $\triangle$
\index{sample}
\index{Type II error}
\index{power (of a statistical test of hypotheses)}
\end{example}
 

There are a number of related forms of tests for means based on the form
of the alternative hypothesis.  For a test with $\alpha = .05$, for instance,
\index{rejection region}
\index{alternative hypothesis}
and $n > 30$, 
\begin{itemize}
\item if $H_a : \mu > \mu_0$ (an ``upper-tail'' test), then we would
reject $H_0$ if $T > 1.645$ 
\item if $H_a : \mu < \mu_0$ (a ``lower-tail'' test), then we would
reject $H_0$ if $Z < -1.645$
\item if $H_a : \mu \ne \mu_0$ (a ``two-tail'' test), then we would
reject $H_0$ if $Z < -1.96$ or $Z > 1.96$.
\end{itemize}

These rejection regions all come from the areas of regions under the standard
normal curve.  Note that the area between $0$ and $1.96$ is approximately $.4750$, so the area
in the upper tail from $1.96$ to $+\infty$ is about $.025$.  Similarly the
area in the lower tail from $-\infty$ to $-1.96$ is also about $.025$ by the
symmetry of the standard normal curve.  This means that the rejection region
for the two-tail test has total area about $.025 + .025 = .05$ and the
chance that $T$ lands in that region under the assumption that $H_0 : \mu = \mu_0$
is true is $\alpha = .05$ (approximately).

\subsection*{Hypothesis Tests and Confidence Intervals}
\index{confidence interval}
As you may suspect from this discussion, there is very close connection
between the rejection region for a two-tail hypothesis test and a confidence interval
for the mean as discussed in Chapter 11.  Recall that we have seen that in this large sample case,
the endpoints of the $95\%$ confidence interval for $\mu$ would be computed from the sample
mean $\overline{Y}$ and $S = $ the sample SD  by the formula
$$\mu = \overline{Y} \pm 1.96 \cdot {S\over \sqrt{n}}$$ 
If we add the assumption that $\mu = \mu_0$ from the null hypothesis, when you look at this formula, 
it is not too difficult to see that it is saying the following.   
The values of $\overline{Y}$ for which
we would {\it not} reject $H_0: \mu = \mu_0$ are exactly the $\overline{Y}$ such that
$$\mu_0 - 1.96 \cdot {S\over \sqrt{n}} < \overline{Y} < \mu_0 + 1.96 \cdot {S\over \sqrt{n}}$$
(since it is exactly those values of $\overline{Y}$ for which 
$T = {\overline{Y} - \mu_0\over S/\sqrt{n}}$
satisfies $-1.96 < Z < 1.96$).  

In other words, the confidence interval gives the range of ``believable'' values
for $\mu$ based on the the mean and SD of the sample.  The rejection 
region for the test is something like the {\it complement of (that is, the part of the 
number line outside)} the
interval computed using the confidence interval formulas, but
using $\mu_0$ from the null hypothesis rather than the sample mean from the data.  

If we only have $n \le 30$ measurements, then the actual 
properties of the $t$-distribution for $\nu = n-1$ degrees of freedom
must be taken into account.  We will see what to do in those ``small sample'' cases later.  

\subsection*{$Z$-tests for a Proportion}
\index{hypothesis testing!large sample test on a proportion}
Say we have asked a random sample of $n$ people from
a particular population a ``yes-or-no'' question and some number $Y$
of them answer ``yes.''  Suppose $P$ is the proportion of
the whole population who would answer ``yes'' if asked and we
want to estimate this proportion.
From the sample, as discussed in Chapter 11, we can estimate $P$ using $\widehat{P} = Y/n$.

The theoretical basis for the test in this case is the following
\index{null hypothesis}
mathematical result:  Under $H_0 : P = P_0$, 
\begin{enumerate}
\item[a)] when $n$ is relatively large 
(the cutoff value $n > 30$ is often used), {\it or more generally} 
\item[b)] when $nP_0$ and $n(1 - P_0)$ are both
relatively large (a typical ``rule of thumb'' is both are $\ge 5$), 
\end{enumerate}
then it can be shown\footnote{Technical note: This uses the same reasoning
based on the Central Limit Theorem that we used in Chapter 11
\index{Central Limit Theorem}
\index{normal approximation to binomial}
to derive the normal approximation
to a binomial distribution.} that the test statistic
$$Z = {\widehat{P} - P_0\over \sqrt{{P_0(1 - P_0)\over n}}}$$
has an approximately standard normal distribution.  

\index{alternative hypothesis}
\index{rejection region}
So we can set up hypothesis tests with $\alpha = .05$ using the
same rejection regions as above in the case of a large-sample
$t$-test for a mean:
\begin{itemize}
\item If $H_a : P > P_0$ (an ``upper-tail'' test), then we would
reject $H_0$ if $Z > 1.645$
\item If $H_a : P < P_0$ (a ``lower-tail'' test), then we would
reject $H_0$ if $Z < -1.645$
\item If $H_a : P \ne P_0$ (a ``two-tail'' test), then we would
reject $H_0$ if $Z < -1.96$ or $Z > 1.96$.
\end{itemize}

\begin{example}
\label{voters}
Supppose that $n = 1000$ likely voters are surveyed
and $Y = 550$ of them say they will vote for candidate Jones in the
next city council election.  With $\alpha =.05$, is there sufficient
evidence to say that Jones will win the coming election?  We take
$H_0 : P = P_0 = .5$ and $H_a : P > .5$.  (Note that this is one case
where the statistical null hypothesis is somewhat different than we
might expect -- Jones will only lose the election if $P < .5$, and
the boundary case $P = .5$ would be a ``dead heat'' in the election.)
In any case,
with the values of $n,Y$ as given, we have $n > 30$ and 
$nP_0 = 500 = n(1 - P_0)$, so both conditions a) and b) above are
satisfied and we can use this approach.
We compute
\index{test statistic}
$$Z = {.55 - .50\over \sqrt{(.5)(.5)\over 1000}} \doteq 3.16 > 1.645.$$
This is strong evidence to indicate that we should reject $H_0$.  Note
that any value $P_0 < .5$ would yield an even larger $Z$.  So taking $P_0 = .5$
in this case is justified since it is the boundary case between a
loss for Jones and a win for Jones.  $\triangle$
\end{example}

\subsection*{Large-Sample Tests for Differences of Means and Differences of 
Proportions}
\index{hypothesis testing!large sample test on a difference of means}
There are similar tests of hypotheses about the difference of 
two population means or two proportions.  The ones we will
discuss are valid {\it only in the large sample case}  (both
groups of samples of size at least $30$).
The analysis is based on the assumption that the samples are random and independent.  

Say we have measured some quantity in a random sample of size $n_1 \ge 30$
from a first group, and measured the same quantity in a random sample
of size $n_2 \ge 30$ from a second group.  (The two numbers $n_1$ and $n_2$ can be
different, but both should be in the large sample range.)  Call the two groups of 
measurements $Y_1,\ldots, Y_{n_1}$ and $X_1, \ldots, X_{n_2}$.  So 
we have two sample means $\overline{Y}$ and $\overline{X}$, as
well as sample SD's $S_1$ (from the $Y_i$) and $S_2$ (from the $X_j$).  

Say $\mu_1$ and $\mu_2$ are the population means of the two
groups.  To test $H_0 : \mu_1 = \mu_2$ versus $H_a : \mu_1 \ne \mu_2$,
we would use the test statistic:
\index{test statistic}
$$Z = {\overline{Y} - \overline{X}\over \sqrt{{S_1^2\over n_1} + {S_2^2 \over n_2}}}$$
since under the assumption $\mu_1 = \mu_2$, $Z$ has (approximately) a standard normal 
distribution.  The rejection region would be set up according to the 
value of $\alpha$ exactly as before.  

Similarly, for a test on a difference of proportions, say we have asked
\index{hypothesis testing!large sample test on a difference of proportions}
the same ``yes-or-no'' question to random samples from two different groups.  
Say $Y_1$ out of $n_1 \ge 30$ in the first group say ``yes'' while
$Y_2$ out of $n_2 \ge 30$ in the second group say ``yes.''  (There is
also a more general ``rule of thumb'' parallel to b) in the discussion
of the one-proportion test above that is sometimes used; we will not
discuss that, however.)   We take 
$\widehat{P_1} = Y_1/n_1$ estimating $P_1$ and $\widehat{P_2} = Y_2/n_2$
estimating $P_2$ (the two population proportions).  Then to test 
$H_0 : P_1 = P_2$ versus $H_a : P_1 \ne P_2$, we need some estimate
of the common value under the null hypothesis in order to construct
our test statistic.  The one that is commonly used is  
\index{estimator!pooled for proportion}
a ``pooled'' estimator of the common proportion (assuming the null hypothesis):
$$P_p = {Y_1 + Y_2\over n_1 + n_2}.$$ 
Then the test statistic is
$$Z = {\widehat{P_1} - \widehat{P_2} \over \sqrt{\widehat{P_p}(1 - \widehat{P_p})}\cdot\sqrt{{1\over n_1} + {1\over n_2}}}$$
and the rejection regions are as before.  

Note that the ``yes-or-no'' question can be anything with only those two
possible answers.  This applies much more generally than to only the 
polling situation.  For instance,

\begin{example}
\label{starlinkcorncont}
Refer back to Example~\ref{starlinkcorn} above.  Is there evidence to show that different 
proportions of corn products 
contain traces of StarLink in the US and in Mexico?  
\index{StarLink corn}
Note that the contaminated proportions are $\widehat{p_{US}} = 99/1100 = .09$,
while $\widehat{p_M} = 100/1200 \doteq .083$.  So
$P_p = {199\over 2300} = .087$.   We have
$$Z = {.09 - .83\over \sqrt{(.087)(.913)} \sqrt{{1 \over 1100} + {1\over 1200}}} \doteq .595$$
For the two-tail test with $\alpha = .05$, we would be looking for $Z > 1.96$.
So this is not strong enough evidence to reject $H_0$.  The proportions in the US
and in Mexico could be the same and the difference could be due to random variation.
$\triangle$
\end{example}

\subsection*{$T$-tests for a Mean (``small sample case'')}
\index{hypothesis testing!small sample test on a mean}
When the number of measurements $n$ is smaller than 
$30$, then the tests for the mean presented
above use the same test statistic, but 
must use different rejection regions coming from the appropriate 
``Student's $t$-distributions'' discussed in Chapter 11.  
Another important thing to note in this situation is that we cannot assume that
\index{Central Limit Theorem} 
the Central Limit Theorem has ``kicked in'' either, so \emph{it is necessary to 
assume, in addition, that the measurements are drawn from a population described
by a normal distribution} in this case.
\index{normal distribution}  

When $n < 30$, we use the same test-statistic as before:
\index{test statistic}
$$T = {\overline{Y} - \mu_0\over S/\sqrt{n}}.$$
But now the rejection regions are found from the $t$-table and will depend on $n$:
\index{rejection region}
\index{t@$t$ distribution}
\index{t@$t$ table}
\begin{itemize}
\item If $H_a : \mu > \mu_0$ (an ``upper-tail'' test), then we would
reject $H_0$ if $T > t_{.05}$ for $n - 1$ degrees of freedom
\item If $H_a : \mu < \mu_0$ (a ``lower-tail'' test), then we would
reject $H_0$ if $T < -t_{.05}$ for $n - 1$ degrees of freedom
\item If $H_a : \mu \ne \mu_0$ (a ``two-tail'' test), then we would
reject $H_0$ if $T < -t_{.025}$ or $T > t_{.025}$ for $n-1$ degrees of freedom.
\end{itemize}
The needed information to set up the two-tail rejection region
for $\alpha = .05$ is given in Figure \ref{fig:PctPtsT} in Chapter 11.  
For example, if we had $n = 10$ sample measurements, then the rejection region
for an upper-tail test with $\alpha = .05$ would be $T > 1.833$ from the $\nu = 9$ row
of the table.  (The $t$-distribution involved is the one that has $9$ degrees
of freedom -- note that the number of degrees of freedom is always
$n - 1$.)  The
corresponding two-tail test would have rejection region $T < -2.262$ or
$T > 2.262$.  Note that these are more restrictive than the corresponding
rejection regions for the large-sample test.  This is because the distribution
of $\overline{Y}$ and the test statistic $T$ {\it has even more variability when
$n$ is small}.  So the rejection region must be smaller than the corresponding
rejection region for a large-sample test with the same $\alpha = .05$.  

\begin{example}
\label{robins} A random sample of ten $100$ ${\rm km}^2$
plots in a forest are chosen and the number of robin nests in
each area is counted, yielding the following data:
$$310, 311, 412, 368, 447, 376, 303, 410, 365, 350$$
The sample mean is $\overline{Y} = 365.2$ and the sample SD is
$S = 48.417$.  Is there sufficient evidence to claim that the
average number of robin nests per $100$ ${\rm km}^2$ in this forest
is less than $380$?   We compute
$$T = {365.2 - 380\over 48.417/\sqrt{10}} \doteq -.967$$
Since $T > -1.833 = -t_{.05}$ for $9$ degrees of freedom, we
{\it do not} have sufficient evidence to reject $H_0: \mu = 380$
with $\alpha = .05$.  $\triangle$   
\end{example}

\specialsection{$p$-values and Hypothesis Tests}
\index{hypothesis testing!attained significance ($p$ value)|(}
When reporting the results of a statistical hypothesis test, it is
common to provide an additional (or sometimes an alternative) piece
of information called the {\it $p$-value} of the test.  Another
name is the {\it attained significance level}.  The reason for
that name is the fact that the Type I error probability $\alpha$
is often called the {\it significance level} of the test.  So,
for instance if we have decided on $\alpha = .05$ and the result
of the test is to reject $H_0$, we might say that the result
is ``significant at the $.05$-level.''  (This is the precise meaning
of statements like ``the results of the experiment were statistically
significant.'')  

Look back at Example~\ref{voters}.
There we had a test where the test statistic had the value
$Z = 3.17$.  We could say ``there is evidence to reject $H_0$
at the $\alpha = .05$ significance level, since $3.17 > z_{.05}$.  
But in fact since
$Z = 3.17$ is quite a bit larger than $z_{.05} = 1.645$, in
a way we are {\it understating the strength of our evidence}
if we stop there.  

\begin{definition}  The attained significance level $p$ is
{\it the smallest value of $\alpha$ for which
$H_0$ would be rejected with the observed value of the test statistic}.
\end{definition}

In Example~\ref{voters}, since
the area under the standard normal curve to the right of
$3.17$ is approximately $.00076$, using an upper-tail test, 
we would reject $H_0$ with this data for any $\alpha \ge .00076$.  We
say the $p$-value is $p = .00076$.

An alternative way to think about what the $p$-value of
a test is telling us is this:  the $p$-value is the
{\it chance of observing the given value of the test
statistic, or something ``more extreme,'' if $H_0$ is true}.  
So {\it very small} values of $p$
indicate {\it very strong evidence for rejecting $H_0$} according
to the Key Idea at the start of this Chapter.  Conversely,
larger values of $p$ (often any $p > .05$) is taken as indicating
the evidence is too weak to reject $H_0$.  Since there is
so much arbitrariness in the $\alpha = .05$ value,
nowadays, in fact, it is
actually common just to report the $p$-value of a test and
leave the interpretation of whether to accept or reject
$H_0$ up to the reader(!)

\begin{example}
Refer back to Example~\ref{lightbulbs}.   Since $n = 50$, we
use the large-sample formulas based on the standard normal curve.
\index{standard normal curve}
\index{standard normal table}
Our test statistic is
$$T = {5.3 - 5\over .5/\sqrt{50}} \doteq 4.24$$
For the upper-tail $z$-test of $H_0: \mu = 5$ versus
$H_a: \mu > 5$, we have a $p$-value $p = 1.1 \times 10^{-5} = .000011$.
This would be interpreted as very strong evidence to reject $H_0$.
Note that it is the sample size $n = 50$ that is making it come out this
way.  If we knew the population $SD$ was $.5$, under the null hypothesis
a single measurement of
$5.3$ would have a $z$-score of
$${5.3 - 5\over .5} = .6$$
This is not very large (less than one SD above $5$), so nothing
remarkable.  But the fact that we have the average
of $n = 50$ measurements being $5.3$ means that $T = .6 \times \sqrt{50} \doteq 4.24$.
$\triangle$
\end{example}

\subsection*{Statistical Significance and Practical Significance}
\bigskip
Some caution is certainly a good thing in applying the
methodology of hypothesis testing.  As we indicated before,
the significance level of a test $\alpha$ (or the $p$-value)
is based entirely on the probability of rejecting the 
null hypothesis when it is actually true.  So we are
really only taking the Type I errors into account.
For that reason, the statistical significance level
of a test is only an imperfect measure of how much
information we get from the test.  

Moreover, there are certainly times when a statistically
very significant test result (say one that gives a very
small $p$-value) has little {\it practical} significance. 
For example, a regression analysis on a large
data set\footnote{See Section 6 below for a more detailed
description of what this means.} might yield the result that the 
slope of the regression line was nonzero with
an attained significance level of $p = .0001$.
But the corresponding confidence interval of values
for the slope could contain only very small numbers
(e.g. $.01$ to $.03$).  There might not be much of a practical
difference between 
\begin{itemize}
\item saying a unit change in $x$ produces
a change of between $.01$ and $.03$ in $y$, or 
\item saying a unit change in $x$
produces no change in $y$
\end{itemize}
if the values of $y$ are in
the 1000's, for instance.  Statistical significance
and real-world practical significance are not the 
same things!
\index{hypothesis testing!attained significance ($p$ value)|)}

\specialsection{Small-sample Test for Differences of Means}
\index{hypothesis testing!small sample test on difference of means|(}
There are tests for differences of population means in the small 
sample case as well.  However, the assumptions required
to apply the tests in their basic forms are \emph{much} 
more restrictive than in the large sample case:
\begin{itemize}
\item We must assume that the two sets of samples come from
normal populations (since we may not be able apply the Central Limit
Theorem to claim normality for the distribution of the sample
means otherwise), and
\item We must assume that the \emph{variances of
the two normal distributions are equal}.  
\end{itemize}

We will only discuss how to determine whether the basic form
is appropriate and how to carry out the test in that case.
For the more sophisticated methods needed to deal with non-normal
data and/or the unequal
variance case, we suggest that the reader refer to more advanced statistics
texts, or perhaps even better, consult an expert statistician.  

\index{hypothesis testing!testing for normality (Shapiro-Wilk)|(}
\index{normal probability (QQ) plot}
First we note that there are a number of statistical
tests for determining whether it is reasonable to assume
that given data is sampled from a normal distribution or not.
One of the most intuitive, visual, and qualitative methods is known as the 
\emph{normal probability plot}, or \emph{QQ plot}.
The normal probability plot is a scatter plot showing a comparison
between a given data set and a reference normal distribution.\footnote{Technical
note: The name ``QQ plot'' stands for \emph{quantile-quantile plot}
and this indicates how the plot is formed.  The normal probability 
plot effectively sorts the data and then compares that sorted list 
with the expected values of the order statistics--the minimum, the next smallest,
the third smallest, etc., up to the maximum--of a sample of the same 
size drawn from a standard normal distribution.}  The
closer the normal probability plot comes to being linear, the closer
the agreement between the data and what we would expect from samples
from a normal distribution.  This could be quantified using the $R^2$
statistic or in other ways.  But in fact, ``eyeballing'' the normal
probability plot is often sufficient for our purposes.  

Other, more quantitative tests are also available, including the 
Shapiro-Wilk W-test.  The Shapiro-Wilk test is a formal test of 
\begin{itemize}
\item a null hypothesis $H_0$: The
data are sampled from a population that is normally distributed, versus
\item an alternative hypothesis $H_a$: The data are sampled from a population
that is not normally distributed.  
\end{itemize} 
The exact form of the test statistic and how rejection regions and $p$-values
are determined are beyond the scope of this discussion, but the conceptual
framework is exactly what we are discussing in this chapter.  

However, we wish to caution the reader that we think most experienced statisticians would say
that \emph{tests such as the Shapiro-Wilk test are probably of little use} 
for basic data analysis where we need to assess whether tests based on
normality assumptions are appropriate.  
The reason is that for small sample sizes, the 
test will often not distinguish between data drawn from normal distributions
and data from other distributions.  On the other hand, for large
sample sizes,\footnote{Technical note: The {\tt shapiro.test} function in {\tt R}
limits the size of the input data list to $n = 5000$ to avoid this issue.} 
even very small deviations from normality (for instance
using a large sample from a $t$-distribution with a large number
of degrees of freedom) will
result in very small $p$-values indicating that the normality
assumption should be rejected.  Since a $t(100)$ distribution is 
for all practical purposes equivalent to a standard normal, this 
is an inappropriately sensitive tool to apply to this sort of question.
\index{hypothesis testing!testing for normality (Shapiro-Wilk)|)}

Software for generating normal probability plots (and also for 
carrying out the Shapiro-Wilk test) is included in the {\tt R} statistical 
package and we will see examples of how both methods are implemented there
in the Chapter Project.  
\index{R@{\tt R} statistical package}

\index{hypothesis testing!test for equality of variances ($F$ test)|(}
Assuming that there is no reason to suspect the two datasets do not come
from normal distributions, the next question to consider is whether
the equal variance assumption is reasonable.  This is where the 
$F$-distributions considered in section 5 of Chapter 11 come into 
their own.  Looking back at Theorem~\ref{S2ChiSqRel} and Definition~\ref{Fdef},
we can see that if $S_1^2$ and $S_2^2$ are the sample variances
of two sets of samples of sizes $n_1$ and $n_2$, 
drawn from normal populations \emph{with equal variances}, then the statistic 
\begin{equation}
\label{eq:Fstat}
\framebox{$\displaystyle{F = \frac{S_1^2}{S_2^2}}$}
\end{equation}
\index{test statistic}
\index{null hypothesis}
\index{alternative hypothesis}
\index{percentage points of a distribution}
\index{R@{\tt R} statistical package}
will have an $F$-distribution with $n_1-1$ degrees of freedom in the numerator
and $n_2-1$ degrees of freedom in the denominator.  Hence, we have the 
makings of a test of 
\begin{itemize}
\item $H_0$: the population variances are equal, versus 
\item $H_a$: the population variances are different.
\end{itemize}
The test statistic is $F$ above.  With a sufficiently detailed
table of percentage points for the $F$-distributions or software
such as {\tt R} we can construct a 2-tail rejection region 
and carry out a test with any given Type I error
probability $\alpha$.  

\begin{example}
\label{Ftest}
({\it Note:}  Hypothetical data is used in this example.)
A state Wildlife Conservation Department carried out a study to understand
the build-up of chemical residues in the brain tissue of birds of a particular
species.  Random
samples of $n_1 = 11$ adult birds and $n_2 = 9$ juveniles produced
mean brain tissue DDT levels of $\overline{Y_1} = .039$ for the adults
and $\overline{Y_2} = .024$ for the juveniles (in units of parts per million).
The sample variances were $S_1^2 = .015$ and $S_2^2 = .008$.  With $\alpha = .05$, 
we ask:  Is there sufficient evidence to say that the population variances were 
different?  

The $F$ statistic from \eqref{eq:Fstat} gives 
$$F = \frac{.015}{.008} = 1.875.$$
Consulting a table of percentage points for the $F$-distributions,\footnote{We used
the tables from Wackerly, Mendenhall, and Scheaffer, {\it Mathematical Statistics
with Applications}, 6th edition here and in the later examples in this Chapter.} 
or using the cumulative distribution function for the 
$F$-distributions in {\tt R}, we find a two-tail rejection region consisting of the union
of the two intervals consisting of real values
$F > 4.30$ together with $F < .2597$.\footnote{Technical note: This is derived
from the tabulated information as follows.  With $11 - 1 = 10$ degrees of freedom
in the numerator and $9 - 1 = 8$ degrees of freedom in the denominator,
$$F_{.025}(10,8) = 4.30 \qquad {\rm and} \qquad F_{.975}(10,8) = 
\frac{1}{F_{.025}(8,10)} = \frac{1}{3.85} \doteq .2597.$$
We are using a useful fact about the percentage points of the $F$-distributions here 
that is not difficult to derive with some algebra by manipulating inequalities: 
$$F_{1 - \alpha}(\nu_1,\nu_2) = \frac{1}{F_\alpha(\nu_2,\nu_1)}$$
(note that the numbers of degrees of freedom are reversed on the right hand side!)}
Since $1.875$ is not in either interval, we do not have sufficient evidence to 
reject the null hypothesis here.  $\triangle$
\end{example}
\index{hypothesis testing!test for equality of variances ($F$ test)|)}

This indicates that we can proceed with the simplest form of the small sample
test for difference of means.  This is usually applied as follows.  Since we want to 
proceed under the assumption that the variances of brain tissue DDT levels
are equal in the two groups, the best way to estimate that variance is 
to \emph{pool the two groups} (that is, to lump them together) and compute
what is known as a pooled estimator for the common variance:
$$S_p^2 = \frac{(n_1 - 1)S_1^2 + (n_2 - 1)S_2^2}{n_1 + n_2 - 2}.$$
Under the null hypothesis that the population means are \emph{equal}
it can be shown that the test statistic
$$T = \frac{\overline{Y_1} - \overline{Y_2}}{S_p\sqrt{\frac{1}{n_1} + \frac{1}{n_2}}}$$
has a $t(n_1 + n_2 - 2)$ distribution.  Rejection regions for tests with any 
$\alpha$ are then derived as in our other examples.

\begin{example}
\label{TTestSmall}
We continue with the data from Example~\ref{Ftest}.  The pooled estimator for
the population variance is 
$$S_p^2 = \frac{10\cdot (.015) + 8\cdot (.008)}{11 + 9 - 2} \doteq .012,$$
so $S_p \doteq .109$.  
This yields the test statistic
$$T = \frac{.039 - .024}{(.109)\sqrt{1/11 + 1/9}} \doteq .306.$$
Since the percentage point $t_{.025}(18) = 2.10092$, we see that $T = .306$
is very far from lying in the rejection region.  This means that there is not
sufficient evidence to reject the null hypothesis that the mean brain tissue
DDT levels for the adults and the juveniles are actually the same.  
The fact that the adult level ``looked'' larger in these samples could be
due to chance variation.  $\triangle$    
\index{hypothesis testing!small sample test on difference of means|)}
\end{example}

\specialsection{Hypothesis Tests on Regression Coefficients}
\index{hypothesis testing!tests on regression coefficients|(}
\index{linear regression|(}
Although you might not have known it at the time, you have possibly seen
$p$-values of hypothesis tests reported before.  For instance,  
this happens when you use the regression functions 
from the Data Analysis package in Excel.

\begin{example}
\label{regresstest}
For example, suppose we enter the following
data in Excel:  in cells A1 - A8, $1,2,3,4,5,6,7,8$ and
in cells B1 - B8, $2.3,3.6,4.1,4.3,5.2,5,6.3,7$.  Highlight
those cells and from the Data/Data Analysis menu, select
Regression.  The output generated (look on the tabs below
for a new output sheet) includes something like this:
\vskip 5pt
\leftline{\rm $\qquad$ SUMMARY OUTPUT}								
\vskip 5pt								
\leftline{\rm $\qquad$ Regression Statistics}								
\leftline{\rm $\qquad$ Multiple R -- 0.97434237}							
\leftline{\rm $\qquad$ R Square -- 0.949343055}							
\leftline{\rm $\qquad$ Adjusted R Square -- 0.94090023}							
\leftline{\rm $\qquad$ Standard Error -- 0.363787396}							
\leftline{\rm $\qquad$ Observations -- 8}	
\leftline{}
\leftline{$\qquad\qquad\qquad \vdots$}
$$\begin{matrix}					
&&{\rm Coeff's} &{\rm Std\ Error} &{\rm t\ Stat} &{\rm P-value} &{\rm Lower\ 95\%} &{\rm Upper\ 95\%}\\
&{\rm Int.}&   2.046428 & 0.283460 &7.219439 &0.00035     &1.352824 &2.740032\\
&{\rm XVar.}  &0.595238 & 0.056133 &10.60395 &4.142E-05 &0.457884 & 0.732592
\end{matrix}$$
\index{R@$R^2$-statistic}
\index{standard error}
\index{hypothesis testing!attained significance ($p$ value)}
\index{confidence interval}
What is going on here? 
Well, as we know, the $R^2$ value is an indication of
how close the data was to lying on a single straight line (pretty close here!)   In 
the block at the bottom, the Coefficients column shows the computed coefficients 
$\hat{b}$ and $\hat{m}$ in the regression line.  This 
shows that the equation of the regression line is
$$y = \hat{b} + \hat{m} x = 2.046428 + 0.595238 x .$$

The rest of that block reports the results of two statistical tests.  Namely, the
line for the $b = $ Intercept (``Int.'') shows that in a test of $H_0: b = 0$
versus $H_a: b \ne 0$, the $p$-value $p = .00035$ indicates quite strong evidence
to reject $H_0$.  Similarly, the last line gives the result of a similar
test of $H_0: m = 0$ versus $H_a: m \ne 0$.  Here the results are even more
striking, and we would conclude that there is very strong evidence for
saying $m \ne 0$ from this data.  This is often a key goal of a statistical regression
analysis because in other words, it says that there is strong evidence
$y$ really does depend on $x$ in a nontrivial way.
The last two columns in the block give 95\% confidence bounds for
intervals containing $m,b$ in an actual linear model describing
the relation between $y$ and $x$ based on this data.  $\triangle$
\end{example}

Since we have not discussed linear regression from quite this perspective
before, in this section we want to describe some of the background behind
the use of regression as a \emph{statistical} technique and explain 
in general terms where this sort of information of $p$-values and 
confidence bounds on regression coefficients comes from.  This is
also the basis for some comments we made in Chapter 6 regarding the
ways naive fitting of power law models via the log-log transform
of the data and linear regression can lead to questionable results.  

In our original presentation in Chapter 4, we
described linear regression essentially simply as a data analysis tool, whereby
we could find a linear model that fit a collection $(x_i,y_i)$ of 
data points as well as possible.  This of course ignores the possibility 
that the measured $y_i$ for given values $x = x_i$ 
in a linear model can also be subject to chance variation following some 
probability distribution.
If we take those sorts of random variations into account by incorporating
a random variable \emph{error term} $\varepsilon$ in our linear models, we get
something of the form:
\begin{equation}
\label{eq:simplinmodel}
\framebox{$y = m x + b + \varepsilon.$}
\end{equation}
In this context, $x$ is often called a \emph{explanatory variable}.  Then $y$ is the 
\emph{response variable} and $y$ incorporates both deterministic and probabilistic
contributions.  

\begin{assumptions}  
\label{regassumptions}
In this context, it is common to make the following
simplifying assumptions regarding $\varepsilon$:
\begin{enumerate}
\item For all $x$, $\varepsilon$ has a known probability distribution 
with $E(\varepsilon) = 0$.  As a result, for each $x$, $y$
has a known probability distribution with expected
value $E(y) = m x + b$.  For instance, $\varepsilon$ is often
assumed to have a normal distribution with some variance $\sigma^2$ and 
mean $\mu = 0$.  
\item The variance $\sigma^2$ is \emph{the same for all} $x$.  This 
assumption has a rather forbidding name you might see if you look at other
sources--\emph{homoscedasticity}.\footnote{This comes from the 
Greek words for ``same scattering,'' or ``same variation.''}
\index{homoscedasticity}
\end{enumerate}
\end{assumptions}

To round out our discussion of issues related to fitting power
law models from Chapter 6, understanding when a log-log transformed 
dataset satisfies these sorts of assumptions is not easy because 
there is little direct connection between these assumptions and 
properties of the original dataset that would be more natural to 
consider.  

However, if we do know that Assumptions~\ref{regassumptions} hold,
then the following result gives the basis for the kinds hypothesis tests
on the regression coefficients that we saw in the output from 
Excel given above in Example~\ref{regresstest}. 
When we take a dataset $(x_i,y_i)$ 
and compute the $\hat{m}$ and $\hat{b}$ coefficients in
the equation of the regression line, we are ``really'' using the 
data to construct \emph{estimators} for the coefficients $m$ and 
$b$ in the model \eqref{eq:simplinmodel}.  As always in Chapter 11, we can think of 
the estimators as random variables, and we have the following
theoretical results.

\begin{theorem}  Under Assumptions~\ref{regassumptions}, and assuming
$\varepsilon$ has a normal distribution with constant variance $\sigma^2$,
then
\begin{enumerate}
\item[(a)]  $\hat{m}$ and $\hat{b}$ have normal distributions with expected
values equal to the coefficients
 $m$ and $b$ in the model \eqref{eq:simplinmodel}.\footnote{Technical note:
This property is usually abbreviated by saying the estimators are \emph{unbiased}.}
\item[(b)]  The variances are $V(\hat{m}) = c_{11} \sigma^2$ and 
$V(\hat{b}) = c_{00}\sigma^2$, where $c_{11}$ and $c_{00}$ are given by explicit formulas
depending on the $x_i$ from the data points.
\item[(c)]  The quantity
$$S^2 = \frac{1}{n - 2}\cdot\sum_{i=1}^n (y_i - (\hat{m} x_i + \hat{b}))^2$$
gives a estimator for $\sigma^2$.  The quantity $\frac{(n-2)S^2}{\sigma^2}$ 
has a $\chi^2$ distribution with $n - 2$
degrees of freedom.  Moreover, $S^2$ is independent from both $\hat{m}$
and $\hat{b}$.
\end{enumerate}
\end{theorem}

Then for tests of hypotheses about the slope of the regression line, 
with a value $m = m_0$ in the null hypothesis, one would use a test statistic
\index{test statistic}
$$T = \frac{\hat{m} - m_0}{S \sqrt{c_{11}}}$$
which has a $t(n-2)$ distribution.\footnote{Technical note:  This follows
from the formulas given in footnote 11 from Chapter 11.}  Similarly for tests about the
$y$-intercept, with a value $b = b_0$ in the null hypothesis one would use
$$T = \frac{\hat{b} - b_0}{S\sqrt{c_{00}}}$$
which also has a $t(n-2)$ distribution.  
Upper-tail, lower-tail, and two-tail rejection regions based on 
\index{rejection region}
given $\alpha$ would be found using the appropriate $t$-distribution. 
\index{t@$t$ distribution} 

It is the results of tests of this form that are reported by Excel and
other statistical software.  Since these computations are (almost) never
done by hand in the real world today, we will not present a worked out
example.  See Example~\ref{regresstest} above and look at the Std Error and 
$t$ Stat columns.  Those were computed using the formulas we are describing.
\index{hypothesis testing!tests on regression coefficients|)}
\index{linear regression|)}

\section{$\chi^2$ Tests for Model Fit and Independence}
\index{c@$\chi^2$ distribution}
In this section we will present additional applications of the $\chi^2$ 
distributions introduced in the final section of Chapter 11 to the 
analysis of categorical data and discrete probability models. 

\subsection*{Tests of Model Fit}
\index{hypothesis testing!tests on model fit|(}
To motivate the sort of problem we will consider, consider the following situation.

\begin{example}
\label{loadeddice}
Suppose we toss a standard six-sided die $n = 100$ times and record the following
total number of times each of the possible rolls happened as in Table~\ref{DiceData}.
\begin{center}
\begin{table}[ht]
\caption{Hypothetical data from $n = 100$ rolls of a die}
\label{DiceData}
\renewcommand\arraystretch{1.3}
\noindent\[
\begin{array}{|c|c|c|c|c|c|c|}
\hline
1&2&3&4&5&6\\
\hline
21&15&13&16&12&23\\
\hline
\end{array}
\]
\end{table}
\end{center}
We are dealing with \emph{categorical} data here because
each roll of the die gives us one of the 6 possible numbers
$1,2,3,4,5,6$.  If we examine the data,
since the numbers of 6's and 1's are seemingly much larger
than the numbers of the other possible rolls, it is natural
to consider the question:  Do we have evidence here that
the die is \emph{``loaded''} (i.e. not a fair die)?  
$\triangle$
\end{example}

Another way to phrase this question is:  Is this data consistent
with the probability model of a fair die, where each of the six
rolls is equally likely?  The British statistician Karl Pearson
(1857 - 1936) introduced the following method for addressing
this sort of question.  Phrasing this in the language of 
hypothesis testing, we want to develop a test with

\begin{itemize}
\item A null hypothesis, $H_0$:  The data is consistent with
a fair die.
\item An alternative hypothesis, $H_a$: it is not.
\end{itemize}

More generally, we could consider similar questions related
to the following sort of categorical data and a 
discrete probability model.  Suppose we 
have a discrete random variable that can take some number
$k$ of possible different values, and we have probability mass
function representing a model giving probabilities 
$p_i$, $i = 1, \ldots, k$ for each value.  We sample some number 
$n$ of values of the random variable, determine the \emph{observed frequencies}
$O_i$, and compare them to the and \emph{expected frequencies} $E_i = p_i n$.
We want to test:
\index{null hypothesis}
\index{alternative hypothesis}
\begin{itemize}
\item A null hypothesis, $H_0$:  The data is consistent with
the probability model, versus
\item An alternative hypothesis, $H_a$: it is not.
\end{itemize}

Pearson introduced the following test statistic for measuring
the deviation from an expected distribution. 
\begin{equation}
\label{eq:X2Stat}
\framebox{$\displaystyle{X^2 = \sum_{i=1}^k \frac{(O_i - E_i)^2}{E_i}}$.}
\end{equation}
He then established that the distribution of $X^2$ would be
\index{test statistic}
\index{c@$chi^2$ distribution}
\index{Pearson, Karl}
approximately a $\chi^2$ distribution with $k - 1$ degrees of
freedom for $k$ sufficiently large.\footnote{Technical note: This 
can be established using the Central Limit Theorem, but the details
are far beyond the scope of this discussion.}  Even if $n$ is 
not large, the percentage points of the appropriate $\chi^2$
distribution can be used define rejection regions and possibly 
to estimate a $p$-value for an upper tail test.   Let us see how 
this works using the data from Example~\ref{loadeddice} above.  

\begin{example}
\label{loadeddicecont}
The data in Table~\ref{DiceData} gives
the $O_i$ for $i = 1, \ldots, 6$.  The $E_i$ come from the probabilities for
a fair die: $E_i = \frac{1}{6} \times 100 \doteq 16.7$ for $i = 1,\ldots, 6$.  
We compute
\begin{align*}
X^2 &= \frac{(21-100/6)^2}{100/6} + \frac{(15-100/6)^2}{100/6} + \frac{(13-100/6)^2}{100/6}\\
&\qquad +\frac{(16-100/6)^2}{100/6} + \frac{(12 - 100/6)^2}{100/6} + \frac{(23-100/6)^2}{100/6}\\
&= \frac{146}{25} = 5.84
\end{align*}
From a table of percentage points for $\chi^2$ distributions, we see that with $6 - 1 = 5$
degrees of freedom, $\chi^2_{.05} = 11.0705$ and $\chi^2_{.1} = 9.23635$. 
Hence with $\alpha = .05$ or even $.1$, 
we do not have enough evidence to reject the null hypothesis that the observed
counts are consistent with a fair die.  The actual
\index{hypothesis testing!attained significance ($p$ value)}
$p$-value of this test is $p \doteq .3221$,\footnote{Technical note:  
This can be computed as 
$$\int_{5.84}^\infty f(t)\ dt$$
using $f(t) = $ the $\chi^2(5)$ density and numerical 
integration software or by means of the functions in the 
{\tt R} statistical package.} which is definitely 
far too large to consider rejecting the null hypothesis. One way 
to interpret the $p$-value here is that if we repeated the experiment
of rolling the die $n = 100$ times, then we would see unequal
distributions of the six possible numbers \emph{at least this extreme\footnote{in the
sense that the $X^2$ statistic computed from the observed frequencies would be at least this 
large} only a bit less than $1/3$ of the time(!)}   $\triangle$
\end{example}
\index{hypothesis testing!tests on model fit|)}

\subsection*{Tests of Independence}
\index{hypothesis testing!tests on independence of categorical variables|(}
Commonly-used hypothesis tests for independence are also based on the 
$\chi^2$ distribution. We will begin by describing what we are looking for
in this context.  Saying two categorical random variables $X,Y$ are
independent should mean that for all possible pairs of values $(x,y)$
in the joint sample space for $X,Y$, the 
events described by the equations $X = x$ and $Y = y$ are independent.
In other words, from \eqref{eq:indep}, we have
\begin{equation}
\label{eq:indepcond}
P(X = x\ {\rm and}\ Y = y) = P(X = x)\cdot P(Y = y).
\end{equation}
Suppose we have gathered data about two such categorical variables, recording
the total frequencies of all the possible pairs.  
\index{contingency table}

\begin{center}
\begin{table}[ht]
\caption{Hair and Eye Color data}
\label{haireyestable}
\renewcommand\arraystretch{1.3}
\noindent\[
\begin{array}{|c|c|c|c|c|c|}
\hline
            & {\rm Black} & {\rm Brunette} & {\rm Red} & {\rm Blond} & {\rm Total}\\
\hline
{\rm Brown} & 68 & 119 & 26 & 7 & 220\\
\hline
{\rm Blue}  & 20 & 84  & 17 & 94 & 215\\
\hline
{\rm Hazel} & 15 & 54 & 14 & 19 & 93\\
\hline
{\rm Green} & 5 & 29 & 14 & 16 & 64\\
\hline
{\rm Total} & 108 & 286 & 71 & 127 & 592\\
\hline
\end{array}
\]
\end{table}
\end{center}

\begin{example}
\label{haireyes}
Table~\ref{haireyestable} shows data frequencies of combinations of
hair color and eye color from a sample of $n = 592$ human subjects.\footnote{From 
R. Snee, ``Graphical display of two-way contingency tables.'' {\it Amer. Statist.} {\bf 38}
(1974), 9 - 12, quoted in: P. Diaconis and B. Efron, ``Testing for Independence in a 
Two-way Table: New Interpretations of the Chi-Square Statistic,'' {\it Ann. Stat.} {\bf 13} (1985), 
845-874.}  We might ask: Given this data, is it reasonable to conclude that the 
hair color (Black, Brunette, Red, Blond) and the eye color (Brown, Blue, Hazel, Green)
are independent characteristics in humans?  Or does the data
favor the existence of some \emph{association} between these traits?  $\triangle$ 
\end{example}

In environmental science, similar questions come from studies of 
frequencies of different species of plants or animals in habitats
with different characteristics and many other related questions.

The format of the data in Table~\ref{haireyestable} is an example
of what is known as a \emph{two-way contingency table}.  The
last column and the last row (labeled Total) give, respectively, the row sums
and the column sums in the main, $4\times 4$ section of the table
at the upper left.\footnote{Technical note: These are often called
the \emph{empirical marginal distributions} of the two traits separately.}  
The sum of the row sums equals the sum of the 
column sums and that entry on the last row at the lower right gives the total 
sample size $n = 592$.  

The basic idea of testing for independence with this sort of 
two-way categorical information is essentially 
to consider \emph{all tables of the same size with the same
row and column sums (and hence the same $n = 592$)}.  It is easy 
to see that there are \emph{an extremely large number of such 
tables}, since for instance we can (repeatedly) take any two rows and two
columns in the $4 \times 4$ upper left section of the table
and change the entries in the pattern $+1, -1$ in the first
of the rows and $-1, +1$ in the second of the two rows, yielding
new contingency tables with the same row and column sums.\footnote{Strictly speaking,
we should also add the requirement that the subtractions leave entries that 
are still non-negative.}  

We will also need to consider such tables where the entries
are not necessarily integers.  Among such more general tables, we find
exactly one corresponding to the independence condition.  To describe this table, 
let us introduce the notation $E_{ij}$ to represent the expected count
under independence in row $i$ and column $j$.  Thinking about \eqref{eq:indepcond},
we see that we want for a general $n$,  
$$\frac{E_{ij}}{n} = \left(\frac{{\rm row\ sum\ for\ row\ } i}{n}\right) \cdot 
\left(\frac{{\rm colum\ sum\ for\ row\ } j}{n}\right),$$
so 
$$E_{ij} = \frac{({\rm row\ sum\ for\ row\ } i)\cdot ({\rm colum\ sum\ for\ row\ } j)}{n}.$$

\begin{center}
\begin{table}[ht]
\caption{Expected Frequencies for Hair and Eye Color Under Independence}
\label{indephaireyestable}
\renewcommand\arraystretch{1.3}
\noindent\[
\begin{array}{|c|c|c|c|c|c|}
\hline
            & {\rm Black} & {\rm Brunette} & {\rm Red} & {\rm Blond} & {\rm Total}\\
\hline
{\rm Brown} & \frac{1485}{37} & \frac{7865}{74} & \frac{3905}{148} & \frac{6985}{148} & 220\\
\hline
{\rm Blue}  & \frac{5805}{148} & \frac{30745}{296}  & \frac{15265}{592} & \frac{27305}{592} & 215\\
\hline
{\rm Hazel} & \frac{2511}{148} & \frac{13299}{296} & \frac{6603}{592} & \frac{11811}{592} & 93\\
\hline
{\rm Green} & \frac{432}{37} & \frac{1144}{37} & \frac{284}{37} & \frac{508}{37} & 64\\
\hline
{\rm Total} & 108 & 286 & 71 & 127 & 592\\
\hline
\end{array}
\]
\end{table}
\end{center}

\begin{example}
\label{haireyescont}
Carrying out this process with the row and column sums from Table~\ref{haireyestable},
you will check in Exercise~\ref{ex:haireyesex} that the resulting independence table 
has the form given in Table~\ref{indephaireyestable}.
These could also be expressed using decimal approximations, of course, but
these rational number values are \emph{exact}.  $\triangle$
\end{example} 

For the test of independence, we use a test statistic of the same form
\index{test statistic}
as in the test of fit.  If there are $r$ rows and $c$ columns of observed
frequencies data $O_{ij}$ (excluding
the row and column sums), then we consider
\begin{equation}
\label{eq:X2Indep}
\framebox{$\displaystyle{X^2 = \sum_{i=1}^r \sum_{j=1}^c \frac{(O_{ij} - E_{ij})^2}{E_{ij}}.}$}
\end{equation}
In words, using the observed frequency $O$ and the expected
frequency $E$ for each cell in the $r \times c$ 
main portion of the table (not including the row and column sums), 
we compute $\frac{(O - E)^2}{E}$ and sum them all.  

It can be shown that for sufficiently large $r,c$, this $X^2$ statistic has a 
$\chi^2$ distribution with $\nu = (r - 1)\cdot (c - 1)$ degrees of 
\index{c@$chi^2$ distribution}
freedom.  This gives the infrastructure needed for a test of 
\begin{itemize}
\item A null hypothesis, $H_0$:  The data is consistent with
the independence model, versus
\item An alternative hypothesis, $H_a$: it is not.
\end{itemize}

\begin{example}
\label{haireyesconc}
Using \eqref{eq:X2Indep}, you will check that $X^2 \doteq 138.29$ for the 
hair and eye color data.  For $(4 - 1)\cdot (4 - 1) = 9$ degrees of freedom, from 
a table of percentage points of the $\chi^2$-distributions, we find
$\chi^2_{.005} \doteq 23.5893$.  So we would reject the null hypothesis
even with the very small $\alpha = .005$.   In fact the actual $p$-value here is on the order
\index{hypothesis testing!attained significance ($p$ value)}
of $2.3 \times 10^{-25}$, so there is very strong evidence to reject the 
null hypothesis that eye and hair color are independent. $\triangle$
\end{example}

One might argue that in this case we are just verifying something that
is obvious to any observant person--there is a perceptible association between
certain hair colors and certain eye colors due to patterns in human genetics and the 
traits of different ethnic groups within the whole human population.  
In a sense, of course, that is true.  However, our purpose here is to
demonstrate a generally applicable statistical method that can be used
to tease out associations in other cases that might not be as familiar
and where our intuition and background knowledge do not necessarily provide 
any help.  
\index{hypothesis testing!tests on independence of categorical variables|)}
\index{statistics!inferential|)}
\index{statistical inference|)}
\index{hypothesis testing|)}
\index{statistics!hypothesis testing|)}

\section{Chapter Project}

\subsection*{The Scientific Question}
Let $X$ be the lengths of male spiders of a particular large species
and let $Y$ be the lengths of female spiders of the same
species (both in mm).
A random sample of $n_X = 9$ $X$ values were taken:
$$20.4, 21.7, 21.9, 21.4, 21.1, 23.6, 18.9, 22.6, 21.3$$
Similarly, a random sample of $n_Y = 13$ observations of $Y$ were
made:
$$20.5, 20.4, 20.3, 21.1, 21.2, 20.9, 21.0. 21.3, 20.9, 20.0, 20.4, 20.8, 20.3$$
Is there a statistically demonstrable 
difference in the length distributions of two sexes, though?  
Is this species of spiders significantly sexually \emph{dimorphic}\footnote{That is, occurring
in two different forms.} with respect to body size?

\subsection*{Answering the Question}
If you think about discussion of small-sample confidence intervals
for a difference of means from the text, you can see that
there would be a corresponding hypothesis test.
However, there were assumptions that we needed to check
to apply the confidence interval formulas based on the $t$-distribution.
\begin{enumerate}
\item  What were they?  
\item  There was one assumption about the overall distributions of
the populations the sample values were coming from.  Is there any reason to
suspect that assumption is not satisfied for this data?  
Check using the {\tt qqnorm} (normal probability plot) and {\tt shapiro.test}
(Shapiro-Wilk test) features of {\tt R}.  You 
can find all the information you need about format and syntax of these commands
in many different places.  Note:  You
should be able to interpret the output from {\tt shapiro.test} by thinking
about what we have said about hypothesis tests.
\item There was also a second assumption about population variances.   This 
will be tested using a hypothesis test 
based on an $F$-statistic as described in Example~\ref{Ftest}.
Test the null hypothesis $H_0 : \sigma_X^2 = \sigma_Y^2$
against the alternative hypothesis $H_a : \sigma_X^2 \ne \sigma_Y^2$.
You may select the significance level $\alpha$.  Interpret your 
results and give the attained significance level ($p$-value).
\item  Then, if there is {\it no demonstrable difference} in 
the variances, test for equality of the means using the 
$t$-test from the text.
Use the pooled estimator $S_p$ for the common variance, and
test statistic 
$$t = \frac{\overline{X} - \overline{Y}}{S_p\sqrt{ \frac{1}{n_X} + \frac{1}{n_Y} }}\ . $$ 
For the alternative hypothesis $H_a : \mu_X \ne \mu_Y$
take the rejection region
$$RR = \{t \mid |t| \ge t_{\alpha/2}(n_X + n_Y - 2)\}.$$
\item  If there {\it is a demonstrable difference}
in the variances, the basic $t$-test is not all that
reliable in some cases.  With small sample sizes, many experienced
statisticians would use a different method due to Welch.  
Use the following test statistic:
$$t = \frac{\overline{X} - \overline{Y}}{\sqrt{\frac{S_X^2}{n_X} + \frac{S_Y^2}{n_Y}}},$$
but set up the rejection region for the test using a $t$-distribution 
with $r$ degrees of freedom where 
$$r = \left[\frac{(S_X^2/n_X + S_Y^2/n_Y)^2} 
{(S_X^2/n_X)^2/(n_X-1) + (S_Y^2/n_Y)^2/(n_Y-1)}\right]\leqno(1)$$
($[z] = $ greatest integer less than or equal to $z$).\footnote{Technical note:
The difference here is that the two samples are \emph{not} being lumped
together to compute the pooled estimator for the variance, but the 
number of degrees of freedom in the $t$-distribution is being 
adjusted to account for the two different estimates $S_X^2$ and $S_Y^2$.}   
In this case, to test the null hypothesis $H_0: \mu_X = \mu_Y$ against
the alternative hypothesis $H_a : \mu_X \ne \mu_Y$, you would compute
$t$ as above and reject $H_0$ if $|t| \ge t_{\alpha/2}(r)$ where $r$
is the number of degrees of freedom given in (1).
\item[] {\bf Note:} You will need to select which of the 
two alternatives 4 and 5 you will use here, how you will
set up the rejection region for your test, 
the test statistic, etc. based on the results of your test from 
part 1.   Explain your choice.  Clearly state the conclusion
you draw from the test you carry out.
\end{enumerate}

\subsection*{Assignment} Write up a report describing what tests
you applied, what the results were, and your conclusion about 
the underlying scientific question about whether this species
is \emph{dimorphic} in body size.

\begin{xcb}{Chapter Exercises}
\vskip 5pt
{\it Directions for questions 1 - 4:}  For each question, first identify
the relevant null and alternative hypotheses.  Then
carry out the appropriate hypothesis test, estimate the 
$p$-value if possible with the information you have, and answer
the other questions. \vskip 5pt  
\begin{enumerate}
\item  $CO_2$ emissions per Megawatt-hour of electric
power produced were measured for $n = 100$ coal-fired power plants.
The average $CO_2$ in lb/Mwh
is $\overline{Y} = 2223.1$ with $SD = 211.3$ lb/Mwh. \vskip 5pt
\begin{enumerate}
\item[(a)]  If this were a random sample, would there
be evidence at the $\alpha = .05$ level
to say that the average coal-fired power plant
emits an amount different from $2200$ lb/Mwh? \vskip 5pt
\item[(b)]  The data used in this problem actually comes
from the 100 largest coal-fired electric power plants in the U.S. (this is
true).   Is this a random sample?  Does the calculation
in part a) really make sense in this setting?  Discuss.
\end{enumerate}
\vskip 5pt
\item  A simple random sample of 2000 Germans in 2001 
showed that 840 thought that all electricity should be
generated by ``green'' energy sources such as wind and 
solar power.  Is there evidence at the $\alpha = .05$ level
to say that at least $40\%$ of all Germans thought the
same about sources of electric power?
\vskip 5pt
\item  In a study of $n = 17$ sea star arm lengths,
the average was $\overline{Y} = 6.8$ cm with an 
SD $= .5$ cm.  Is there evidence at the $\alpha = .05$ level
to say that the population arm length is different 
from $7$ cm?  Estimate the $p$-value of the test
using the information in Figure~\ref{fig:PctPtsT}. 
\vskip 5pt
\item
A medical study compared the resting pulse
rates of random, independent samples of 100 smokers and 100 nonsmokers.  The 
smokers had an average pulse rate of $\overline{Y} = 86$
beats per minute with $SD_1 = 5.4$, while the nonsmokers had an
average pulse rate of $\overline{X} = 80$ and $SD_2 = 4.9$.
Is there evidence at the $\alpha = .05$ level to say that
nonsmokers have a lower average pulse rate?   
\vskip 5pt
\item Criticize the following statement.  Is it true?  Is it
false?  Is it a reasonable way to think about what the $p$
value means?   ``\emph{The $p$-value of a
statistical test of hypotheses is the probability that the null 
hypothesis is true.}''
\vskip 5pt
\item (Hypothetical data) 
A new diet regimen is being tested against an old diet
for farmed tilapia.\footnote{Tilapia are an important aquaculture species.  They 
are fast growing, lean and short-lived, so they do not accumulate mercury and 
other toxins to the same level that other food fish do.  They are a good protein
source, but their flesh generally does not contain as much of the beneficial omega-3
fatty acids as other food fish species such as salmon.} To 
compare the eventual body weights of the fish raised
under the old and new diets, 
random samples with $n = 30$ are selected from
fish raised with each diet. 
Each fish is weighed after reaching 240 days in age (full growth).
Let $X$ be the weight of a fish raised on the new diet, and $Y$ be 
weight of a fish raised on the old diet, both in units of grams.  The goal
is to determine whether the new diet increases the adult body weight
of the fish.  The data collected was 
as follows:
$$X: 
\begin{matrix}
1250 & 1210 & 990 & 1310 & 1320 & 1200 & 1290 & 1360 & 1200 & 1150\\
        1120 & 1360 & 1310 & 1110 & 1320 & 980 & 950 & 1430 & 1100 & 1080\\
        960 & 1050 & 1310 & 1240 & 1420 & 1170 & 1470 & 1060 & 1230 & 1300 
\end{matrix}$$
and 
$$Y: 
\begin{matrix}
1180 & 1360 & 1310 & 1190 & 920 & 1060 & 1440 & 1010 & 1000 & 950\\
        1310 & 980 & 1310 & 1030 & 960 & 800 & 1280 & 1080 & 900 & 1030\\
        930 & 1050 & 1010 & 1310 & 940 & 860 & 1450 & 1070 & 840 & 1100 
\end{matrix}$$
\begin{enumerate}
\item[(a)] Construct parallel ``box-and-whisker plots'' 
for these data sets as in Chapter 9 and make an informal 
conjecture about whether or not the new process (the $X$
data) has increased the fish body weight, compared with the $Y$ data.\vskip 5pt
\item[b)] Describe an appropriate test of the null hypothesis
$H_0: \mu_X = \mu_Y$ versus $H_a: \mu_X > \mu_Y$.  Say what your
assumptions about the data are, what your 
test statistic is, what the rejection region will be, and so forth.\vskip 5pt
\item[c)]  Carry out your test at the $\alpha = .01$ level of 
significance.  Give a clear and concise statement of the conclusion
you draw from your test. \vskip 5pt
\item[d)]  What is the attained significance level of your test
(the $p$-value)?  What does this say? 
\end{enumerate}
\vskip 5pt
\item A researcher found that in two independent random samples,
107 out of 200 apple seeds germinated when they were planted in soil maintained 
at $5 {}^\circ C$, while 123 out of 200 seeds germinated when they 
were planted in soil maintained at $15 {}^\circ C$.
\vskip 5pt
\begin{enumerate}
\item[(a)]  Construct a $95\%$ confidence interval for 
the difference in the proportions of seeds that germinate
at the two temperatures using our discussion of confidence
intervals from Chapter 11.  State any assumptions you are making.\vskip 5pt
\item[(b)]  Do the results of this study suggest that 
a greater proportion of apple seeds will germinate at the 
higher temperature in general?  Construct an appropriate 
test of hypotheses, carry it out, and interpret your results.
How does your answer relate to what you found in part (a)?
\end{enumerate}
\vskip 5pt
\item Consider the following measurements of the heat-producing
capacity of the natural gas produced by two fields of gas wells
(in calories per cubic meter):
$$\begin{matrix}
{\rm Field\ 1:}& 8.26 & 8.13 & 8.35 & 8.07 & 8.34 &\\
{\rm Field\ 2:}& 7.95 & 7.89 & 7.90 & 8.14 & 7.92 & 7.84
\end{matrix}$$
Let $\mu_i$ ($i = 1,2$) be the population mean heat-producing capacity
of the natural gas from field $i$.  
\vskip 5pt
\begin{enumerate}
\item[(a)] What assumptions do you need to make in order 
to use the appropriate test of $H_0: \mu_1 = \mu_2$ versus
$H_a: \mu_1 \ne \mu_2$?  How could you determine whether 
it is reasonable to assume those assumptions are satisfied?  
\item[(b)]  Set up and carry out appropriate tests
to determine whether the assumptions from part (a) are satisfied.
\item[(c)]  Carry out the test on the population mean heat-producing
capacities and state your conclusion clearly and succinctly.
\end{enumerate}
\vskip 5pt
\item
\label{ex:haireyesex}
Verify the form of the expected frequencies
$(E_{ij})$ in Table~\ref{indephaireyestable}.
\vskip 5pt
\item 
Check the value $X^2 \doteq 138.29$ obtained from Tables~\ref{haireyestable} and 
\ref{indephaireyestable}.  Which of the terms in the sum from \eqref{eq:X2Indep}
are especially large in this case?  What do these large terms mean in real
world terms?
\vskip 5pt
\item Suppose we have an $r \times c = 2\times 2$ two-way contingency
table as in Table~\ref{twobytwo} with all $a,b,c,d > 1$.  The row and
column headings (i), (ii), (I), (II) just indicate that there
are two possible values for each of the categorical variables.  
\begin{center}
\begin{table}[ht]
\caption{A general $2\times 2$ two-way contingency table}
\label{twobytwo}
\renewcommand\arraystretch{1.3}
\noindent\[
\begin{array}{|c|c|c|c|}
\hline
            & (i) & (ii) & {\rm Total}\\
\hline
(I) & a & b & a+b \\
\hline
(II)  & c & d & c+d\\
\hline
{\rm Total} & a+c & b+d & a+b+c+d\\
\hline
\end{array}
\]
\end{table}
\end{center}
Show that the $X^2$ statistic in this case is computed by the 
shortcut formula:
$$X^2 = \frac{n(ad - bc)^2}{(a+b)(a+c)(b+d)(c+d)}$$
Hint: This is just a big algebra exercise(!)  Unfortunately, 
for larger tables, this kind of approach quickly becomes totally
unwieldy. \vskip 5pt

\item
A New York Times/CBS poll conducted between April 5 and April 12, 2010 
included the question ``Do you approve or disapprove of the way your 
Representative in Congress is handling his or her job?''  The poll was 
carried out in two stages.  First a large national 
simple random sample was asked the question.  The results were reported
in three categories:
$46\%$ of respondents said they approved, $36\%$ said they disapproved, and
$18\%$ said they had no opinion. 
Then the same question was asked of a separate sample of $n = 881$ people, 
all of whom had identified themselves as supporters of the ``Tea Party'' 
movement.  The responses broke down like this in the same 
three categories:
$$\begin{matrix} & {\rm Approve} &  {\rm Disapprove}  & {\rm No\ opinion} \\
{\rm Tea\ Party\ Sample} &   352   &   432   &   97 
\end{matrix}$$
(Data is adapted from the New York Times web site.)  It seems from the numbers that 
``Tea Party'' supporters may differ from the general population when it comes to
their opinions concerning their Congressional Representatives.  
But is this a real difference, or could it just be a product of chance
variation in the sampling process?  Test using an appropriate $\chi^2$ test of
model fit.
\vskip 5pt
\item 
In a study of bird populations in different habitats in California,
Latta et al.\footnote{Latta, S.C., C.A. Howell, M.D. Dettling, and R.L. Cormier. 
``Use of data on avian demographics and site persistence during overwintering to assess quality 
of restored riparian habitat,'' Conservation Biology {\bf 26} (2012), 482-492.} 
collected samples in ``remnant'' habitats (that, is areas along rivers with 
mostly native vegetation) and ``restored'' habitats (environmentally damaged areas
where native vegetation was replanted by humans). 
They observed the data of numbers of birds of different species
in each type of habitat (the ``Uncommon'' category is a
catch-all for less-common species) given in Table~\ref{birdtable}.
\begin{center}
\begin{table}[ht]
\caption{Bird counts in remnant and restored habitats}
\label{birdtable}
\renewcommand\arraystretch{1.3}
\noindent\[
\begin{array}{|c|c|c|}
\hline
{\rm Species} & {\rm Remnant} & {\rm Restored}\\
\hline
{\rm Ruby-crowned\ kinglet} & 677 & 198\\
\hline
{\rm White-crowned\ sparrow} & 408 & 260\\
\hline
{\rm Lincoln's\ sparrow} & 270 & 187\\
\hline
{\rm Golden-crowned\ sparrow} & 300 & 89\\
\hline
{\rm Bushtit} & 198 & 91\\
\hline
{\rm Song\ sparrow} & 150 & 50\\
\hline
{\rm Spotted\ towhee} & 137 & 32\\
\hline
{\rm Bewick's\ wren} & 106 & 48\\
\hline
{\rm Hermit\ thrush} & 119 & 24\\
\hline
{\rm Dark-eyed\ junco} & 34 & 39\\
\hline
{\rm Lesser\ goldfinch} & 57 & 15\\
\hline
{\rm Uncommon} & 457 & 125\\
\hline
\end{array}
\]
\end{table}
\end{center}
Do these data support the conclusion that 
there is an association between the frequencies of 
different bird species and the type of habitat, or 
are those two variables independent? 
\end{enumerate}
\end{xcb}
\endinput
