\chapter{Random Variables and Probability Distributions}

\specialsection{Introduction}
In Chapter 2, we discussed how ratios are used to quantify the probability
\index{probability} of an event
in the standard, frequentist developments of probability theory.  Recall that the 
idea was that if we perform some (possibly large) number of trials of an experiment or 
measurement, then the probability of a certain event or outcome could be 
estimated by a ratio of the form:
\begin{equation}
\label{eq:Freq}
\frac{{\rm Number\ of\ times\ event\ is\ observed}}{{\rm Total\ number\ of\ trials}}.
\end{equation}
For instance, if we throw a fair standard six-sided die some number of times, we could
count the number of times each of the six possible numbers comes up and divide each
of those counts by the total number of throws.    In this situation,
we would usually also have a theoretical \emph{probability model} of a fair die in mind where
the probability of each number coming up is exactly $\frac{1}{6}$.  While we expect
some deviation from the exact 
$\frac{1}{6}, \ \frac{1}{6}, \ \frac{1}{6}, \ \frac{1}{6}, \ \frac{1}{6}, \ \frac{1}{6}$
distribution in any real-world instance of this sort of experiment, if the die is not ``loaded'' to a significant
degree, we would also expect
the ratios as in \eqref{eq:Freq} to be close to $\frac{1}{6}$ for each of the 6 numbers.

In this chapter we will develop more of the theory behind probability distributions in 
both discrete and continuous cases.  In that sense, this chapter and the next one will be somewhat 
different from the others, since the goal here will be to develop some new mathematical
and statistical tools, and not so much to show how those tools used
for environmental questions.  

The starting point will be a discussion of several additional concepts from the theory 
of probabilities, culminating in the important notion of \emph{independence} of events.  
We will introduce several types of discrete and continuous random variables and
discuss computing their expected value and variance.  Finally, we will introduce an important family
of continuous probability distributions known as the \emph{normal distributions}.  
In the Chapter Project, we will illustrate a key result known as the Central Limit Theorem, 
which shows that normal distributions naturally arise in processes where we average
large numbers of measurements.  The formal statement will be given in a later chapter.

\specialsection{Conditional Probabilities and Independence}
In section 4 of Chapter 2, we discussed assigning probabilities to events $E$ in 
sample spaces $S$ in an intuitive way.  More formally, we can also
describe the properties that such an assignment of probabilities should have as 
in Assumptions~\ref{probs} below.  In this and later in this section, we will 
need to use the following notation from basic set theory.  If $A,B$ are sets, then 
\begin{itemize}
\item $A \cup B$ is the \emph{union} of $A$ and $B$, defined as the set consisting of 
all elements that are \emph{in $A$, in $B$, or in both}.
\item $A \cap B$ is the \emph{intersection} of $A$ and $B$, defined as the set consisting
of all elements that are \emph{both in $A$ and in $B$}.  
\end{itemize}
For instance if $A = \{1,2,3,4,5,6,7\}$ and $B = \{2,4,6,8,10\}$, then 
$$A \cup B = \{1,2,3,4,5,6,7,8,10\}$$ 
and 
$$A \cap B = \{2,4,6\}.$$

\begin{assumptions} 
\label{probs} 
\index{probability space (axioms for)}
If $S$ is a sample space for an experiment we will assume
we have a way to assign a probability $P(A)$ to each subset $A \subseteq S$ 
satisfying the following properties
\begin{enumerate}
\item $P(A) \ge 0$ for all $A \subseteq S$,
\item $P(S) = 1$, and
\item If $A_1,\ldots,A_n$ is any sequence of \emph{pairwise disjoint} 
subsets of $S$\footnote{This means that $A_i \cap A_j = \emptyset$, the 
empty set, whenever $i$ and $j$ are distinct indices.} then
$$P(A_1 \cup \cdots \cup A_n) = P(A_1) + \cdots + P(A_n).$$
\end{enumerate}
\end{assumptions}

The last assumption here, for instance, is what is needed to justify computations
of probabilities of compound events by summing probabilities of their elements as in 
Examples~\ref{FishPoolContinued} and \ref{TwoDice} from Chapter 2.  

\subsection*{Conditional Probabilities}
\index{conditional probability|(}
\index{probability!conditional|(}
Suppose we know that we have rolled an odd number on roll of a single fair die.
What is the probability that the roll is a $1$?  A moment's thought will convince
you that the answer here should be $1/3$.  Knowing that the roll is odd cuts the possibilities
down to $B = \{1,3,5\}$ and if the die is fair, each of those three possibilities 
is equally likely.  This is an example of what is known as a \emph{conditional probability}.  
The formal definition looks like this:

\begin{definition}
The \emph{conditional probability} of an event $A$, given that another event $B$
with $P(B) \ne 0$
has occurred,\footnote{Saying an event ``has occurred'' is too suggestive to resist, but
it really just means that we are looking at outcomes in the intersection $A \cap B$.}
written $P(A|B)$, is defined to be 
\begin{equation}
\label{eq:condprobdef}
\framebox{$\displaystyle{P(A|B) = \frac{P(A \cap B)}{P(B)}.}$}
\end{equation}
\end{definition}

In other words, we in effect restrict our attention to a new sample space consisting of the 
subset $B \subseteq S$ and consider the probability of $A$ in that new context, 
ignoring what happens for any part of $A$ outside of $B$.  For
future reference, we note that \eqref{eq:condprobdef}
can also be rearranged to the form
\begin{equation}
\label{eq:condprobdefrearranged}
P(A \cap B) = P(A|B) P(B).
\end{equation}

\begin{example} 
\label{SimpleDice}  
The simple example given at the start of this subsection translates into this 
language as follows.  The set of odd rolls $B = \{1,3,5\}$ 
sits inside the sample space $S = \{1,2,3,4,5,6\}$
for the roll of a single die.  Since the die is fair, each roll has probability 
$1/6$, so $P(B) = 3/6 = 1/2$.  $A = \{1\}$ so $A \cap B = \{1\}$ as well.  Hence
$$P(A|B) = \frac{P(A\cap B)}{P(B)} = \frac{1/6}{1/2} = 1/3,$$
as we expected.

Similarly, if we throw two distinguishable fair dice (say one with white background
and dark spots, and the other with dark background and white spots) as in Example~\ref{TwoDice}, 
we can ask, given that the total on the dice is a 7, what is the probability that one of the
dice shows a 5 or a 6?  Here $B$ represents the possible throws yielding a total of $7$:
$$B = \{(1,6),(2,5),(3,4),(4,3),(5,2),(6,1)\}.$$
$A$ is the event that one of the two dice shows a $5$ or a $6$.  That is true in $4$ of the
$6$ elements in $B$, so we can see $P(A|B) = 4/6 = 2/3$.  This would also come from the 
definition, since 
$$P(A|B) = \frac{P(A \cap B)}{P(B)} = \frac{4/36}{6/36} = 2/3. $$
On the other hand, you will verify in Exercise~\ref{ex:Dice} that $P(A) = 5/9 < 2/3$. $\triangle$
\end{example}

In both cases considered in Example~\ref{SimpleDice}, it
turned out that $P(A|B)$ was greater than $P(A)$.  It is also possible for 
$P(A|B)$ to be equal to $P(A)$ or less than $P(A)$.  For example, if $A \cap B$ is 
empty, then $P(A|B)$ will usually equal zero, even if $P(A) \ne 0$.  In any case the idea is that 
knowing $B$ \emph{gives more information} and hence can change the probability of $A$,
making it more or less likely in some circumstances.  
Here is a famous and counterintuitive example of this phenomenon.

\begin{example}
\label{MontyHallSetup}
\index{Monty Hall problem|(}
On the classic daytime TV show ``The Price is Right,'' the host Monty Hall
\index{Hall, Monty}
would often play the following game with contestants.  Monty would present
three doors.  Behind one was a desirable prize, a new car, while 
what were considered undesirable ``prizes,'' 
live goats, were behind the other two.\footnote{There are certainly some people in the world
who would in fact be happier to win a live goat than a car, so the game 
probably only works as intended for American or other 
``first-world'' contestants!}  The contestant would be asked to choose
one of the three doors.  Monty would then reveal one of the two remaining
doors with one of the goats.  Note that whichever door the contestant chose to begin
with, there would be a goat behind one of the two remaining doors, and \emph{very 
importantly}, Monty always knew which doors had which prizes.  At this point,
the contestant was asked whether he or she wanted to stick with the door
he or she chose to start, or whether he or she wanted to switch to the 
other remaining door.  

The question is:  What was the best strategy for the contestant to use--stick with 
the original choice, or switch?  

If you have never heard of this example before, 
you are encouraged to pause and experiment by simulating this game with three playing cards 
and a friend before continuing to the next example.  
Take an ace (representing the car) and two twos (representing the 
goats).  Have your friend play Monty Hall, who always knows where the goats are
and shows you one of them, giving you the chance to switch.  Play 
the game 25 or 50 times using the ``always stick with the original choice'' strategy 
and record how many times you win the car.  Then repeat using the ``always switch''
strategy and record how many times you win the car in 25 or 50 tries. $\triangle$
\end{example}

\begin{example}
\label{MontyHallAnalysis}
If you carried out the simulation suggested in Example~\ref{MontyHallSetup}, you 
very likely found that always switching yielded significantly better results than always sticking
with your original choice.  Here is the explanation.  

If you stick with your original
choice, then everything after that first choice, including Monty revealing one of the 
goats, is actually \emph{irrelevant}(!)  Your 
probability of winning the car is exactly $1/3$ because only one of the three doors
has the car.  

On the other hand, if you always switch, then something very interesting happens.
If you chose the door with the car to begin with, then you will lose because 
you switch to the door with the other goat (the one Monty did not show).  \emph{However},
if the door you chose originally had a goat, then \emph{you always win the car by switching}
because Monty showed you the other goat and the remaining door must be the one with the car.
Moreover, since there were two goats out of the three choices, the probability 
that you chose a goat to begin with was $2/3$(!)  We can say this formally using
the language of conditional probabilities and \eqref{eq:condprobdefrearranged} as follows.
Let $A$ be the event that you win the car and $B_1$ be the event that you 
choose the car to start, while $B_2$ is the event that you choose one of the two
goats to start.  Note that $B_1 \cup B_2 = S$, the whole sample space for
your original choice.  Moreover $B_1 \cap B_2 = \emptyset$.  By the result
of Exercise~\ref{ex:AdditionLaw}, if you stick
to your original choice, then
\begin{align*}
P(A) &= P(A \cap B_1) + P(A \cap B_2)\\
     &= P(A|B_1)P(B_1) + P(A|B_2)P(B_2)\\
     &= 1\cdot (1/3) + 0\cdot (2/3)\\
     &= 1/3.
\end{align*}

On the other hand, under the 
``always switch'' strategy, 
\begin{align*}
P(A) &= P(A \cap B_1) + P(A \cap B_2)\\
     &= P(A|B_1)P(B_1) + P(A|B_2)P(B_2)\\
     &= 0\cdot (1/3) + 1\cdot (2/3)\\
     &= 2/3.
\end{align*}
An interesting sidelight to this example is that essentially this analysis 
was publicized by Marilyn Vos Savant in her \emph{Parade} magazine 
\index{Vos Savant, Marilyn}
column ``Ask Marilyn'' in several articles in 1990 and 1991.  A number of
professional mathematicians wrote her to chide her for missing the ``obvious''
observation that there should be a $50-50$ chance of winning the game 
after Monty revealed the first goat--switching or not should make no
difference.  \emph{They were wrong!}  Conditional probabilities can be tricky
and counterintuitive.
\index{Monty Hall problem|)}
$\triangle$
\end{example}

The computations we did in Example~\ref{MontyHallAnalysis} are special cases 
of an important result called
\index{Law of Total Probability}
the \emph{Law of Total Probability}.  Suppose we have a sample space $S$
that is the union of pairwise disjoint subsets $S = S_1 \cup \cdots \cup S_n$.
If $A$ is any event in $S$, then by Assumptions~\ref{probs} and \eqref{eq:condprobdefrearranged},
\begin{align*}
P(A) &= P(A\cap S_1) + \cdots + P(A \cap S_n)\\
     &= P(A|S_1)P(S_1) + \cdots + P(A|S_n)P(S_n).
\end{align*}

\subsection*{Independence}
\index{independence|(}
We now come to a central concept in probability, namely what it means to say
that two events are \emph{independent}.  The correct intuition here is that
an event $A$ and an event $B$ are independent exactly when knowing that 
$B$ occurred \emph{has no effect on the probability of $A$}.  In the form of 
an equation, we say $A,B$ are independent exactly when
$$P(A|B) = P(A).$$
Since $P(A|B) = \frac{P(A\cap B)}{P(B)}$ from \eqref{eq:condprobdef}, this
condition can also be rewritten in the following form
\begin{equation}
\label{eq:indep}
\framebox{$P(A \cap B) = P(A)P(B)$.}
\end{equation}
From this we see that if $P(A) \ne 0$, then by dividing by $P(A)$ the independence relation has a symmetric nature:
$$P(A|B) = P(A) \Leftrightarrow P(B|A) = P(B) \Leftrightarrow P(A \cap B) = P(A)P(B).$$
Note that saying $A$ and $B$ are independent is \emph{almost always different from} saying
that $A \cap B = \emptyset$. Here is a simple example.

\begin{example}
\label{indepex}
Consider one throw of the two distinguishable fair dice from Example~\ref{SimpleDice} above.  
Let $A$ be the event that the first die shows an even number and let $B$ be the
event that the second die shows an odd number.  We claim that $A$ and $B$ are independent.
Since the even numbers are half of the possibilities for the first die while
the second die can show any number in $A$, we have $P(A) = 18/36 = 1/2$.
Similarly, $P(B) = 1/2$.  The event $A\cap B$ consists of the pairs
$$\{(2,1),(2,3),(2,5),(4,1),(4,3),(4,5),(6,1),(6,3),(6,5)\}.$$
Hence $P(A \cap B) = 9/36 = 1/4 = P(A)P(B)$.  This shows that $A$ and $B$
are independent.  $\triangle$.  
\end{example}
\index{probability!conditional|)}
\index{conditional probability|)}
\index{independence|)}

\specialsection{Discrete Distributions and Probability Mass Functions}
\index{random variable}
\index{sample space}
\index{distribution!discrete}
\index{countably infinite}
\index{discrete distribution}
\index{discrete random variable}
\index{random variable!discrete}
A random variable can be thought of as a \emph{function defined on a sample space}.  
We will begin by studying several commonly-encountered \emph{discrete} random variables 
and their probability distributions.  In these cases, the sample space is finite, or 
perhaps countably infinite.\footnote{Technical Note: A countably infinite set is one that
can be put into one-to-one correspondence with the natural numbers, 
$\mathbb{N} =\{1,2,3,\ldots\}$.  A celebrated result of the 19th century German 
mathematician Georg Cantor shows that there
\index{Cantor, Georg}
\index{uncountably infinite}
are also infinite sets that are not countably infinite.  These are known 
as \emph{uncountably infinite} sets.  In particular, the interval $[0,1]$ in 
the real numbers is uncountably infinite.  The proof that establishes this result 
is sketched in the (optional) Exercise~\ref{ex:CantorDiagonal} below.}

\subsection*{Binomial and Hypergeometric Random Variables}  One of the simplest examples 
\index{Bernoulli trial}
\index{random variable!Bernoulli trial}
\index{distribution!Bernoulli trial}
begins with a sample space consisting of just two elements, often denoted $1$ and $0$ or 
``success'' and ``failure.''\footnote{The ``success'' and ``failure''
language is merely a convention.  You should not necessarily associate a ``success'' in 
this sense with a ``good'' outcome
or a ``failure'' with a ``bad'' one.}  A \emph{Bernoulli trial} is a random variable $X$ that models the process of performing
an experiment with two possible outcomes, each occurring with \emph{fixed probabilities}.  The probability of the 
``success'' outcome is $P(X = 1) = p$ for some real number $0 \le p \le 1$, and then the probability of the other outcome
is $P(X = 0) = 1 - p$.  This complementary ``failure'' probability is often denoted by $q = 1 - p$.  

\index{binomial experiment}
We use Bernoulli trials to define what is known as a \emph{binomial experiment}.  
A binomial experiment is one in which:
\begin{enumerate}
\item[(1)]  there are a fixed number, $n$, of Bernoulli trials, all with the same 
success probability $p$,
\item[(2)]  the trials are \emph{independent} in the sense of \eqref{eq:indep}--outcome
of one trial has no effect on the outcomes of the others.   
\item[(3)]  the random variable of interest is $X = $ the \emph{number of successes}.
\end{enumerate}
If these conditions are satisfied,\footnote{possibly after interpreting a problem to put it into
this framework} then  we say $X$ is a \emph{binomial random variable}.  
\index{random variable!binomial|(}
\index{binomial distribution|(}
\index{distribution!binomial|(}

\begin{example}
\label{urn}
A standard example of a binomial experiment given in many traditional textbooks on probability
theory consists of the following:  Imagine we have a large urn filled with balls of two colors -- say red and 
black.  Each trial consists of  drawing one ball at random from the urn and recording its color.\footnote{You are
supposed to visualize reaching into the urn to select a ball without being able to see any of the colors before
the ball is pulled out!}  After
each trial, the ball is \emph{replaced in the urn} and the contents are mixed thoroughly.   At the end, the 
number of red balls chosen in the $n$ trials is totaled.  
\index{with replacement}

Because the ball
is replaced after each trial, the numbers of balls of each color in the urn stay the same on each 
of the trials.  If there are $n_1$ red balls and $n_2$ black balls, and we associate choosing
red ball with the ``success'' outcome, this means that the probability $p$ of choosing a 
red ball and the complementary probability of choosing a black ball will be 
$$p = \frac{n_1}{n_1 + n_2} \qquad {\rm and}  \qquad  q = \frac{n_2}{n_1+n_2}$$
on each trial.  

The \emph{independence} referred
\index{independence}
to above consists in the fact that the outcome of any one of the trials should have no influence on
the outcomes of the others.  This is ensured by the fact that the ball selected on any one trial
is coming from the same collection of balls and the random selection should mean that the previous
choices have no influence on the later choices.  In more precise terms, what this means is
that the probabilities of particular outcomes on the successive trials are \emph{multiplied} to 
obtain the probability of a sequence of trials. 
That is, the probability of drawing red, black, red, red on four successive draws is
$$p\cdot q\cdot p \cdot p = p^3 q$$
under the assumption that the trials are independent (using \eqref{eq:indep}).

The random variable of interest in a binomial experiment is the number of successes.
However, different sequences of draws will lead to a given number of successes 
in most cases.  For instance, with $n = 4$ trials we could end up with three red balls
with any of the following sequences of outcomes of the individual trials:
\begin{align*}
{\rm red,\ red,\ red,\ black}\\
{\rm red,\ red,\ black,\ red}\\
{\rm red,\ black,\ red,\ red}\\
{\rm black,\ red,\ red,\ red}
\end{align*}
These are distinct, disjoint outcomes so the total probability of getting
$k =3$ reds in drawing $n = 4$ balls from the urn would be 
$$p^3q + p^3q + p^3q + p^3q = 4p^3q.$$
We can think of the coefficient $4$ here as the total number of ways to distribute
the three reds over the four trials.  $\triangle$
\end{example}

In general, now, if we have a binomial experiment with $n$ trials, the success probability 
$= p$, and we want the probability of $k$ successes, then we need to have
a way to count the number of ways to distribute the $k$ successes over the 
$n$ trials.  It is not difficult to see that this is \emph{the same as the coefficient
of $p^k q^{n-k}$ in the binomial expansion}:
\index{binomial expansion}
\index{binomial coefficient}
$$(p + q)^n = \sum_{k=0}^n \binom{n}{k} p^k q^{n-k}.$$
The reason, of course, is that in expanding out the power $(p+q)^n$, 
after rearranging the products to put $p$'s before $q$'s, the terms of the form $p^k q^{n-k}$ 
will come from exactly the same sort of selection of $k$ of the $p$ terms in 
the factors and $n - k$ of the $q$ terms.  The number 
$\binom{n}{k}$ multiplying the $p^k q^{n-k}$ term in the expansion
is called the \emph{binomial coefficient} and 
is often read as ``$n$ choose $k$.''\footnote{Technical Note:  If you have learned about 
combinations in high school, this is the same as the number of combinations
of $n$ things taken $k$ at a time.}  Fortunately, there is a convenient
formula for $\binom{n}{k}$ using the idea of the \emph{factorial} of
\index{factorial}
an integer.  Recall that if $n$ is a positive integer
$$n! = n\cdot (n-1)\cdot (n-2) \cdots 3\cdot 2\cdot 1.$$
So, for example, we have
\begin{align*}
1! &= 1\\
2! &= 2\cdot 1 = 2\\
3! &= 3\cdot 2\cdot 1 = 6\\
4! &= 4\cdot 3! = 24\\
5! &= 5\cdot 4! = 120
\end{align*}
and so forth.  By convention, $0! = 1$.  The formula for the binomial
coefficient is 
\begin{equation}
\label{eq:BC}
\framebox{$\displaystyle\binom{n}{k} = \frac{n!}{k!(n-k)!}.$}
\end{equation}
For example $\binom{4}{3} = \frac{4!}{3! 1!} = \frac{24}{6} = 4$, as we saw 
at the end of Example~\ref{urn}.  Note that we can also cancel the factors
in $(n-k)!$ between the numerator and denominator, yielding the following
somewhat more efficient formula 
$$\binom{n}{k} = \frac{n(n-1)\cdots (n - k + 1)}{k!}.$$

Putting everything together, we have what is called the 
\index{probability mass function (p.m.f.)}
\emph{probability mass function} for a binomial random variable $X$:
\begin{equation}
\label{eq:binpmf}
\framebox{$\displaystyle P(X = k) = \binom{n}{k} p^k q^{n-k}$.}
\end{equation}
The values where $P(X = k)$ would be nonzero are $k = 0, 1, \ldots, n$. 
See Exercise~\ref{ex:pmfs} below for some general properties of 
this and other probability mass functions.

\begin{example}
\label{surgery}
A new surgical procedure has a success probability of $p$ each time it is performed.  
Here each individual surgery done with the procedure constitutes a Bernoulli
trial, and under suitable assumptions (what assumptions? --  see below) we can apply the 
binomial probability formulas.  
\begin{enumerate}
\item If $p = .8$ and the surgery is performed 5 times, what is the probability
that the surgery will succeed for exactly 4 patients?   With $p = .8$, $q = 1 - .8 = .2$.
This is the binomial
probability
$$P(X = 4) = \binom{5}{4} (.8)^4 (.2) =.4096.$$
\item If $p = .7$, what is the probability that the surgery will succeed
for at least two patients?  With $p = .7$, $q = 1 - .7 = .3$.  The direct way to compute that binomial
probability would be 
$$P(X \ge 2) = P(X = 2) + P(X = 3) + P(X = 4) + P(X = 5).$$
This would yield the value 
$$\binom{5}{2}(.7)^2(.3)^3 + \binom{5}{3}(.7)^3(.3)^2 + \binom{5}{4}(.7)^4(.3) + \binom{5}{2}(.7)^5.$$
A somewhat easier, but equivalent, computation would be to take the complementary probability
that the surgery is successful 0 or 1 times and subtract that from 1:
\begin{align*}
P(X \ge 2) &= 1 - P(X < 2)\\ 
&= 1 - (P(X = 0) + P(X = 1))\\
&=  1 - \left(\binom{5}{0}(.3)^5 + \binom{5}{1}(.7)(.3)^4\right)\\ 
&\doteq .9692. 
\end{align*}
\end{enumerate}
We are assuming that the outcomes of different operations are independent in order to apply the 
binomial probability formulas in this way.  $\triangle$
\end{example}

\begin{example}
\label{trout}
Suppose we have a large tank containing $334$ male trout and $275$ female
trout.  We pick $10$ of the fish at random one at a time, and replace them in the tank
after determining the sex.  We ask: what is the probability that exactly 6 of the fish selected
were females?  

This is exactly like Example~\ref{urn} with the red balls corresponding to the females
and the black balls to the males.   So it can be modeled as a binomial experiment with $n = 10$, and 
$p = \frac{275}{275+334} = \frac{275}{609} \doteq .4516$.  If $X$ is the 
number of females selected, using the more efficient cancelled formula we have
$$\binom{10}{6} = \frac{10\cdot 9\cdot 8\cdot 7 \cdot 6 \cdot 5}{6\cdot 5 \cdot 4\cdot 3 \cdot 2 \cdot 1} = \frac{10\cdot 9\cdot 8 \cdot 7}{4\cdot 3\cdot 2\cdot 1} = 210$$
and then 
$$P(X = 6) = \binom{10}{6}(.4516)^6 (.5484)^4 = 210 \cdot .008482\cdot .09045 \doteq .1611.$$
That is, there is about a $16\%$ chance of this happening -- four females is the most likely single outcome because of the imbalance in
numbers.  
\begin{figure}
\begin{center}
\includegraphics[width=2.0in,height=2.0in]{ProbHist.png}
\caption{Binomial probabilities, $n = 10$, $p \doteq .4516$.}
\label{fig:TroutPMF}
\end{center}
\end{figure}

We can also represent the probability mass function values for all integers $k$, $0 \le k\le 10$, with a bar chart like the one in Figure~\ref{fig:TroutPMF}.  
The bars extend from $k - \frac{1}{2}$ to $k + \frac{1}{2}$ so their widths are equal to 1 and they are centered on the integer
$k$.  The sum of the heights of the bars is 1 and hence the total area of the bars sums to $1$.  This is an example of a 
\index{probability histogram}
\index{histogram}
\emph{probability histogram}.  There is a corresponding probability histogram for each $n$ and each choice of $p$.  When $p \ne .5$, the 
histogram will not be symmetric. $\triangle$
\end{example}
\index{random variable!binomial|)}
\index{binomial distribution|)}
\index{distribution!binomial|)}

\index{without replacement}
\index{hypergeometric distribution}
\index{distribution!hypergeometric}
\index{random variable!hypergeometric}
In Example~\ref{trout}, note that we were careful to say that the fish were replaced in the 
tank after each one was selected.  But that probably seems like an unreasonable way to 
carry out the process of selecting 10 of the fish because we could be selecting the same 
fish more than once.    If we did not do the selection with replacement, 
then the process would be approximately, but
not exactly, a binomial experiment.  The reason is that, for instance, after 
selecting one of the female fish, the remaining numbers in the tank are 274 females and 
334 males. Hence the probability of
selecting a female the next time would be $\frac{274}{608}$, which is slightly smaller 
than $\frac{275}{609}$.  
Similarly, the probability of selecting a male would be $\frac{334}{608}$ which is slightly larger than 
$\frac{334}{609}$.  There is another family of discrete random variables known as the \emph{hypergeometric}
family that models selections done \emph{without replacement}.  The hypergeometric p.m.f. would 
yield the value
$$P(Y = 6) = \frac{\binom{275}{6}\cdot\binom{334}{4}}{\binom{609}{10}} \doteq  .1613.$$ 
This is close to, but not exactly the same as, the probability $.1611$ we obtained from the binomial
experiment for the probability of selecting 6 females out of the 10 fish.  

To write the hypergeometric p.m.f. in general, we will use the language from Example~\ref{urn}.  If 
we draw $n$ balls \emph{without replacement} from an urn containing $N$ balls, of which $n_1$ are red
and $n_2$ are black, then the probability that we obtain exactly $k$ red balls is given by 
\begin{equation}
\label{eq:hypergeom}
\framebox{$\displaystyle P(k\ {\rm red\ balls}) = \frac{\binom{n_1}{k}\cdot \binom{n_2}{n - k}}{\binom{N}{n}}$.}
\end{equation}
The numerator counts the total number of ways of picking $k$ of the red balls and $n - k$ of the white balls.
The denominator is the total number of ways to pick $n$ balls from the $N$ in the urn.  Hence the
formula follows from the basic ratio methods of finding theoretical probabilities from Chapter 2.
It can be shown that this is close to the binomial probability with $p = \frac{n_1}{N}$ and $q = \frac{n_2}{N}$ when $N, n_1, n_2$ are all large
(as was true for instance in the example discussed above).  However, when $N$ is small, this can differ
significantly from the binomial probability.  

\subsection*{Geometric Distributions and Probability Mass Functions}
\index{geometric distribution|(}
\index{distribution!geometric|(}
\index{random variable!geometric}
Now suppose we are in a similar situation to the binomial experiments described above, but with two important
differences.  Namely, 
\begin{enumerate}
\item[(1)]  we do Bernoulli trials, all with the same success probability $p$, but the number of trials is no longer fixed,
\index{Bernoulli trial}
\item[(2)]  the trials are \emph{independent} in the sense discussed above, and
\item[(3)]  the random variable of interest is $Y = $ the \emph{number of the trial on which the first success is observed}.
\end{enumerate}
In this situation, we have $Y = k$ exactly when trials $1,2,\ldots, k - 1$ produced failure outcomes, but trial number $k$
produced the first success.  Hence by the independence of the individual trials, we see the probability
is a product of $k -1$ factors of $q$ and one final factor of $p$:
\begin{equation}
\label{eq:geometric}
\framebox{$P(Y = k) = q \cdot q \cdots q \cdot p = q^{k-1} p$ for all $k \ge 1$.}
\end{equation}
Random variables having a probability mass function of the form given in \eqref{eq:geometric} are said
to have a \emph{geometric} distribution.   Unlike binomial random variables, geometric random variables
have $P(Y = k)$ nonzero for all $k \ge 1$.  Even for very large $k$, there is a small but strictly positive chance
that the first success occurs on trial $k$.  

The reason for the name ``geometric random variables'' becomes somewhat clearer if we think 
of summing the probabilities in \eqref{eq:geometric} for $k \ge 1$:
\begin{equation}
\label{eq:GeomTotal}
\sum_{k=1}^\infty q^{k-1} p = p + pq + pq^2 + pq^3 + \cdots\ .
\end{equation}
This is an (infinite) geometric series, 
\index{infinite geometric series} 
that is, a series of the form
$$\sum_{n = 0}^\infty a r^n = a + ar + ar^2 + ar^3 + \cdots .$$
When $-1 < r < 1$ (or equivalently $|r|<1$), this sum
converges to (that is, ``adds up to'')\footnote{See the (optional) Exercise~\ref{ex:InfGeomSeries}
below for the derivation of this fact.} the sum given in this formula:
\begin{equation}
\label{eq:GeomSeriesSum}
\framebox{$\displaystyle \sum_{n = 0}^\infty a r^n = \frac{a}{1 - r}$.}
\end{equation}
When $|r| \ge 1$, the series does not add up to a finite sum.
Hence the series in \eqref{eq:GeomTotal}, where $a = p$ and $r = q$, adds up to the value 
$$\frac{p}{1 - q} = \frac{p}{p} = 1,$$ 
and we obtain the sum $1$ as we expect.   We have covered \emph{all of the 
possible outcomes} by including the terms with $k = 1,2,3,\ldots\ $, so we should obtain $1$ by summing all of
those probabilities.  

\begin{example}
\label{ex:geoex}
Suppose that a prey organism lives in a habitat with predators, and each prey individual has a $10\%$ chance
of being killed by a predator or dying from other causes each month.  We will assume the prey individuals are solitary,
so that the fate of one of the animals has no effect on the fates of the others.\footnote{This is actually 
rarely true in real life, of course -- prey animals tend to live in larger groups for ``safety in numbers.''}  We will also 
assume that whether an animal survives the predation in one month is independent of whether they survive in other months.  Then
we can model their survival probabilities using geometric random variables.  Here the ``success'' probability
as we have described it would actually correspond to the ``bad'' outcome of being eaten by a predator or dying from other causes, 
so our previous caveat about interpreting the words ``success'' and ``failure'' needs to be taken seriously.\footnote{Or alternatively,
we might look at things from the point of view of the predators!}
\begin{enumerate}
\item[(a)]  What is the probability that a prey individual survives $4$ months before being eaten in the month $5$?  This is 
$P(Y = 5) = (.9)^4\cdot (.1) = .06561$, or less than a $7\%$ chance.  
\item[(b)]  What is the probability that a prey individual survives \emph{at least four months}? This means that we want
$$P(Y \ge 5) = \sum_{k = 5}^\infty P(Y = k) = (.9)^4\cdot (.1) + (.9)^5\cdot (.1) + (.9)^6\cdot (.1) + \cdots .$$
We see that this is another infinite geometric series with ratio $r = .1$, but now the first term is $(.9)^4\cdot (.1)$.  
Hence, by the formula from Exercise~\ref{eq:GeomSeriesSum}, the sum is
$$= \frac{(.9)^4\cdot (.1)}{1 - .9} =.6561,$$
or roughly a $66\%$ chance.  $\triangle$
\end{enumerate}
\end{example}
\index{geometric distribution|)}
\index{distribution!geometric|)}

\subsection*{Poisson Distributions and Probability Mass Functions}
\index{Poisson distribution|(}
\index{distribution!Poisson|(}
\index{random variable!Poisson}
Binomial random variables count the number of ``successes'' in a fixed number of independent Bernoulli trials and geometric
random variables deal with the number of the trial on which the first success is observed.   The final type of discrete random
variable we will consider, the so-called \emph{Poisson random variables}, are used to model \emph{the probabilities of different numbers of 
occurrences of a particular event within a fixed region in time or space}.   Here is a typical example of the type of problem
that leads to Poisson random variables.

\begin{example}
\label{PoissonEx}
Suppose that a forester knows that maple seedlings are randomly dispersed through a certain area of the 
forest, with an average density of 2 seedlings per square meter.  If the forester marks
out five one meter square plots in the forest, what is the probability that \emph{none} of the plots
contains a maple seedling?  Similarly, what is the probability that the five plots together contain
exactly two seedlings? $\triangle$
\end{example}

A discrete random variable $Y$  is said to have a \emph{Poisson distribution} if its p.m.f. has the form 
\begin{equation}
\label{eq:Poisson}
\framebox{$\displaystyle P(Y = k) = \frac{\lambda^k}{k!} e^{-\lambda}$,}
\end{equation}
where $\lambda > 0$ is a constant representing the \emph{average value} of $Y$ (in a sense to be made precise in
the next section), and $e^{-\lambda}$ is a value of the natural exponential function introduced in Chapter 5,
for which the base is the number $e \doteq 2.71828\cdots $.  
This formula for the Poisson p.m.f. can be derived from the binomial p.m.f. by a sort of limiting
process (subdividing the interval in time or space into subintervals small enough that either 
$1$ or $0$ events occur).  We will not discuss the details of this derivation in the main text, 
though, since they rely on some results typically only established in calculus courses.  
See the (optional) Exercise~\ref{ex:PoissonDerivation} below.

\begin{example}
\label{PoissonExCont}
We continue with the question introduced in Example~\ref{PoissonEx}.
Here it is the number of seedlings in a one meter square area that would be the random variable $Y$
with a Poisson distribution.  The parameter $\lambda = 2$, since we are given that the average density of 
maple seedlings is $2$ seedlings per square meter.   The probability mass function \emph{for the number
of seedllings in a one square meter area} is then
$$P(Y = k) = \frac{2^k}{k!}e^{-2}$$ 
for all $k \ge 0$.  The Poisson random variables are like geometric random variables in that there
is a strictly positive probability that $Y$ has each possible value $k \ge 0$, although those
probabilities drop off quickly as $k$ increases.  See the Poisson probability histogram plotted
\index{probability histogram}
\index{histogram}
in Figure~\ref{fig:Poisson}.
\begin{figure}
\begin{center}
\includegraphics[width=2.0in,height=2.0in]{PoissonHist.png}
\caption{Poisson probabilities, $\lambda = 2$.}
\label{fig:Poisson}
\end{center}
\end{figure}
The probability that a one meter square plot contains no seedlings is 
$$P(Y = 0) = \frac{2^0}{0!} e^{-2} \doteq .1353.$$
Hence the probability that none of the five plots contains a seedling
is (assuming independence) $(.1353)^5 \doteq 4.54 \times 10^{-5}$
(in other words, virtually no chance of that happening!)  

The second part of the question was to determine what the probability
was that the 5 one meter square plots contain exactly two seedlings
between them.  Note that this could happen in several ways.
One of the plots could contain two seedlings, while the other
four had none.  Since any one of the five plots could be the
one containing the two seedlings, the probability of that happening
would be 
$$5\cdot P(Y = 2) \cdot P(Y = 0)^4 = 5 \cdot \frac{2^2}{2!} e^{-2} \cdot \left(\frac{2^0}{0!} e^{-2}\right)^4 \doteq .000454.$$
Next, there are $\binom{5}{2} = 10$ ways for 2 plots to have one seedling each and the others to have none.  This 
gives a contribution of 
$$10\cdot P(Y = 1)^2 \cdot P(Y=0)^3 =10\cdot \left(\frac{2^1}{1!} e^{-2}\right)^2 \cdot \left(\frac{2^0}{0!} e^{-2}\right)^3 \doteq .001816.$$
Adding, we obtain approximately 
\begin{equation}
\label{eq:PoissonAnswer}
{\rm Probability\ of\ two\ seedlings} = .000454 + .001816 = .002270
\end{equation}
(about a $0.2\%$ chance -- we would not expect to see this often!).  

Before leaving this example, we want to make another observation about what is happening here.  
Note that if the 5 one meter square plots were \emph{contiguous}, then we might immediately
notice that
the average number of seedlings in the $5$ square meters would be $5\cdot 2 = 10$.   
But that is true in any case for 
\emph{any} region consisting of $5$ square meters of the forest.  Moreover, the Poisson formula
with $\lambda = 10$ would give 
$$P(Y = 2) =  \frac{10^2}{2!}e^{-10} \doteq .002270,$$
which is exactly the same number as in \eqref{eq:PoissonAnswer} above before and after rounding.  In fact, \emph{the Poisson p.m.f. 
formula always applies in exactly the same way, whether the region in space or time is contiguous 
or not.}  We only need to know the appropriate value of the parameter $\lambda$ for the whole region in order
to apply it. $\triangle$
\end{example}
\index{Poisson distribution|)}
\index{distribution!Poisson|)}

\specialsection{Expected Value and Variance}
\index{expected value|(}
In the previous chapter we discussed computing the mean and the standard deviation of a collection of numerical data. 
In this section we will turn to the corresponding concepts for discrete random variables described by probability 
mass functions (p.m.f.'s) as in the last section.   The concept corresponding to the mean is usually called the \emph{expected value}
of a random variable, and the concept corresponding to the standard deviation goes by that name in this context too.  But
in fact we will begin by showing how to compute the \emph{variance} of a discrete random variable and the standard deviation
will be obtained by taking the square root of the variance.  

\subsection*{Expected Values}
The expected value of a discrete random variable is computed as a \emph{weighted average}
\index{weighted average}
of the values.  The weights are the associated probabilities or p.m.f. values.  Hence we can write a
general formula for computing the expected value in the following form, when it is true that
the values are a subset of the integers $\ge 0$ (i.e. not just some set in one-to-one correspondence
with the natural numbers).  The notation we will
use for this is $E(Y)$:
\begin{equation}
\label{eq:EVDef}
\framebox{$E(Y) = \sum_{k=0}^\infty k \cdot P(Y = k)$.}
\end{equation}

If $P(Y = k)$ is zero except for a finite collection of $k$ values, this is 
an ordinary finite sum.  In other cases, it will be an infinite series
and some care is needed to determine whether the sum converges
to a finite result.  
To see what this really means, we will work out a few examples.

\begin{example}
\label{lottery}
As is true for many of the concepts in probability theory, the idea of the expected value
goes back to the analysis of outcomes in the \emph{games of chance} that people have 
played in most cultures throughout history.\footnote{The mathematical treatment of 
probabilities in games of chance goes back to works of Blaise Pascal, Ren\'e
Descartes and others. Jacob Bernoulli's \emph{Ars Conjectandi}, 
\index{Bernoulli, Jacob}
\index{Pascal, Blaise}
\index{Descartes, Ren\'e}
first published in 1713, put the theory in something close to its modern form.  Bernoulli
trials are named after him.} 
For instance, suppose you play a simple
lottery game where you buy a $\$1$ ticket along with 999 other people (1000 total
tickets sold).  There are five $\$10$ prizes and one $\$100$ grand prize.  We
could ask:  What is our expected return from the $\$1$ purchase?  

Assume the grand prize winner is drawn first and the five other winners are
chosen at random (without replacement) from the remaining $999$ people. 
By \eqref{eq:hypergeom}, if you do not win the grand prize, you still have
probability of 
$$\frac{\binom{1}{1} \cdot \binom{998}{4}}{\binom{999}{5}} = \frac{5}{999}$$
of winning one of the other prizes.

You have a random variable $R = $ your return in dollars from the lottery with 
\begin{align*}
P(R = 100) &= \frac{1}{1000} = .001,\\
P(R=10) &= \frac{5}{999} \doteq .005, \ {\rm and}\\
P(R = 0) &= 1 - \frac{1}{1000} - \frac{5}{999} \doteq .994.
\end{align*}
According to \eqref{eq:EVDef}, your expected return is
$$E(R) \doteq 0\cdot .994 + 10\cdot .005 + 100\cdot .001 \doteq .15$$
or about $\$0.15$.\footnote{The organizations that run lotteries (often state governments, charitable organizations
such as churches, etc.) are not in that business out of the kindness of their hearts or for the entertainment of those
who gamble on winning;  they are usually 
doing it to \emph{make money}.}  $\triangle$
\end{example}

\begin{example}
\label{BinEV}
\index{binomial distribution!expected value}
\index{distribution!binomial}
Consider the binomial random variable $X$ with $n = 5$ and $p = \frac{1}{3}$.  
\index{random variable!binomial}
\index{probability mass function (p.m.f.)}
The values of the p.m.f. 
are given in Table~\ref{PMFTable}.
\begin{table}[ht]
\caption{A binomial p.m.f. with $n = 5$ and $p = \frac{1}{3}$ }\label{PMFTable}
\renewcommand\arraystretch{2.2}
\noindent\[
\begin{array}{|c|c|c|c|c|c|c|}
\hline
k & 0 & 1 & 2 & 3 & 4 & 5\\
\hline
P(Y = k) & \frac{32}{243} & \frac{80}{243} &\frac{80}{243} &\frac{40}{243} & \frac{10}{243} & \frac{1}{243}\\
\hline
\end{array}
\]
\end{table}
According to the formula \eqref{eq:EVDef} above, we want to compute 
\begin{align*}
E(Y) &= 0\cdot P(Y = 0) + 1\cdot P(Y=1) + 2\cdot P(Y = 2) + 3\cdot P(Y=3)\\
        & \qquad  + 4\cdot P(Y=4) + 5\cdot P(Y=5)\\
&= 0\cdot \frac{32}{243} + 1\cdot \frac{80}{243} + 2\cdot \frac{80}{243} + 3\cdot \frac{40}{243} \\
       &\qquad + 4\cdot \frac{10}{243} + 5\cdot \frac{1}{243}\\
&= \frac{80 + 160 + 120 + 40 + 5}{243}\\
&= \frac{405}{243}\\
&= \frac{5}{3},
\end{align*}
since $405 = 5\cdot 81$ and $243 = 3\cdot 81$. 

In hindsight, this result should look almost ``obvious'' -- we are
doing five identical Bernoulli trials with a success probability of $p = \frac{1}{3}$ on each trial.  So the expected
value of the number of successes could hardly be anything other than $n p = \frac{5}{3}$(!)  $\triangle$
\end{example}

The same pattern holds in general.

\begin{theorem}  Let $Y$ be a binomial random variable based on $n$ trials, with success probability $p$.  Then
$E(Y) = np$.
\end{theorem}

The proof consists of some clever manipulations of the formula
$$E(Y) = \sum_{k=0}^n \left( k\cdot \binom{n}{k} p^k q^{n-k}\right),$$
but we will not go into the details here because the intuition gained through the interpretation given 
at the end of Example~\ref{BinEV} is more important.  

\begin{example}
\label{GeomEV}
\index{random variable!geometric}
To find the expected value for a geometric random variable using \eqref{eq:EVDef}, we need to sum
$$E(Y) = \sum_{k = 1}^\infty k \cdot q^{k-1} p = p(1 + 2q + 3q^2 + 4q^3 + \cdots ).$$
We can identify the sum in the parentheses if we think of breaking it up into several 
geometric series, summed vertically then horizontally as in the following diagram:
\[
\begin{array}{ccccccc}
1 &+q &+q^2 &+q^3 &+q^4 \\
   &+q &+q^2 &+q^3 &+q^4 \\
   &     &+q^2 &+q^3 &+q^4 \\ 
   &     &         &+q^3 &+q^4 \\
   &     &         &  & \ddots 
\end{array}
\]
If we reorder the sum (this is permissible in this case provided $0 < q < 1$) by adding across each row, then adding
the row sums, we can apply \eqref{eq:GeomSeriesSum} to obtain
$$\frac{1}{1 - q} + \frac{q}{1 - q} + \frac{q^2}{1 - q} + \cdots = \frac{1 + q + q^2 + \cdots }{1 - q} = \frac{1}{(1 - q)^2},$$
using \eqref{eq:GeomSeriesSum} once more.  Hence the formula for the expected value of the geometric
random variable $Y$ is 
$$E(Y) = \frac{p}{(1 - q)^2}  = \frac{1}{p}$$
since $1 - q = p$.   Again, this should look ``almost obvious'' with the benefit of hindsight.  If
we have a success probability of $p$, then it should take $\frac{1}{p}$ trials on average before
we get a success.  For instance, with $p = \frac{1}{5}$, then we would expect roughly one success
every $5$ trials, so on average, we might expect it to take $5$ trials to get the first success. 
$\triangle$
\end{example}
 
\begin{theorem} Let $Y$ be a geometric random variable with success probability $p$.  Then 
$E(Y) = \frac{1}{p}$.  
\index{geometric distribution!expected value}
\index{distribution!geometric}
\end{theorem}

Finally we consider the Poisson random variables introduced in section 10.2.  For a Poisson
\index{random variable!Poisson}
variable with parameter $\lambda$, from \eqref{eq:EVDef} we obtain
$$E(Y) = \sum_{k = 0}^\infty k \cdot \frac{\lambda^k}{k!} e^{-\lambda}.$$
By canceling the $k$ in the numerator with one of the factors in the $k!$ in the
denominator, this can be rewritten as 
$$ = \lambda \cdot \sum_{k=1}^\infty \frac{\lambda^{k-1}}{(k-1)!} e^{-\lambda}.$$
If we rewrite the sum using a new index $\ell = k - 1$, then we obtain
$$ = \lambda \cdot \left(\sum_{\ell = 0}^\infty \frac{\lambda^\ell}{\ell!} e^{-\lambda}\right).$$
Now the sum in the parentheses on the right is just the sum of the values of the p.m.f. of the 
Poisson random variable.  \emph{Hence it must equal $1$}.\footnote{Technical note:  This fact can also 
be seen from the fact that the sum $ \sum_{\ell = 0}^\infty \frac{\lambda^\ell}{\ell!}$
is the Taylor series for the exponential $e^\lambda$, so we have $e^\lambda \cdot e^{-\lambda} = 1$.}
As a result, we see:

\begin{theorem}  Let $Y$ be a Poisson random variable with parameter $\lambda$.  Then
$E(Y) = \lambda$.
\index{Poisson distribution!expected value}
\index{distribution!Poisson}
\end{theorem}

This agrees with the way we described the meaning of the parameter $\lambda$
when we introduced the Poisson random variables in the last section.
\index{expected value|)}

\subsection*{Variances}
\index{variance|(}
The variance of a random variable is perhaps best understood as the expected
value of a certain \emph{function of the random variable}.  Namely we consider
the weighted average of the \emph{square deviation from the expected value} (as in the discussion
of the formula for the standard deviation from the last chapter), again weighted
by the p.m.f.  This defines the variance of a discrete random variable
with values in the non-negative integers, denoted $V(Y)$:
\begin{equation}
\label{eq:VarDef}
\framebox{$V(Y) = E((Y - E(Y))^2) = \sum_{k = 0}^\infty (k - E(Y))^2 P(Y = k)$.}
\end{equation}

\begin{example}
\label{BinVar}
Consider the binomial random variable with p.m.f. given in Table~\ref{PMFTable}.
According to \eqref{eq:VarDef}, the variance will be
\begin{align*}
V(Y) &= \left(0 - \frac{5}{3}\right)^2 \cdot P(Y = 0) +  \left(1 - \frac{5}{3}\right)^2\cdot P(Y=1) +  \left(2 - \frac{5}{3}\right)^2\cdot P(Y = 2)\\
        &\qquad +  \left(3 - \frac{5}{3}\right)^2\cdot P(Y=3) +  \left(4 - \frac{5}{3}\right)^2\cdot P(Y=4) + \left(5 - \frac{5}{3}\right)^2\cdot P(Y=5)\\
&= \frac{25}{9} \cdot \frac{32}{243} + \frac{4}{9}\cdot \frac{80}{243} + \frac{1}{9} \cdot \frac{80}{243}
   + \frac{16}{9} \cdot \frac{40}{243}  + \frac{49}{9} \cdot \frac{10}{243} + \cdot \frac{100}{9} \cdot \frac{1}{243}\\
&= \frac{800 + 320 + 80 + 640 + 490 + 100}{9\cdot 243}\\
&= \frac{2430}{9\cdot 243}\\
&= \frac{10}{9}.
\end{align*}
This value is perhaps less ``obvious'' than what we saw before for the expected value of $Y$.  
Nevertheless, note that it is equal to $5 \cdot \frac{1}{3} \cdot \frac{2}{3} = n p q$.  
We could also define a standard deviation $\sigma(Y)$ by taking the square root:
$$\sigma(Y) = \sqrt{\frac{10}{9}} = \frac{\sqrt{10}}{3}.$$
We will discuss interpretations of the standard deviation for a random
\index{binomial distribution!variance}
\index{distribution!binomial}
\index{random variable!binomial}
variable later.  
$\triangle$
\end{example}

The pattern seen in Example~\ref{BinVar} is also true in general for binomial random variables.

\begin{theorem}  Let $Y$ be a binomial random variable based on $n$ trials, with success probability $p$.  Then
$V(Y) = npq$.
\end{theorem}

This and the corresponding formulas for geometric and Poisson random variables can be 
shown in general by more involved summation manipulations along the lines of the ones
we have seen so far.  The final results of a whole lot of algebra are given in the following statements.

\begin{theorem}  Let $Y$ be a geometric random variable with success probability $p$.  Then
\index{geometric distribution!variance}
\index{distribution!geometric}
\index{random variable!geometric}
$V(Y) = \frac{q}{p^2}$.
\end{theorem}

\begin{theorem}  Let $Y$ be a Poisson random variable with parameter $\lambda$.  Then
$V(Y) = \lambda$.
\index{Poisson distribution!variance}
\index{distribution!Poisson}
\index{random variable!Poisson}
\end{theorem}
\index{variance|)}

\specialsection{Continuous Distributions and Probability Density Functions}
\index{continuous random variable}
\index{random variable!continuous}
We now turn to considering what are known as \emph{continuous} random variables and their 
probability distributions.  This is a topic where a complete discussion requires techniques
from integral calculus, so we will only sketch the main ideas and indicate some representative
examples.  

To begin, continuous random variables can take any value in an interval on the real
number line, an uncountably infinite set of values.  To describe them, we use 
what are known as \emph{probability density functions} or p.d.f.s for short. 
The power law distributions considered in Chapter 6 can be thought of as examples,
where the p.d.f. is a power function and nonzero for all positive real numbers.
 For
\index{probability density function (p.d.f.)}
reasons that should become clearer soon, we will use the variable name $t$
for the input to the p.d.f. of a continuous random variable.  The
idea is to consider something like the probability histograms we saw in Figures~\ref{fig:TroutPMF}
\index{histogram}
\index{probability histogram}
and \ref{fig:Poisson}, but where the collection of boxes centered on the integer $t$-values 
is replaced by a general region between a graph $y = f(t)$ and the $t$-axis.  The function $f(t)$ giving
the graph is the p.d.f. and it must satisfy the following properties:
\begin{enumerate}
\item $f(t) \ge 0$ for all real $t$, and
\item The total area between the graph $y = f(t)$ and the $t$-axis is equal to 1.\footnote{Technical note:  This is
a first place where integral calculus comes into play.  The precise way to state this is that
we require $\int_{-\infty}^\infty f(t)\ dt = 1$.}
\end{enumerate}
For a continuous random variable $X$ with p.d.f. equal to $f(t)$ we define:\footnote{Technical note: Again, the 
precise meaning uses calculus:  $P(a \le X \le b) = \int_a^b f(t)\ dt$.}
\begin{equation}
\label{eq:ContProbDef}
\framebox{$P(a \le X \le b) = {\rm total\ area\ between\ }t{\rm-axis\ and\ } y = f(t), a \le t \le b$.}
\end{equation}
\index{probability}
Because of conditions 1 and 2 above, this implies
$$0 \le P(a \le X \le b) \le 1$$
for all real numbers $a,b$ and we can interpret the numbers we get as in the discrete case.  

The p.d.f. also makes it possible to compute what is known as the 
\index{cumulative distribution function (c.d.f.)}
\emph{cumulative distribution function}, or c.d.f. of a continuous random variable.\footnote{Technical note: 
Continuity of a random variable is usually defined in more advanced treatments of probability
theory by saying that the c.d.f. of the random variable is a continuous function.
The c.d.f. of a discrete random variable with values in the integers can also defined, but it will have a jump 
discontinuity at each integer $k$ where $P(X = k) \ne 0$.  
There are also random variables
that can be seen as ``mixtures'' of the continuous and discrete types.  We will not consider them.}  
The c.d.f. is defined as 
\begin{equation}
\label{eq:CDFDef}
\framebox{$F(x) = P(X \le x) = {\rm total\ area\ between\ }t{\rm-axis\ and\ } y = f(t), t \le x$.}
\end{equation}
We use the name $t$ for the variable in the p.d.f. here to distinguish it from the $x$ which 
is the input to the c.d.f. $F(x)$.  

\begin{figure}
\begin{center}
\includegraphics[height=3cm, trim={0 4.5cm 0 5cm},clip]{UniformPDF.png}
\caption{p.d.f. of a uniform random variable on $2 \le t \le 5$.}
\label{fig:UnifPDF}
\end{center}
\end{figure}

Here is a simple, but representative example.

\begin{example}
\label{Uniform}
Continuous random variables with a p.d.f. of the form 
\index{uniform distribution}
\index{distribution!uniform}
\index{random variable!uniform}
\begin{equation}
\label{eq:UniformPDF}
f(t) = \begin{cases}  \frac{1}{d - c}, & c < t < d\\
                                           0, & {\rm otherwise}  
                                           \end{cases}
\end{equation}
for some $c < d$ are known as \emph{uniform} random variables.  

For any uniform random variable, the probability that $X$ lies in an interval $(a,b)$
contained in $(c,d)$ is simply equal to the area of the \emph{rectangle} under the graph $y = f(t)$
for $a< t < b$.  The corresponding area is the ratio 
$$P(a < X < b) = (b -a) \cdot \frac{1}{c - d} = \frac{b - a}{c - d}.$$
If $(a,b)$ does not lie completely within $(c,d)$, then we would take the area 
under $y = \frac{1}{d - c}$ for $t$ in the part of $(a,b)$ that does lie within $(c,d)$.  

For instance, if  $c = 2$ and $d = 5$, then the graph of the p.d.f. is shown in Figure~\ref{fig:UnifPDF}.
The open circles at $y = \frac{1}{3}$ and the solid circles at $y = 0$
indicate that the values $f(2)$ and $f(5)$ are actually equal to $0$ according to this definition.
For a random variable $Y$ with that p.d.f., we would have 
$P(3 < Y < 4) = (4 - 3)\cdot \frac{1}{5 - 2} = \frac{1}{3}$.  Similarly, 
$P(Y > 4.5) = \frac{1}{6}$, since the part of the interval of $t > \frac{9}{2}$ within $(2,5)$ has length $\frac{1}{2}$, 
so we are looking at the area of  a rectangle of width $\frac{1}{2}$ and height $\frac{1}{3}$.

The shaded red rectangle here is the region under the graph of the p.d.f. between
$t = 2$ and $t = 3.1$, which is the same as the region for $t \le 3.1 = \frac{31}{10}$.  We see 
from this that 
$$P\left(2 \le x \le \frac{31}{10}\right) = \frac{1}{3}\cdot \left(\frac{31}{10} - 2\right) = \frac{11}{30} \doteq .3667.$$
Similarly, since $f(t) = 0$ for all $t \le 2$, we have
$$F(3.1) = P(X \le 3.1) = \frac{11}{30} \doteq .3667.$$

If we imagine $x$ beginning to the left of $t = 2$, sweeping to the right past $t = 2$, through the 
interval (including the value $t = 3.1$ shown above in Figure~\ref{fig:UnifPDF}, then through $t = 5$ and beyond,
we can see that the value of the c.d.f. $F(x)$ will be equal to 0 for all $x \le 2$, then increase steadily and linearly (we're
adding area at a constant rate if $x$ increases at a constant rate) from 
$0$ up to $1$ on the interval $2 \le x \le 5$, then remain at $1$ for all $x \ge 5$:
$$
F(x) = \begin{cases}  0, & x \le 2 \\
                           \frac{1}{3} (x - 2), & 2 < x < 5\\
                                      1, & x \ge 5.
           \end{cases}
$$
The c.d.f. is shown in Figure~\ref{fig:UnifCDF}.  Note that this
graph is continuous even though the p.d.f. had jump discontinuities
at $t = 2$ and $t = 5$.  As is true in this example, c.d.f.s of continuous
random variables are always ``non-decreasing'' -- that is $x_1 \le x_2$
always implies $F(x_1) \le F(x_2)$.   $\triangle$
\end{example}

\begin{figure}
\begin{center}
\includegraphics[height=3cm, trim={0 4.5cm 0 5cm},clip]{UniformCDF.png}
\caption{c.d.f. of the uniform random variable from Example~\ref{Uniform}.}
\label{fig:UnifCDF}
\end{center}
\end{figure}

Verifying the details claimed in the following example requires some
calculus, but we will present it in any case because it illustrates
the connection between p.d.f.s and c.d.f.s even more clearly.

\begin{example}
\label{AnotherPDFCDFPair}
\index{p.d.f.s and c.d.f.s}
The graph of the ``tent function''  
$$f(t) = \begin{cases}
             0, & t \le 1\\
             t - 1, & 1 < t \le 2\\
             -t + 3, & 2 < t \le 3\\
             0, &  t > 3
             \end{cases}$$
is shown in Figure~\ref{fig:AnotherPDF}.  This is a valid
p.d.f. because $f(x) \ge 0$ for all real $x$, and the total area
between the graph and the $x$-axis is 1 since it is made up
of two triangles with base 1 and height 1.  

\begin{figure}
\begin{center}
\includegraphics[width=8cm, trim={0 3cm 0 4cm},clip]{TentPDF.png}
\caption{p.d.f. from Example~\ref{AnotherPDFCDFPair}.}
\label{fig:AnotherPDF}
\end{center}
\end{figure}

The corresponding c.d.f. is given by 
$$F(x) = \begin{cases} 
               0, & x \le 1\\
               \displaystyle \frac{x^2}{2} - x + \frac{1}{2},  & 1 < x \le 2\\
               \displaystyle \frac{-x^2}{2} + 3x - \frac{7}{2}, & 2 < x \le 3\\
              1, x > 3.
              \end{cases}$$
This function is shown in Figure~\ref{fig:AnotherCDF}.
   
\begin{figure}
\begin{center}
\includegraphics[width=8cm, trim={0 2.5cm 0 4cm},clip]{TentCDF.png}
\caption{c.d.f. from Example~\ref{AnotherPDFCDFPair}.}
\label{fig:AnotherCDF}
\end{center}
\end{figure}         

Readers who know some calculus might recognize that in both of the cases we have
worked out, the c.d.f. has the property that \emph{its derivative is the corresponding}
p.d.f.\footnote{Technical note: This is a consequence of the first part of the Fundamental Theorem
of Calculus:
$$F'(x) = \frac{d}{dx} \int_{-\infty}^x f(t)\ dt = f(x).$$}
Even if you have not seen these ideas from calculus before, note
how the graph of the c.d.f. is ``smoother'' than the graph of the p.d.f.
in both cases.  $\triangle$            
\end{example}

The formulas for computing expected values and variances from Section 10.3 generalize to 
the case of continuous random variables as well.  However, this is one place where tools
from calculus are actually very necessary for computations in almost all 
cases.\footnote{Technical note:
The expected value of the continuous random variable $X$ with p.d.f. $f(x)$ is computed by 
\index{expected value}
$$E(Y) = \int_{-\infty}^\infty x\cdot f(x)\ dx$$
and the variance is 
\index{variance}
$$V(Y) = E((Y - E(Y))^2) = \int_{-\infty}^\infty (x - E(Y))^2 f(x)\ dx.$$}
For instance, for the uniformly distributed random variables, we have:

\begin{theorem}
\label{UniformEVVar}
\index{random variable!uniform}
\index{distribution!uniform}
\index{uniform distribution!expected value and variance}
Let $Y$ be a random variable with a uniform distribution on the interval
$(c,d)$.  Then 
$$E(Y) = \frac{c + d}{2} \qquad {\rm and} \qquad V(Y) = \frac{(d - c)^2}{12}.$$
\end{theorem}

In this case, the expected value is just the midpoint of the interval $(c,d)$ where
the density is nonzero.  The variance is proportional to the square of the length 
of the interval and hence the standard deviation is proportional to the length of 
the interval, corresponding to the intuitively clear idea that the larger $d - c$ is,
the more variability there will be in values of the random variable.  

To conclude this section, we will state without proof an important fact about
collections of random variables $Y_1,\ldots, Y_n$ satisfying the following
two properties:
\begin{itemize}
\item We will assume that the $Y_i$ all have \emph{the same}  distribution (i.e. the same p.m.f. in the discrete 
case or the same p.d.f. in the continuous case), 
\item We will assume that the $Y_i$ are \emph{independent} in the sense that 
$$P(a_i < Y_i < b_i\ {\rm and}\ a_j < Y_j < b_j) = P(a_i < Y_i < b_i)\cdot P(a_j < Y_j < b_j)$$
for all pairs $i \ne j$ and all intervals $(a_i,b_i)$ and $(a_j,b_j)$.\footnote{Technical note: Independence is usually defined using the 
idea of the \emph{joint p.d.f.} of the collection of random variables and this statement about probabilities is a consequence.}
\end{itemize}
These conditions are often abbreviated by saying the $Y_i$ are i.i.d. (``independent and identically distributed'') random variables.
\index{random variable!i.i.d. collection}

\begin{theorem} 
\label{IIDFacts}
\index{expected value}
\index{variance}
Let $Y_1,\ldots,Y_n$ be an i.i.d. collection of random variables with $E(Y_i) = \mu$ and $V(Y_i) = \sigma^2$
for some $\mu$ and $\sigma$.  Let $\overline{Y} = \frac{1}{n}(Y_1 + \cdots + Y_n)$.  Then
$\overline{Y}$ is a new random variable with 
$$E(\overline{Y}) = \mu\qquad {\rm and}\qquad V(\overline{Y}) = \frac{\sigma^2}{n}.$$
\end{theorem}

The statement about $V(\overline{Y})$ is the ultimate explanation for what you saw in question G 
in the Chapter Project from Chapter 9.  Recall that the SDs of the sample means seemed to 
decrease as the sample size increased.  In fact, the exact pattern is that the means of samples
of size $n$ will have $SD = \frac{\sigma}{\sqrt{n}}$.  

\specialsection{Normal Distributions}
\index{normal distribution|(}
\index{distribution!normal|(}
The most important continuous probability distributions in traditional methods
for analyzing scientific data and making statistical inferences are certain the \emph{normal distributions}.
A continuous random variable $Y$ is said to have a normal distribution if its p.d.f. has the form
\index{random variable!normal}
\index{expected value}
\index{probability density function (p.d.f.)}
\index{variance}
\begin{equation}
\label{eq:normalpdf}
f(t) = \frac{1}{\sqrt{2\pi\sigma^2}} e^{\displaystyle -(t - \mu)^2/(2\sigma^2)}.
\end{equation}
This formula includes two parameters $\mu$ and $\sigma$ and it can be shown, 
via tools from calculus, that $\mu = E(Y)$ and $V(Y) = \sigma^2$, so changing 
$\mu$ changes the expected value, and changing $\sigma$ changes the variance
and hence the standard deviation.   The constant multiplier  $1/\sqrt{2\pi\sigma^2}$ 
is included to make the total area between the graph $y = f(t)$ and the $t$-axis
equal to 1.  

\begin{figure}
\begin{center}
\includegraphics[width=8cm, trim={0 0cm 0 0cm},clip]{StdNormalPDF.png}
\caption{The standard normal p.d.f. }
\label{fig:StandardNormal}
\end{center}
\end{figure}      

Among the normal p.d.f.s, a special role is played by the function of the form 
from \eqref{eq:normalpdf} with $\mu = 0$ and $\sigma = 1$:
\begin{equation}
\label{eq:Zpdf}
f(t) = \frac{1}{\sqrt{2\pi}} e^{-t^2/2},
\end{equation}
called the \emph{standard normal p.d.f.}.  This function is plotted for
$-4 \le t \le 4$ in Figure~\ref{fig:StandardNormal}.  This is the standard
``bell-shaped'' curve that defines normally-distributed random variables.
\index{probability density function (p.d.f.)}
\index{random variable!standard normal}
\index{distribution!standard normal}
\index{standard normal distribution}

Using numerical techniques, we can compute the area of the region between
this graph and the $t$-axis for $-4 \le t \le 4$, and the result is approximately
$.99994$, very close to $1$, but a bit less.  The ``missing area'' lies in the 
\emph{tails of the distribution}:  $t > 4$ and $t < -4$.   
\index{tails of normal distribution}
\index{normal distribution!tails}

Changing $\mu$ while leaving $\sigma$ unchanged merely shifts the whole
p.d.f. plot to the right or left.  Changing $\sigma$ without changing $\mu$
changes the height and the width of the \emph{peak}, or spike, of the graph
at $x = \mu$.  For instance, a large value of $\sigma$ will give a short, 
widely-spread peak, while a small value of $\sigma$ will yield a
tall, narrow peak.  Several examples plotted together on the
same set of axes in Figure~\ref{fig:Normals}.  The red graph is
the normal p.d.f. with $\mu = 2$, $\sigma = .2$; the green graph has 
$\mu = 1$ and $\sigma = .8$, while the blue graph has $\mu = -1$ and $\sigma = 2$.  

\begin{figure}
\begin{center}
\includegraphics[width=8cm, trim={0 0.8cm 0 0.2cm},clip]{NormalPDFs.png}
\caption{Several normal p.d.f.s plotted together}
\label{fig:Normals}
\end{center}
\end{figure}      

We will introduce several other continuous distributions related to the normals, including
$\chi^2$ and $t$ distributions in future chapters.

\specialsection{$Z$-scores and Computing Normal Probabilities}
\index{z@$z$-scores|(}
\index{probability!normal|(}
The reason that the standard normal from \eqref{eq:Zpdf} and Figure~\ref{fig:StandardNormal}
is so important here is the following fact, shown in more advanced probability and mathematical
statistics courses.

\begin{theorem}
\label{standardize}
If $Y$ has a normal distribution with expected value $\mu$ and standard deviation
$\sigma$, then 
$$Z = \frac{Y - \mu}{\sigma}$$
has a standard normal distribution.
\index{normal distribution!standardization formula}
\end{theorem}

This means that any computation of areas under any normal p.d.f. graph can be translated
into an equivalent problem using the standard normal p.d.f..  The idea is the following.  If
$Y$ is a normal random variable with expected value $\mu$ and standard deviation $\sigma$,
then the inequalities $a \le Y \le b$ can be converted into an \emph{equivalent pair of inequalities}
by subtracting $\mu$ and dividing through by the positive number $\sigma$:
$$\frac{a - \mu}{\sigma} \le \frac{Y - \mu}{\sigma} \le \frac{b - \mu}{\sigma}.$$
By the theorem the middle term here is a standard normal, and we have reduced the 
problem to finding an area between the graph of the standard normal p.d.f. graph and the 
horizontal axis.  Moreover, 
\begin{equation}
\label{eq:Zscores}
\framebox{$\displaystyle P(a \le Y \le b) = P\left(\frac{a - \mu}{\sigma} \le Z \le \frac{b - \mu}{\sigma}\right),$}
\end{equation}
because the two sets of inequalities are equivalent.

If $y$ is a value of the normally-distributed random variable $Y$ 
(with expected value $\mu$ and standard deviation $\sigma$), the number 
$$z = \frac{y - \mu}{\sigma}$$
is often called the \emph{$Z$-score corresponding  to $y$}.   So \eqref{eq:Zscores} says that to find 
the probability that a normal random variable takes a value in a given interval, we simply compute the 
$Z$-scores corresponding to the endpoints of the interval and find the area under the standard
normal for the interval with endpoints at the $Z$-scores.  
\index{probability!normal}
\index{standard normal table}

Traditionally this is done by way of a \emph{table of standard normal curve areas}\footnote{From: {\tt http://statistics.wdfiles.com/local--files/ch7/normDistTable.pdf}.
License: CC BY-SA: Attribution-ShareAlike, downloaded October 10, 2017.}, although use of mathematical
and statistical software for this purpose is also becoming more and more common.\footnote{Technical note:
the standard normal p.d.f. does not have an elementary antiderivative, so symbolic computation with the 
Fundamental Theorem of Calculus is insufficient for finding the corresponding integrals.  The computation of these
integrals is done by approximate numerical integration and the table in the text
and similar tables available in many places are all generated in that way. } 
One such table is given in Figure~\ref{fig:NormalCurveAreas} on
the next page.

\begin{figure}
\begin{center}
\includegraphics[width=13cm,height=20cm,trim={0.5cm 0 0 0}, clip]{normalcurveareas.jpeg}
\caption{Table of values of $P(0 < Z < z)$}
\label{fig:NormalCurveAreas}
\end{center}
\end{figure} 

%\vfill\eject

To learn how to use the table, note the heading showing another plot of the standard normal p.d.f. with the shaded region starting at
$t = 0$ and extending to the right until $t = z$.   The ones digit and the tenths digit of the number $z > 0$
are given along the left column and the hundredths digit of $z$ is given across the top and again at 
the bottom of the table.  

For example, to find the area representing $P(0 < Z < 1.21)$, we would look in the 
row labeled $1.2$ and the column labeled $.01$ to find
$$P(0 < Z < 1.21) \doteq .3869.$$
Similarly the area representing $P(0 < Z < 0.47)$ is found in the 
row labeled $0.4$ and the column labeled $.07$:
$$P(0 < Z < 0.47) \doteq .1808.$$
Note that by the time $z$ reaches $z = 3.59$ the area has ``settled down'' to a number very 
close to $.5000$, so the table does not continue to higher values of $z$.


By adding or subtracting areas, and/or making use of the \emph{symmetry} of the standard normal p.d.f. ($f(-t) = f(t)$) 
we can compute many other types of areas under the standard normal curve besides those of the form $P(0 < Z < z)$. 
The following discussion covers (almost) all of the cases that arise in practice:
\begin{enumerate}
\item[(1)]  By the symmetry $P(Z > 0) = P(Z < 0)$ and each of these  $= .5000$, since the total area must equal $1.0$.  
\item[(2)]  Hence if $z > 0$ the \emph{upper tail area} $P(Z > z)$ equals the difference between $P(Z > 0) = .5000$ and the area between $0$ and $z$:
$$P(Z > z) = P(Z > 0) - P(0 < Z < z) = .5000 - P(0 < Z < z).$$
For example, $P(Z > .89) = .5000 - P(0 < Z < .89) = .5000 - .3133 = .1867$.
\item[(3)]  The area of a region extending from a negative number $z$ to 0 can be obtained
since $P(-z < Z < 0) = P(0 < Z < z)$.  For example, 
$$P(-2.30 < Z < 0) = P(0 < Z < +2.30) \doteq  .4893.$$
\item[(4)]  Combining (2) and (3) we see that when $z < 0$ the \emph{lower tail area} $P(Z <  z) = P(Z > - z) = .5000 - P(0 < Z < -z)$ (note $-z > 0$ if
$z < 0$ as we are assuming here).  For instance $P(Z < -.63) = P(Z > .63) = .5000 - .2357 = .2643$.  
\item[(5)]  For areas extending from one positive number $z_1 > 0$ to another $z_2 > z_1$, such as $P(z_1 < Z < z_2)$, we
can obtain the desired area by subtraction:
$$P(z_1 < Z < z_2) = P(0 < Z < z_2) - P(0 < Z < z_1).$$
For example, 
\begin{align*}
P(0.48 < Z < 1.64) &= P(0 < Z < 1.64) - P(0 < Z < 0.48)\\
 &= .4495 - .1844\\ 
 &= .2651.
 \end{align*}
Applying the same idea and symmetry, an area $P(z_1 < Z < z_2)$ with $z_1 < z_2 < 0$ can also be obtained easily. 
\item[(6)]  Finally, for intervals containing both positive and negative numbers, we can split
the interval at zero and sum intervals of the types given in 
the previous parts to find the desired areas.  For example, 
\begin{align*}P(-.37 < Z < 1.28) &= P(-.37 < Z < 0) + P(0 < Z < 1.28)\\
&= P(0 < Z < .37) + P(0 < Z < 1.28)
\end{align*}
(by the symmetry).  From the table we find this equals $.1443 + .3997 = .5440$.    Similarly 
\begin{align*}
P(Z > -1.00) &= P(-1.00 < Z < 0) + P(Z > 0)\\
&= P(0 < Z < 1.00) + P(Z > 0)
\end{align*}
(using symmetry again).  From the table, this equals $.3413 + .5000 = .8413$.
\end{enumerate}

It is my experience that trying to memorize the formulas that apply in each of the cases here is \emph{not an efficient way
to learn to carry out these calculations}.  Instead, if you draw (or, with some practice, just visualize) a schematic picture of the standard normal 
p.d.f. and the locations of $a$ and $b$, you should see how to apply symmetry and addition or subtraction to 
find the desired area $P(a < Z < b)$ in all of these cases.

We conclude with an example showing the full process of computing a normal probability
using the $Z$-scores and the normal curve area table.

\begin{example}
\label{NormalEx}
Suppose that the weights of adult individuals of a certain species of fish are known to be normally distributed with 
mean $\mu = 500$ grams and standard deviation $\sigma = 30$ grams.  We ask, if we catch a fish at 
random, what is the probability that its weight will be between $455$ and $570$ grams?

We begin by computing the $Z$-scores corresponding to the endpoints of the interval of weights we want:
$$y = 455 \longleftrightarrow z = \frac{455 - 500}{30} = -1.5$$
and 
$$y = 570 \longleftrightarrow z = \frac{570 - 500}{30} \doteq 2.33.$$
From \eqref{eq:Zscores}, we know that 
$$P(455 < {\rm weight} < 570) = P(-1.5 < Z < 2.33).$$
This is a case where the endpoints are on opposite sides of $z = 0$, so we split and use the symmetry:
\begin{align*}
P(-1.5 < Z < 2.33) &= P(-1.5 < Z < 0) + P(0 < Z < 2.33)\\
&= P(0 < Z < 1.5) + P(0 < Z < 2.33) \\
&\doteq .3531 + .4901 = .8432.
\end{align*}
In other words, there is about an $84\%$ chance that an adult fish randomly chosen from the whole population will have
a weight in that interval.  $\triangle$
\end{example}

In case you are wondering what happens with the $Z$-score $Z = 0$, which apparently does not end up
in either interval, rest assured.  We could have included it in either interval or in both and the results
would be exactly the same.  The probability that a \emph{continuous} random variable takes any 
single value is always \emph{equal to zero}.  
\index{z@$z$-scores|)}
\index{probability!normal|)}
\index{normal distribution|)}
\index{distribution!normal|)}

\specialsection{Chapter Project}

The goals of this chapter's project (which will take several days) are:
\begin{itemize}
\item to introduce some basic features of the R statistics 
\index{R@{\tt R} statistical package|(}
package (a much more extensive and powerful piece of software
than the statistical routines in Excel -- in fact R is probably the go-to
 ``industry standard'' software for academic statisticians at the moment), 
\item to see how R computes some descriptive statistics 
and graphics for understanding the ``shape'' or ``distribution'' 
of a data set, and 
\item to gain an intuitive understanding of the \emph{Central Limit Theorem}, 
a basic result in statistics that underlies the wide applicability of normally-distributed
random variables.  
\index{Central Limit Theorem}
\end{itemize}

\subsection*{Background on R -- Getting Started}
The instructions here are geared toward the Windows version
of R.   Since R is a free,
open source package, you can also obtain a copy to run
on your own computer if you want, and there are Windows, 
Mac-OS, and Linux versions available.  These have the same 
functionality but slightly different interfaces.   

On PC's when you double-click the R shortcut from the
standard desktop, you will launch the R-GUI window.
Inside this is a workspace window with a white background.
It \emph{(only)} allows you to 
\begin{itemize}
\item enter commands, and 
\item view numerical output.
\end{itemize}
It does not let you create integrated documents, and
graphics are displayed in separate windows within the 
R-GUI, as you will see.

You can \emph{print} the workspace window or any graphics
window at any time.  Just highlight that window and
press the toolbar printer button.  

To save your work, with comments and answers to questions
(e.g. for the project report), I recommend 
\begin{itemize}
\item opening another editor window (either Word or NotePad
are fine for this on Windows systems), and
\item using it to create a text file that will be
your permanent record of your work.
\item When you have a result you want to save from
the R workspace, copy both the input command and
the output and paste into the text file.
\item Any time you want to add a comment to explain
a calculation or answer a question, enter that 
directly into the text file. 
\item Save that text file in your personal network
folder and print it when you want a hardcopy
(from Word or NotePad).  
\end{itemize}

\subsection*{A First R Session}
\vskip 10pt
In this first session, you will basically work through some 
examples illustrating features of R and some of the 
descriptive statistics we have talked about so far in 
the course.  Keep a record of what you do for future reference.

In its most basic form, R provides a command-driven
``statistical calculator'' in which you type in data and
commands to generate numerical and graphical output that
are displayed.  When you have the R window open, note the
$>$ input prompt after the opening message.  This is 
where you should start. 

(1)  The following data set records
systolic blood pressure readings from 15 adults (not a very
healthy group, on the whole!)
First we need to get the systolic blood pressure readings
into R so that we can work with them.  The basic way is to 
enter (after the $>$ prompt):
\vskip 10pt
\begin{center}
{\tt bpdata <- c(172,140,123,130,115,148,108,129,137,161,123,152,133,128,142)}
\end{center}
\noindent
and press ENTER.  Note:  the $<-$ is formed by typing the $<$ and $-$
characters right next to each other.  It is the {\it assignment operator}
in R.  This takes the data on the right and assigns it to the name on
the left.   The $c$, followed by the list
of numbers, separated by commas is R's way of constructing 
{\it an ordered list or vector}.  You should just get another
input prompt if everything went OK with this (error messages if not).
If you did get an error, try again, and try to follow the above exactly.
You can verify that the numbers are entered correctly by typing in the 
name {\tt bpdata} as a command; the numbers will be displayed in a row.

(a)  We can compute the mean $\overline{y}$ and standard deviation 
$s$ of the data in R (in several different ways!):  Type in 
\vskip 10pt
\begin{center}
{\tt ybar <- sum(bpdata)/length(bpdata)}\\
{\tt mean(bpdata)}\\
{\tt s <- sqrt(sum((bpdata - ybar)\^{}2)/(length(bpdata) - 1))}\\
{\tt sd(bpdata)}
\end{center}
\noindent
and press ENTER after each line to see the results.  Note the way the sum function
works in a very nice way here -- in the computation of {\tt ybar}, we only need
to put in the name of the data list; the sum of all the elements is computed by 
default.  Then in the computation of {\tt s}, we can subtract \emph{each entry}
of the {\tt bpdata} vector from {\tt ybar} as shown!  Of course we will usually
want now to use the built-in functions {\tt mean} and {\tt sd}, but
the first of each pair of computations shows how we can also use more primitive functions
in R when we need them.  

(b)  Chebyshev's theorem mentioned in Chapter 9 is a result about the distribution of data 
sets.  It says in a particular case
that \emph{there will always be at least $75\%$ of the numbers in a data set 
within 2 standard deviations of the mean}.  Let's use R to check that this is true here.  In its most basic
use, R can function as a simple numerical calculator(!)
\vskip 10pt
\begin{center}
{\tt upperb <- ybar + 2*s}\\
{\tt lowerb <- ybar - 2*s}
\end{center}
\noindent
How many of the data values are in the range $[\overline{y} - 2 s, \overline{y} + 2s]$,
though?  We could just count by hand, of course, with a data set this small.
But R also contains a very rich collection of facilities for selecting, subsetting, 
and massaging data in numerous ways.  Here is a basic way to select the portion 
of the data set in the range we want:
\vskip 10pt
\begin{center}
{\tt middle <- subset(bpdata, bpdata <= upperb \& bpdata  >= lowerb)}\\
{\tt length(middle)/length(bpdata)}
\end{center}
(This should be pretty self-explanatory; ask if you don't understand what this did!)
Did Chebyshev's theorem ``work'' here?


(2)  Now let's look at a different data set.  This gives the number of parasites found in 
each of a sample of 100 individual foxes.
The information is given there in table format:
\begin{center}
\begin{tabular}{l|ccccccccc}
Parasites & 0 & 1 & 2 & 3 & 4 & 5 & 6 & 7 & 8\\ \hline
Foxes & 69 & 17 & 6 & 3 & 1 & 2 & 1 & 0 & 1\\
\end{tabular}
\end{center}
So we want to create a data set containing 69 0 entries, 17 1 entries, etc.
Clearly this is going to be tedious and error prone if we do it the most basic
way(!)  Fortunately, R lets us construct lists in more flexible ways
besides just listing the entries.  We can indicate how many times to repeat
a given value if we want, like this:
\vskip 10pt
\begin{center}
{\tt foxpar <- rep(c(0,1,2,3,4,5,6,7,8),c(69,17,6,3,1,2,1,0,1))}\\
\end{center}
\noindent
The {\tt rep} stands for ``repetition'' here.
Note that the first list inside the {\tt rep} gives the values of the number of parasites -- the first row in the table format -- and the 
second list gives number of times each value appears in the data -- the second row.  The first list could also be abbreviated as
{\tt 0:8} (R's notation for a list of successive integer values).  

(a)  To construct a relative frequency histogram
\index{frequency histogram!relative}
\index{histogram}
for the number of parasites per fox in R, let's begin by entering
\vskip 10pt
\begin{center}
{\tt hist(foxpar)}\\
\end{center}
\noindent
The histogram will be generated in a separate graphics window.  
This is a ``default'' version of the command.  The {\tt hist}
command takes a number of options that control how the ``bins''
are selected and how the histogram is drawn as well.  You 
will often want to modify the default results.   For
instance, looking at the output, how were the bins chosen here?
Is that optimal for showing the different numbers of foxes
with each number of parasites?  Suppose we want bins centered
on the integer values $0,1,2,3,4,5,6,7,8$ as we did in other
examples.  We can put in an option to make
R do it that way too.  The {\tt breaks} option takes
a list of values and makes those the boundaries of the bins 
for the histogram:
\vskip 10pt
\begin{center}
{\tt hist(foxpar,breaks=c(-0.5,0.5,1.5,2.5,3.5,4.5,5.5,6.5,7.5,8.5))}\\
\end{center}
\noindent
How is this different and why is it better than the default
way that R drew the histogram?  

(b) To practice using the commands we talked about
before, calculate $\overline{y}$ and $s$ for this data set.
How many of the numbers were within two standard deviations
of the mean in this case?  

(c)  The list of breaks for the histogram can be
anything in R.  What happens if you change the command above
to 
\vskip 10pt
\begin{center}
{\tt hist(foxpar,breaks=c(-0.5,0.5,1.5,2.5,3.5,4.5,5.5,8.5))}\\
\end{center}
\noindent
(deleting the 6.5 and 7.5 break points, so we're lumping
together the counts for the values 6,7,8 parasites)? 
What is different about the new histogram that R drew in this case?  
Look at the graph carefully: do the heights of the boxes still represent 
\emph{frequencies}?  (Hint: what
they do represent are \emph{densities} that are computed
by taking the frequency count and dividing by the 
width of the box in each case.  What is true if we 
add the areas of the boxes drawn this way?  We will see
other reasons why this is a preferable way to draw histograms
with unequal bin sizes later. )
If we \emph{really want frequencies}, we can make R do it that way:
\vskip 10pt
\begin{center}
{\tt hist(foxpar,breaks=c(-0.5,0.5,1.5,2.5,3.5,4.5,5.5,8.5),freq=TRUE)}\\
\end{center}
\noindent
However note that now R generates a warning message saying
the AREAS in the plot are ``wrong.''  Think about this.
Why is the last box in this version of the histogram
\emph{misleading} if drawn this way?  

\subsection*{Background on Probability Distributions in R}
The R package contains built-in functions for computing
probability functions for the discrete random 
variables we have discussed, plus functions for the probability
density functions and cumulative distributions of the 
standard types of continuous random variables.  Here's how it works. 
Each type of random 
variable is covered by a \emph{family} of 4 functions distinguished by a 
\emph{prefix letter} {\tt d, p, q}, or {\tt r}:
\vskip 10pt
\begin{itemize}
\item {\tt d} -- the p.m.f.  in the discrete case, or the p.d.f. in the continuous case,
\item {\tt p} -- the c.d.f.
\item {\tt q} -- the quantile function (we won't be using this one, but 
it can compute quartiles, deciles, percentiles, etc.)
\item {\tt r} -- random number generator -- for sampling random values
of a random variable with that distribution
\end{itemize}
Following the prefix letter comes the rest of the function 
name and the inputs needed to compute the corresponding values.
\vskip 10pt
\noindent
\subsection*{Distributions of Sample Means}
\index{Central Limit Theorem|(}
Work through the following example before and then follow the 
pattern here in the other parts of question A below. 
You do not need to hand in any work for this preliminary
discussion, just for the parts of the question itself.
\begin{enumerate}
\item Begin by generating a list of $100$ random numbers from a particular
\emph{exponential} distribution (containing a parameter called the ``rate'' in R).\footnote{This is a type of continuous random
variable that we did not discuss in the text.  You will see roughly what
it's p.d.f. looks like below.}  
Use the command:
\vskip 10pt
\begin{center}
{\tt x <- rexp(100,1/3)}
\end{center}
\noindent
to assign the result to the name {\tt x}.  We will think of 
this as a \emph{sample} of values from that distribution.
\item Plot the density histogram for this data set {\tt x} using the 
command
\vskip 10pt
\begin{center}
{\tt hist(x,freq=FALSE)}
\end{center}
This should look roughly like the part of the graph of the 
p.d.f.  for positive $y$, reflecting the fact that the random numbers
are generated according to probabilities given by that density function.
\item Our main goal is to understand what happens if we generate \emph{lots}
of such samples, find their sample means, then consider \emph{the distributions
of those sample means}.  
\item Here is one way to do this in {\tt R} using
a simple \emph{for loop}.  First we create space to store the means of 
1000 different random samples:
\vskip 10pt
\begin{center}
{\tt means <- array(1:1000)}
\end{center}
Next, we generate 1000 different random samples, find the 
means of each of them, and store them in the array created above:

\begin{verbatim}
                for (i in 1:1000) 
                   { 
                   x <- rexp(100,1/3)
                   means[i] <- mean(x)
                   } 
\end{verbatim}

(Note: This could all be entered as one input line provided you leave
spaces in the appropriate places.  However, I think it is more readable 
(and it is easier to identify typos if you happen to make one) 
if you press ENTER at the end of each of the lines as
you are typing this.  If you do it that way, you should note that {\tt R} generates a
new input prompt {\tt +} each time, indicating that you 
are still in the body of the {\tt for} loop.  The final {\tt \}}
will close off the loop and take you back to the $>$ prompt.)
\item What does the distribution of the sample means look like?
We can see that by generating a density histogram for the data in the {\tt means}
array:
\vskip 10pt
\begin{center}
{\tt hist(means,breaks=20,freq=FALSE)}
\end{center}
This should look completely different from the histogram showing
the distribution of the single sample above.  In fact, what
does this histogram remind you of?
\item If you said, ``a normal p.d.f.,'' good!!  If you 
said something different, go back and look again.
\item Now, which normal density is it?  Well,  by Theorem~\ref{IIDFacts}
from the text above, if 
we have 100 sampled values $Y_1, \ldots, Y_{100}$ from any distribution
where $E(Y_i) = \mu$ for all $i$, and $V(Y_i) = \sigma^2$ for all $i$, and
$$\overline{Y} = \frac{1}{100}(Y_1 + \cdots + Y_{100})$$
is the sample mean, then we will have
$$E(\overline{Y}) = \mu$$
and 
$$V(\overline{Y}) = \frac{\sigma^2}{100},$$
so the SD of the sample means will be $\frac{\sigma}{10}$.   
For the exponential random variables we are looking at, $\mu = 3$ and 
$\sigma = 9$.  Hence the expected value of the sample means is also 
$3$ and the SD of the sample means is $.3$.
\item  Let's overlay that normal density and see how well things match:
\vskip 10pt
\begin{center}
{\tt curve(dnorm(x,3,.3),add=TRUE)}
\end{center}
(The {\tt dnorm} is the normal p.d.f.  the 3 is the $\mu$ and the .3 is the $\sigma$.  You
will use other normal densities in the questions below.  They are specified in the same way.)
This should match quite well (not perfectly, though!)
\end{enumerate}

\subsection*{Assignment} 
Repeat the parts of the worked example below for the means of 1000 random
samples of size 100 from each of the following distributions.  Note:  In
each case to find the appropriate normal density curve, you will need to 
recall facts about expected values and variances of the $Y_i$ individually,
then use Theorem~\ref{IIDFacts}.
\vskip 10pt
\begin{enumerate}
\item[(A)] A uniform distribution with $min = 4$ and $max = 10$.  (These
are the endpoints of the interval where the uniform density different
from zero.)   Use {\tt runif(100,4,10)} to generate a random sample
of size 100 from this uniform distribution.
See Theorem~\ref{UniformEVVar}  in the text for the appropriate $\mu$ and $\sigma^2$ in this case.
Then apply Theorem~\ref{IIDFacts} as in the example above.  
\item[(B)]  A Poisson distribution with $\lambda = 2$.  From the discussion in the text, we know
$\mu = \lambda$ and $\sigma^2 = \lambda$ for Poisson random variables.
The appropriate R function to generate the samples of size 100 is 
{\tt rpois(100,2)}.  
\end{enumerate}
What is happening in each case?  Explain.  Is the distribution of the sample
means approximately normal every time?  
\index{R@{\tt R} statistical package|)}
\index{Central Limit Theorem|)}



\begin{xcb}{Chapter Exercises}
\vskip 5pt
\begin{enumerate}
\item
\label{ex:Dice}
Show that if all rolls of two distinguishable dice are equally likely, then  
the probability of having a 5 or a 6 on at least one of the dice is $5/9$.  
Hint:  Count how many pairs $(m,n)$ have $m = 5,6$, or $n = 5,6$ or both.  
\vskip 5pt

\item 
\label{ex:AdditionLaw}
In the text we used the fact that if $S = S_1 \cup S_2$ and $S_1\cap S_2 = \emptyset$,
then $P(A) = P(A \cap S_1) + P(A \cap S_2)$.  Explain why the following more general statement
should hold.  Suppose $A = A_1 \cup A_2$, but we do not require $A_1 \cap A_2 = \emptyset$.
Then 
$$P(A) = P(A_1 \cup A_2) = P(A_1) + P(A_2) - P(A_1\cap A_2).$$
Hint: A Venn diagram of the union $A_1 \cup A_2$ showing a possibly nonempty $A_1 \cap A_2$
will explain what is going on here.  

\item In the situation of Example~\ref{surgery}, what is the probability that all 5 surgeries are successful if
$p = .9$?  What if $p = .7$?  What if $p = .5$?  \vskip 5pt

\item Compute the values $P(X = k)$ for all $0 \le k \le 10$ in Example~\ref{trout} and compare with Figure~\ref{fig:TroutPMF}.
\vskip 5pt


\item Suppose an urn contains $n_1 = 20$ red balls and $17$ black balls.  If $n = 5$ balls
are chosen from the urn at random, what is the probability that exactly 2 of the balls are red, \vskip 5pt
\begin{enumerate}
\item[(a)] assuming the selection is done \emph{with replacement}, then\vskip 5pt
\item[(b)] assuming the selection is done \emph{without replacement}.
\end{enumerate}
\vskip 5pt


\item  We toss a fair die repeatedly.\vskip 5pt
\begin{enumerate}
\item[(a)]  What is the probability the first 6 comes up on the 21st toss?  \vskip 5pt
\item[(b)]  What is the probability that first 6 comes up sometime on or after the 21st toss?
\end{enumerate}
\vskip 5pt

\item An oil company will drill a succession of test wells in a given area until a productive well is
found.   Assume that the probability that any particular well will be productive is $.3$ and that
the wells are independent of each other. \vskip 5pt
\begin{enumerate}
\item[(a)]  What is the probability that the company will drill 9 wells before it finds the
first productive one?  \vskip 5pt
\item[(b)]  What is the probability that exactly one well out of 10 wells drilled will be
productive?  Explain why your answers for part (a) and this part are different. 
\end{enumerate}
\vskip 5pt

\item Compute the values of the p.m.f.  for a binomial random variable $Y$ based on $n = 6$
trials with $p = .36$.  Verify the statement that $E(Y) = np = 2.16$ by computing $E(Y)$
directly.
\vskip 5pt

\item (The binomial coefficients and ``Pascal's triangle'')
\index{binomial coefficient!``Pascal's triangle''}
\vskip 5pt
\begin{enumerate}
\item[(a)]  Compute the numbers $\binom{n}{k}$ for $n = 0,1,2,3,4,5$ and $0\le k \le n$ in each case.  Recall that $0! = 1$
by definition.  \vskip 5pt
\item[(b)]  Arrange your results with one row for each $n$ and the values for $\binom{n}{k}$ in order from $k = 0$
on the left to $k = n$ on the right.  You should notice something about how the numbers on the row for one $n$ relate
to the numbers for the next $n$ (apart from the two 1's on the ends).  Try to describe the pattern in words and in an
equation.\vskip 5pt
\item[(c)]  Show using \eqref{eq:BC} that for all $n,k$, 
$$\binom{n+1}{k} = \binom{n}{k} + \binom{n}{k-1}.$$
How does this relate to what you said in part (b)?
\end{enumerate}
\vskip 5pt


\item 
\label{ex:pmfs}
The \emph{probability mass function} (p.m.f.) of a discrete random variable with values in 
\index{probability mass function (p.m.f.)}
$\{0,1,2,3,\ldots\}$ is defined as the function $f(k) = P(X = k)$.  \vskip 5pt
\begin{enumerate}
\item[(a)]  Explain why $f(k) \ge 0$ for all $k$. \vskip 5pt
\item[(b)]  Explain why $\sum_{k=0}^\infty f(k)$ should equal $1$.  Note that when the values $f(k) = 0$
for all sufficiently large $k$ (as is true for instance for binomial random variables), this is just a finite sum.  \vskip 5pt
\item[(c)]  Explain why the equation from part (b):
$$\sum_{k=0}^n \binom{n}{k} p^k q^{n-k} = 1$$
is valid for a binomial p.m.f. 
\end{enumerate}
\vskip 5pt

\item Your front curb extends for 40 feet along the street.  A delivery 
drone drops its cargo at a random point uniformly distributed
along that 40 foot stretch.\vskip 5pt
\begin{enumerate}
\item[(a)] What is the probability that the drop point is between $5$ feet
and $15$ feet along the curb from one endpoint?  \vskip 5pt
\item[(b)]  Give a ``piece-wise'' formula for the c.d.f. $F(x)$ for this random variable. \vskip 5pt
\item[(c)] What is the probability that the drop point is closer to the center
than it is to either endpoint of the 40 foot stretch of curb? \vskip 5pt
\item[(d)] What is the probability that the drop point is within one SD 
of the midpoint?  (See Theorem~\ref{UniformEVVar}.)
\end{enumerate}
\vskip 5pt

\item Consider the p.d.f. for a random variable $Y$ given in Example~\ref{AnotherPDFCDFPair}.
\vskip 5pt
\begin{enumerate}
\item[(a)]  What is the probability $P(1.5 < Y < 2)$?  \vskip 5pt
\item[(b)]  What is the probability $P(Y > 2.5)$?
\end{enumerate}
\vskip 5pt

\item Using the table in Figure~\ref{fig:NormalCurveAreas}, determine each of the following areas
under the standard normal curve.  \vskip 5pt
\begin{enumerate}
\item[(a)]  $P(0 < Z < 1.43)$ \vskip 5pt
\item[(b)]  $P(0.72 < Z < 1.43)$ \vskip 5pt
\item[(c)]  $P(-2.17 < Z < -0.97)$ \vskip 5pt
\item[(d)]  $P(Z < 1.43)$ \vskip 5pt
\item[(e)]  $P(-0.62 < Z < 2.67)$
\end{enumerate}
\vskip 5pt

\item The tail lengths of a species of lizard are normally distributed with mean $\mu = 75$ mm and 
SD $\sigma = 10$ cm.  \vskip 5pt
\begin{enumerate}
\item[(a)]  What is the probability that a randomly caught lizard of this species will 
have a tail length between $70$ and $82$ mm long?  \vskip 5pt
\item[(b)]  What is the probability that the tail length of a randomly caught lizard
will be greater than $68$ mm?  \vskip 5pt
\item[(c)]  If 10 of the lizards are caught independently, what is the probability that
strictly fewer than $3$ of them will have tail lengths greater than $68$ mm?  
(Use your answer from part (b).  How?)  \vskip 5pt
\item[(d)]  Suppose that 10 of the lizards are caught independently.  What
is the probability that the mean tail length for those 10 is greater than $68$ mm?
(Use Theorem~\ref{IIDFacts} and your answer to part (b).)
\end{enumerate}
\vskip 5pt

\item (Optional) This exercise requires some knowledge of limits.
\index{infinite geometric series}
\index{geometric series!infinite}
\label{ex:InfGeomSeries}
Let $a$ be a real number at let $r$ be a real number satisfying $-1 < r < 1$, or equivalently, $|r|<1$.  In this
exercise, you will show that the infinite geometric series
\begin{equation}
\label{eq:InfGS}
\sum_{n = 0}^\infty a r^n = a + ar + ar^2 + ar^3 + \cdots\ 
\end{equation}
with first term $a$ and ratio $r$ between successive terms 
converges to (i.e. adds up to)    $\frac{a}{1 - r}$.  
\vskip 5pt
\begin{enumerate}
\item[(a)]  Convergence of infinite series is defined using the finite \emph{partial sums}, here
$$S_N = \sum_{n=0}^N a r^n = a + ar + ar^2 + \cdots + a r^N.$$  
Show that this partial sum equals $\frac{a(1 - r^{N+1})}{1 - r}$.  (Hint:  Compute $S_N - r S_N$ and note
the cancellations.) \vskip 5pt
\item[(b)]  What is true about $\lim_{N\to\infty} r^{N+1}$ if $-1 < r < 1$?  \vskip 5pt
\item[(c)]  Use your answer to part (b) to show 
$$\lim_{N\to\infty} S_N = \frac{a}{1 - r}.$$
This is what is meant by saying that the infinite series \eqref{eq:InfGS} converges to $\frac{a}{1 - r}$.  
\vskip 5pt
\item[(d)]  Determine the sum of the infinite series 
$$\sum_{k = 0}^\infty \frac{3}{4^n}.$$ 
\item[(e)]  Same question as part (d) for the infinite series
$$\sum_{k=0}^\infty \frac{(-1)^n 4^n}{5^n}.$$
\end{enumerate}
\vskip 5pt

\item 
\label{ex:PoissonDerivation}
(Optional) This exercise requires  knowledge of limits, and in particular the limit formula $\lim_{n\to\infty} \left( 1 + \frac{x}{n}\right)^n = e^x$.
\begin{enumerate}
\item[(a)] Start with the probability mass function for binomial random variable $Y$ based on $n$ trials and success probability $p$.
Write $\lambda = np$ and consider
$$P(Y = k) = \binom{n}{k} = \frac{n(n-1)\cdots (n - k + 1)}{k!} \left(\frac{\lambda}{n}\right)^k \left(1 -\frac{\lambda}{n}\right)^{n-k}.$$
Show that this can be rearranged to the form 
$$\frac{\lambda^k}{k!} \cdot \left(1 - \frac{\lambda}{n}\right)^n \cdot \left(1 - \frac{\lambda}{n}\right)\cdot \left(1 - \frac{1}{n}\right)\cdots \left(1 - \frac{k-1}{n}\right).$$
\item[(b)]  Now take the limit as $n \to \infty$ and use the limit formula given in the introduction to the problem to show that 
$$\lim_{n\to\infty} P(Y = k) = \frac{\lambda^k}{k!} e^{-\lambda}.$$
This is the Poisson p.m.f. given in the text.  
\end{enumerate}  
\vskip 5pt

\item \label{ex:CantorDiagonal}  (Optional)  This exercise sketches a proof of the fact that the set of real
\index{Cantor diagonal argument}
numbers in the interval $[0,1]$, namely the set of all real numbers $x$ with $0 \le x \le 1$, is not a 
countably infinite set.  The idea is known as the ``Cantor diagonal argument.''   First recall that any real 
number with $0 \le x \le 1$ can be written as a decimal fraction of the form 
$$x = .d_1d_2d_3\cdots $$
where $d_i$ are the decimal digits (all equal to $0,1,2,\ldots,8$, or $9$) of the number.  Also
recall that the decimal expansions of some rational numbers have either \emph{terminating}
or \emph{infinite repeating} expansions.  For instance
$$\frac{1}{2} = .5 = .4\overline{9} = .49999\cdots .$$
The proof will be by contradiction.  Assume that \emph{there is a one-to-one correspondence} between the natural numbers
$\mathbb{N}$ and the numbers in $[0,1]$ given, for instance, by 
\begin{align}
\label{eq:List} \nonumber
1 &\longleftrightarrow .d_{11}d_{12}d_{13} \cdots \\ 
2 &\longleftrightarrow .d_{21}d_{22}d_{23}\cdots \\ \nonumber
3 &\longleftrightarrow .d_{31}d_{32}d_{33}\cdots \\ \nonumber
   & \quad \vdots
\end{align}
In this listing, assume that the numbers with terminating decimal expansions are listed by their
terminating forms with all zero digits after some point.  Now consider the number with a decimal expansion 
$x = .e_1e_2e_3 \cdots $ that is obtained
in this way:  $e_1$ is any digit \emph{different from $d_{11}$ in the first number in \eqref{eq:List}, and also different from $9$},
then $e_2$ is any digit \emph{different from $d_{22}$ in the second number in \eqref{eq:List}, and also different from $9$},
then $e_3$ is any digit \emph{different from $d_{33}$ in the third number in \eqref{eq:List}, and also different from $9$}, 
and so forth.  Explain why $x$ cannot equal any of the numbers in \eqref{eq:List}.  But on the other hand $0\le x \le 1$
and \eqref{eq:List} was supposed to be a complete list of all the numbers in $[0,1]$.  This contradiction shows
no list like \eqref{eq:List} can exist(!)  \vskip 5pt


\end{enumerate}
\end{xcb}

\endinput
