Mathematics 375 – Probability and Statistics 1 Discussion 2 – Moment Generating Functions October 15, 2009

Background

For a discrete random variable Y, we have introduced the expected value,

E(Y) = ∑_{values y} yP (Y = y).

We have seen that the variance can be computed in terms of E(Y) and

E(Y^{2}) = ∑_{values y} y^{2}P (Y = y),

since V (Y) = E(Y^{2}) − (E(Y))^{2}. The two quantities E(Y) and E(Y^{2}) are two early terms in the sequence of moments of Y about the origin, which are defined in general as follows: For each k ≥ 1, the kth moment about the origin is

E(Y^{k}) = ∑_{values y} y^{k}P (Y = y).

We can also extend this to k = 0 by setting E(Y^{0}) = E(1) = 1.

When dealing with a sequence of numbers like the E(Y^{k}), k ≥ 0, it is common in mathematics to try to “package” the sequence into a single object we can deal with using techniques for functions defined by infinite series. The way we will do this is to form the so called moment (exponential) generating function:

(1)  m_{Y} (t) = ∑_{k=0}^{∞} E(Y^{k})\frac{t^{k}}{k!} .

Discussion Questions

A) For the purposes of this question, you can take it as given that the expected value operator will behave in the same linear fashion for infinite sums of functions of Y that we have seen with finite sums.

1) Assuming that, show that

m(t) = E(e^{tY}),

where e^{tY} is treated as a function of the random variable Y, depending on the parameter t. (Hint: What does the Taylor series of e^{u} look like?)

2) Show that if you know m_{Y} (t) as a function of t, then you can recover all the moments of Y about the origin by differentiating m_{Y} (t) sufficiently often. The kth moment of Y is the value at t = 0 of the kth derivative of m_{Y} (t):

E(Y^{k}) = m_{Y}^{(k)}(0).

(This is one reason why putting the factorials in the denominator in (1) is a good thing to do! Without them, we would have to keep track of extra factorials in the derivatives.)

B) Let Y have a binomial distribution based on n trials with success probability p. By computing E(e^{tY}) from the definition of the expected value, show that

m_{Y} (t) = (pe^{t} + q)^{n}.

Also, use this to compute E(Y) and E(Y^{2}) and check our formula for V (Y) in this case.

C) Let Y have a geometric distribution with success probability p. By computing E(e^{tY}) from the definition of the expected value, show that

m_{Y} (t) = \frac{pe^{t}}{1 − qe^{t}} .

The main importance of moment generating functions comes from the following Uniqueness Theorem:

Theorem. Let X and Y be two (discrete) random variables and assume m_{X}(t) and m_{Y} (t) both exist and are equal. Then X and Y have the same distribution (that is, X and Y have the same probability mass function).

D) Suppose you have random variables U and W whose moment generating functions look like this:

m_{U}(t) = \frac{.45e^{t}}{1 − (.55)e^{t}}  and  m_{W} (t) = (.7 + .3e^{t})^{12}.

What can you say about the distributions of U and W?

E) Say m_{Z}(t) = \frac{1}{6}e^{t} + \frac{1}{3}e^{2t} + \frac{1}{2}e^{3t}.

1) Compute E(Z) and V (Z) from the information in the moment generating function.

2) What is the distribution of Z?

Assignment

Group write-ups due in class on Tuesday, October 20.