Search in book...
Toggle Font Controls
Create new playlist

Name your new playlist

Playlist description (optional)
Sign In

Email address

Password

Forgot Password?

or

Continue with Facebook

Continue with Google
Sign Up

Full Name

Email address

Confirm Email Address

Password

or

Continue with Facebook

Continue with Google

3
MOMENTS AND GENERATING FUNCTIONS

3.1 INTRODUCTION

The study of probability distributions of a random variable is essentially the study of some numerical characteristics associated with them. These so-called parameters of the distribution play a key role in mathematical statistics. In Section 3.2 we introduce some of these parameters, namely, moments and order parameters, and investigate their properties. In Section 3.3 the idea of generating functions is introduced. In particular, we study probability generating functions, moment generating functions, and characteristic functions. Section 3.4 deals with some moment inequalities.

3.2 MOMENTS OF A DISTRIBUTION FUNCTION

In this section we investigate some numerical characteristics, called parameters, associated with the distribution of an RV X. These parameters are (a) moments and their functions and (b) order parameters. We will concentrate mainly on moments and their properties.

Let X be a random variable of the discrete type with probability mass function p_k = P{X = x_k}, k = 1, 2, …. If

(1)

we say that the expected value (or the mean or the mathematical expectation) of X exists and write

(2)

Note that the series may converge but the series may not. In that case we say that EX does not exist.

Example 1.

Let X have the PMF given by

Then

and EX does not exist, although the series

is convergent.

If X is of the continuous type and has PDF f, we say that EX exists and equals , provided that

A similar definition is given for the mean of any Borel-measurable function h(X) of X. Thus, if X is of the continuous type and has PDF f, we say that Eh(X) exists and equals , provided that

We emphasize that the condition must be checked before it can be concluded that EX exists and equals . Moreover, it is worthwhile to recall at this point that the integral exists, provided that the limit exists. It is quite possible for the limit to exist without the existence of . As an example, consider the Cauchy PDF:

Clearly

However, EX does not exist since the integral diverges.

Remark 1. Let X(ω) = I_A(ω) for some A∈ . Then EX = P(A).

Remark 2. If we write h(X) = |X|, we see that EX exists if and only if E|X| does.

Remark 3. We say that an RV X is symmetric about a point α if

In terms of DF F of X, this means that, if

holds for all x ∈, we say that the DF F (or the RV X) is symmetric with α as the center of symmetry. If α = 0, then for every x

In particular, if X is an RV of the continuous type, X is symmetric with center α if and only if the PDF f of X satisfies

If α= 0, we will say simply that X is symmetric (or that F is symmetric).

As an immediate consequence of this definition we see that, if X is symmetric with α as the center of symmetry and E|X| < ∞, then EX = a. Asimple example of a symmetric distribution is the Cauchy PDF considered above (before Remark 1). We will encounter many such distributions later.

Remark 4. If a and b are constants and X is an RV with E|X| < ∞, then E|aX + b| < ∞ and E{aX + b} = a EX + b. In particular, E{X −μ} = 0, a fact that should not come as a surprise.

Remark 5. If X is bounded, that is, if P{X < M} = 1, 0 < M < ∞, then EX exists.

Remark 6. If {X ≥ 0}= 1, and EX exists, then EX ≥ 0.

Theorem 1.

Let X be an RV, and g be a Borel-measurable function on . Let Y = g(X). If X is of discrete type then

(3)

in the sense that, if either side of (3) exists, so does the other, and then the two are equal. If X is of continuous type with PDF f then in the sense that, if either of the two integrals converges absolutely, so does the other, and the two are equal.

Remark 7. Let X be a discrete RV. Then Theorem 1 says that

in the sense that, if either of the two series converges absolutely, so does the other, and the two sums are equal. If X is of the continuous type with PDF f, let h(y) be the PDF of Y = g(X). Then, according to Theorem 1,

provided that E|g(X)| < ∞.

Proof of Theorem 1. In the discrete case, suppose that P{X ∈ A} = 1. If y = g(x) is a one-to-one mapping of A onto some set B, then

We have

In the continuous case, suppose g satisfies the conditions of Theorem 2.5.3. Then

by changing the variable to y = g(x). Thus

The functions h(x) = xⁿ, where n is a positive integer, and h(x) = |x|^α, where α is a positive real number, are of special importance. If EXⁿ exists for some positive integer n, we call EXⁿ the nth moment of (the distribution function of) X about the origin. If E|X|^α < ∞ for some positive real number α, we call E|X|^α the α th absolute moment of X. We shall use the following notation:

(4)

whenever the expectations exist.

Theorem 3.

Let X be an RV on a probability space (Ω, ,P). Let E|X|^k < ∞ for some k > 0. Then

Proof. We provide the proof for the case in which X is of the continuous type with density f. We have

It follows that

But

completing the proof.

Remark 8. Probabilities of the type P{|X| > n} or either of its components, P{X > n} or P{X < −n}, are called tail probabilities. The result of Theorem 3, therefore, gives the rate at which P{|X| > n} converges to 0 as n →∞.

Remark 9. The converse of Theorem 3 does not hold in general, that is,

doesnot necessarily imply that E|X|^k < ∞, for the RV

where c is a constant determined from

We have

and nP{X > n} → 0 as n → ∞. (Here and subsequently ≈ means that the ratio of two sides → 1 as n → ∞.) But

In fact, we need

for some δ > 0 to ensure that E|X|^k < ∞. A condition such as this is called a moment condition.

For the proof we need the following lemma.

Lemma 1. Let X be a nonnegative RV with distribution function F. Then

(5)

in the sense that, if either side exists, so does the other and the two are equal.

Proof. If X is of the continuous type with density f and EX < ∞, then

On integration by parts we obtain

But

and, since E|X| < ∞, it follows that

We have

If , then

and it follows that E|X| < ∞.

We leave the reader to complete the proof in the discrete case.

Corollary. For any RV X, E|X| < ∞ if and only if the integrals and both converge, and in that case

Actually we can get a little more out of Lemma 1 than the above corollary. In fact,

and we see that an RV X possesses an absolute moment of order α > 0 if and only if is integrable over (0,∞).

A simple application of the integral test leads to the following moments lemma.

Lemma 2.

(6)

Note that an immediate consequence of Lemma 2 is Theorem 3. We are now ready to prove the following result.

Theorem 4.

Let X be an RV with a distribution satisfying as n → ∞

for some α > 0. Then E|X|^β < ∞ for 0 < β < α.

Proof. Given ε>0, we can choose an N = N(ε) such that

It follows that for 0 < β < α

Remark 10. Using Theorems 3 and 4, we demonstrate the existence of random variables for which moments of any order do not exist, that is, for which E|X|^α = ∞ for every α > 0. For such an RV n^αP{|X| >n}0 as n→∞ for any α > 0. Consider, for example, the RV X with PDF

The DF of X is given by

Then for x > e

and x^αP{|X| > x}→∞ as x→∞ for any α > 0. It follows that E|X|^α = ∞ for every α > 0. In this example we see that P{|X| > cx}/P{|X| > x} → 1 as x → ∞ for every c > 0. A positive function L(∙) defined on (0,∞) is said to be a function of slow variation if and only if L(cx)/L(x)→1 as x→∞ for every c > 0. For such a function x^αL(x)→∞ for every α > 0 (see Feller [26, pp. 275-279]). It follows that, if P{|X| > x} is slowly varying, E|X|^α = ∞ for every α>0. Functions of slow variation play an important role in the theory of probability.

Random variables for which P{|X| > x} is slowly varying are clearly excluded from the domain of the following result.

PROBLEMS 3.2

Find the expected number of throws of a fair die until a 6 is obtained.
From a box containing N identical tickets numbered 1 through N, n tickets are drawn with replacement. Let X be the largest number drawn. Find EX.
Let X be an RV with PDF
where c = Γ(m)/[Γ(1/2)Γ(m−1/2)]. Show that EX^2r exists if and only if 2r < 2m −1. What is EX^2r if 2r < 2m −1?
Let X be an RV with PDF

Show that E|X|^α < ∞ for α< k. Find the quantile of order p for the RV X.
Let X be an RV such that E|X| < ∞. Show that E|X −c| is minimized if we choose c equal to the median of the distribution of X.
Pareto's distribution with parameters α and β (both α and β positive) is defined by the PDF

Show that the moment of order n exists if and only if n < β. Let β > 2 Find the mean and the variance of the distribution.
For an RV X with PDF

show that moments of all order exist. Find the mean and the variance of X.
For the PMF of Example 5 show that

and

where 0 ≤ p ≤ 1, q = 1 − p.
For the Poisson RV X with PMF

Show that and
For any RV X with E|X|⁴ < ∞ define

Here α₃ is known as the coefficient ofskewness and is sometimes used as a measure of asymmetry, and α₄ is known as kurtosis and is used to measure the peakedness ("flatness of the top") of a distribution. Compute α₃ and α₄ for the PMFs of Problems 8 and 9.
For a positive RV X define the negative moment of order n by EX⁻ⁿ, where n > 0 is an integer. Find for the PMFs of Example 5 and Problem 9.
Prove Theorem 6.
Prove Theorem 7.
In each of the following cases, compute EX, var(X), and EXⁿ (for n > 0, an integer) whenever they exist.
1. f(x) = 1, −1/2 ≤ x ≤ 1/2, and 0 elsewhere.
2. f(x) = e^−x, x ≥ 0, and 0 elsewhere.
3. f(x) = (k–1)/x^k, x ≥ 1, and 0 elsewhere; k >1 is a constant.
4. f(x) = 1/[π(1+x²)], −∞ <x < ∞.
5. f(x) = 6x(1−x), 0 < x < 1, and 0 elsewhere.
6. f(x) = xe^−x, x >0, and 0 elsewhere.
7. P(X = x) = p(1−p)^x−1, x = 1, 2, …, and 0 elsewhere: 0 <p <1.
Find the quantile of order p(0 < p < 1) for the following distributions.
1. f(x) = 1 /x², x ≥ 1, and 0 elsewhere.
2. f(x) = 2xexp(−x²), x > 0, and 0 otherwise.
3. f(x) = 1/θ, 0 ≤ x ≤ θ, and 0 elsewhere.
4. P(X = x) = θ(1— θ)^x−1, x = 1, 2, …, and 0 otherwise; 0 < θ < 1.
5. f(x) = (1/β²)x exp(−x/β), x > 0, and 0 otherwise; β > 0.
6. f(x) = (3/b³)(b−x)²,0 < x < b, and 0 elsewhere.

3.3 GENERATING FUNCTIONS

In this section we consider some functions that generate probabilities or moments of an RV. The simplest type of generating function in probability theory is the one associated with integer-valued RVs. Let X be an RV, and let^.

with .

Example 2.

Let X be an RV with geometric distribution, that is, let

Then

Remark 1. Since P(1) = 1, series (1) is uniformly and absolutely convergent in |s| ≤ 1 and the PGF P is a continuous function of s. It determines the PGF uniquely, since P(s) can be represented in a unique manner as a power series.

Remark 2. Since a power series with radius of convergence r can be differentiated termwise any number of times in (−r, r), it follows that

where P^(k) is the kth derivative of P. The series converges at least for −1 < s < 1. For s = 1 the right side reduces formally to E{X(X −1)(X−k+1)}which is the kth factorial moment of X whenever it exists. In particular, if EX < ∞ then P'(1) = EX, and if EX² < ∞ then P"(1) = EX(X −1) and Var(X) = EX²−(EX)² = p"(1)−[P'(1)]²+P'(1).

Example 10.

Let X be a normal RV with PDF

Then

Note that’ sin tx is an odd function and so also is sin tx e^–x2/². Thus the second integral on the right-side vanishes and we have

Remark 4. Unlike an MGF which may not exist for some distributions, a CF always exists which makes it a much more convenient tool. In fact, it is easy to see that ϕ is continuous on , |ϕ (t)| ≤ 1 for all t, and where is the complex-conjugate of ϕ. Thus is the CF of −X. Moreover, ϕ uniquely determines the DF of RV X. For these and many other properties of characteristic functions we need a comprehensive knowledge of complex variable theory, well beyond the scope of this book. We refer the reader to Lukacs [69].

Finally, we consider the problem of characterizing a distribution from its moments Given a set of constants the problem of moments asks if they can be moments of a distribution function F. At this point it will be worthwhile to take note of some facts.

First, we have seen that if the M(s) = Ee^sX exists for some X for s in some neighborhood of zero, then for all n ≥ 1. Suppose, however, that for all n ≥ 1. It does not follow that the MGF of X exists.

PROBLEMS 3.3

Find the PGF of the; RVs with the following PMFs:
1. .
2. .
3. .
Let X be an integer-valued RV with PGF P(s). Let α and β be nonnegative integers, and write . Find the PGF of Y.
Let X be an integer-valued RV with PGF P(s), and suppose that the mgf M(s) exists for . How are M(s) and P(s) related? Using for positive integral k, find EX^k in terms of the derivatives of P(s) for values of k = 1,2,3,4.
For the Cauchy PDF

does the MGF exist?
Let X be an RV with PMF

Set P{X > j} = q_j,j = 0, 1, 2, …. Clearly q_j= p_j+p_j+₂+…, j ≥ 0. Write Then the series for Q(s) converges in |s| < 1. Show that

where P(s) is the PGF of X. Find the mean and the variance of X(when they exist) in terms of Q and its derivatives.
For the PMF

where and find the PGF and the MGF in terms of f.
For the Laplace PDF

show that the MGF exists and equals
For any integer-valued RV X, show that

where P is the PGF of X.
Let X be an RV with MGF M(t), which exists for t ∈ . Show that

for any fixed s, 0 < s < t₀, and for each integer n ≥ 1. Expanding e^tx in a power series, show that, for ,

(Since a power series can be differentiated term by term within the interval of convergence, it follows that for |t| < s,

for each integer k ≥ 1.) (Roy, LePage, and Moore [95])
Let X be an integer-valued random variable with

Show that X must be degenerate at n.
[Hint: Prove and use the fact that if EX^k < ∞ for all k, then

Write P(s) as
Let p(n, k) = f (n, k) /n! where f (n, k) is given by

Let

be the probability generating function of p(n, k). Show that

(P_n is the generating function of Kendall's τ-statistic.)
For k = 0, 1, …, let u_n (k) be defined recursively by

with u₀(0) = 1, u₀(k) = 0 otherwise, and u_n(k) = 0 for k < 0. Let be the generating function of {u_n}. Show that

If p_n(k) = u_n(k)/2ⁿ, find {p_n(k)} for n = 2, 3, 4. (P_n is the generating function of one-sample Wilcoxon test statistic.)

3.4 SOME MOMENT INEQUALITIES

In this section we derive some inequalities for moments of an RV. The main result of this section is Theorem 1 (and its corollary), which gives a bound for tail probability in terms of some moment of the random variable.

PROBLEMS 3.4

For the RV with PDF

where λ ≥ 0 is an integer, show that
Let X be any RV, and suppose that the MGF of X, M(t) = Ee^tx, exists for every t > 0. Then for any t > 0
Construct an example to show that inequalities (4) and (5) cannot be improved.
Let g(.) be a function satisfying g(x) > 0 for x > 0, g(x) increasing for x > 0, and . Show that
Let X be an RV with EX = 0, var(X) = σ², and EX⁴ = μ₄. Let K be any positive real number. Show that

In other words, show that bound (7) is better than bound (3) if and worse if 1 ≤ . Construct an example to show that the last inequalities cannot be improved.

.
Use Chebychev's inequality to show that for any k > 1, e^k⁺¹ ≥ k².
For any RV X, show that

where .
Let X be an RV such that P(a ≤ X ≤ b) = 1 where –∞ < a < b < ∞. Show that var(X) ≤ (b – a)²/4.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.

Table of Contents for 3 MOMENTS AND GENERATING FUNCTIONS

Create new playlist

Sign In

Sign Up

3.1 INTRODUCTION

3.2 MOMENTS OF A DISTRIBUTION FUNCTION

PROBLEMS 3.2

3.3 GENERATING FUNCTIONS

PROBLEMS 3.3

3.4 SOME MOMENT INEQUALITIES

PROBLEMS 3.4

Table of Contents for
3 MOMENTS AND GENERATING FUNCTIONS