Is a random variable that can assume an infinite number of values in an interval between two 2 specific values?

1.2.3 Continuous Random Variables

Discrete random variables have a set of possible values that are either finite or countably infinite. However, there exists another group of random variables that can assume an uncountable set of possible values. Such random variables are called continuous random variables. Thus, we define a random variable X to be a continuous random variable if there exists a nonnegative function fX(x), defined for all real x∈(−∞,∞), having the property that for any set A of real numbers,

P[X∈A]=∫AfX(x)dx

The function fX(x) is called the probability density function (PDF) of the random variable X and is defined by

fX(x)=dFX(x)dx

This means that

FX (x)=∫−∞xfX(u)du

Geeta Distribution

A discrete random variableX is said to have Geeta distribution with parameters θ and α if

f(x)=1αx−1(αx−1x)θx−1(1−θ)αx−x,x=1,2,…,0<θ<1,1<α<θ−1.

The distribution is L-shaped and unimodal with

μ=(1−θ ˆ)(1−αθ)−1

and

μ 2=(α−1)θ(1−θ)(1−αθ)−3.

A recurrence relation is satisfied by the central moments of the form

μr+1=θμdμrdθ+rμ2μr−1 ,r=1,2,….

Estimation of the parameters can be done by the method of moments or by the maximum likelihood method. Moment estimates are

μˆ=x¯ and αˆ=S 2−x¯(x¯−1) S2−x¯2(x¯−1)

with x¯ and s2 being the sample mean and variance. The maximum likelihood estimates are

μˆM=x¯

and αˆ is iteratively obtained by solving the equation

(α−1)x¯αx¯−1=exp⁡[−1nx¯∑x=2k∑i=2xxfxx−i]

with n= ∑x=1kfx as the sample size and fx is the frequency of x=1,2,…,k. For details, see Consul (1990). As the Geeta model is a member of the MPSD, the reliability properties can be readily obtained from those of the MPSD discussed in Section 3.2.2.

8.1.3 Poisson distribution

The discrete random variableX follows the Poisson distribution with parameter λ>0, if

P{X=k}=λk k!e-λ,k=0,1,…,

and we shall denote it by X∼P(λ). It is easy to show that

EX=VarX=λ,

and so

VarXEX=1.

The distribution function of X is

(8.4)P{X⩽m }=∑k=0mλkk!e-λ=1-Iλ(m+1),

where

Ix(f)=1Γ(f)∫0xtf-1e-tdt,x>0

is the incomplete gamma function. Often, for large values of λ, to compute (8.4), we can use a normal approximation

P{X⩽m}=Φm+0.5-λλ+O1λ,λ→∞.

Let { Xn}n=1∞ be a sequence of independent and identically distributed random variables following the same Bernoulli distribution with parameter p,0<p<1, with

P{Xi=1}=p ,P{Xi=0}=q=1-p.

Let

μn=X1+⋯+Xn,Fn(x)=Pμn-npnpq⩽x,x∈R1.

Then, uniformly for x∈R1, we have

Fn(x)→Φ(x)=12π∫-∞x e-t2/2dt,n→∞.

From this result, it follows that for large values of n,

Pμn-npnpq ⩽x≈Φ(x).

Often this approximation is used with the so-called continuity correction given by

P μn-np+0.5npq⩽x≈Φ(x).

We shall now describe the Poisson approximation to the binomial distribution. Let {μn} be a sequence of binomial random variables, μn∼B(n,pn), 0<pn<1, such that

npn→λif n→∞andλ>0.

Then,

limn→∞ P{μn=m|n,pn}= λmm!e-λ.

In practice, this means that for “large” values of n and “small” values of p, we may approximate the binomial distribution B(n,p) by the Poisson distribution with parameter λ=np, that is,

P{μn=m|n,p}≈λmm!e-λ.

It is of interest to note that (Hodges and Le Cam, 1960)

supx∑m=0xnmpm(1-p)n-m-∑m=0xλmm !e-λ⩽Cn,whereC⩽3λ.

Hence, if the probability of success in Bernoulli trials is small, and the number of trials is large, then the number of observed successes in the trials can be regarded as a random variable following the Poisson distribution.

Section 5.7 Covariance and Correlation Coefficient

5.20

Two discrete random variablesX and Y have the joint PMF given by

pXYxy =0x=−1,y=013x=−1,y=113x=0,y=00x=0, y=10x=1,y=013x=1,y=1

Are X and Y independent?

What is the covariance of X and Y?

Two events A and B are such that PA=14,PB|A=12 and PA|B=14. Let the random variable X be defined such that X = 1 if event A occurs and X = 0 if event A does not occur. Similarly, let the random variable Y be defined such that Y = 1 if event B occurs and Y = 0 if event B does not occur.

Find E[X] and the variance of X

Find E[Y] and the variance of Y.

Find ρXY and determine whether or not X and Y are uncorrelated.

A fair die is tossed three times. Let X be the random variable that denotes the number of 1’s and let Y be the random variable that denotes the number of 3’s. Find the correlation coefficient of X and Y.

2.4.1 Random variables

For a discrete random variableX with distinct values such as the number of cars passing through a junction, each value xi may occur with certain probability p(x i). In other words, the probability varies and is associated with the corresponding random variable. Traditionally, an uppercase letter such as X is used to denote a random variable, whereas a lowercase letter such as xi represents its values. For example, if X means a coin-flipping event, then xi=0 (tail) or 1 (head). A probability function p(xi) is a function that assigns probabilities to all the discrete values xi of the random variable X.

As an event must occur inside a sample space, the requirement that all the probabilities must be summed to one, which leads to

(2.33)∑i=1np(xi)=1.

For example, the outcomes of tossing a fair coin form a sample space. The outcome of a head (H) is an event with probability P(H)=1/2, and the outcome of a tail (T) is also an event with probability P(T)=1 /2. The sum of both probabilities should be one, that is,

(2.34)P(H)+P(T)=12+12=1.

The cumulative probability function of X is defined by

(2.35)P(X≤x)=∑xi<xp(xi) .

Two main measures for a random variable X with given probability distribution p(x) are its mean and variance. The mean μ or expectation of E[X] is defined by

(2.36)μ≡E[X]≡<X>=∫xp(x)dx

for a continuous distribution and the integration is within the integration limits. If the random variable is discrete, then the integration becomes the weighted sum

(2.37)E[X]=∑i xip(xi).

The variance var[ X]=σ2 is the expectation value of the deviation squared, that is, E[(X−μ) 2]. We have

(2.38)σ2≡var[X]=E[(X−μ)2]=∫(x−μ) 2p(x)dx.

The square root of the variance σ=var[X] is called the standard deviation, which is simply σ.

The above definition of mean μ=E[X] is essentially the first moment if we define the kth moment of a random variable X (with a probability density distribution p(x)) by

(2.39)μk≡E[Xk]=∫xkp(x)dx(k=1,2,3,…).

Similarly, we can define the kth central moment by

(2.40)νk ≡E[(X−E[X])k]≡E [(X−μ)k]=∫(x−μ)kp(x)dx(k=0, 1,2,3,…),

where μ is the mean (the first moment). Thus, the zeroth central moment is the sum of all probabilities when k=0, which gives ν0=1. The first central moment is ν1=0. The second central moment ν2 is the variance σ2, that is, ν2=σ2.

4.4 The normal approximation to the binomial distribution

We know that a binomial random variable Y, with parameters n and P = P(success), can be viewed as the number of successes in n trials and can be written as:

Y=∑i=1nXi

where

Xi={1with probabilityp0withprobability(1−p).

The fraction of successes in n trials is:

Yn=1n∑i=1nXi=X¯.

Hence, Y/n is a sample mean. Since E(Xi) = P and Var (Xi) = P(1 – P), we have:

E(Yn)=E(1n∑i=1nXi)=1nnp=p

and

Var(Yn)=1n2∑i =1nVar(Xi)=p(1−p)n.

Because Y=nX¯, by the central limit theorem, Y has an approximate normal distribution with mean μ = np and variance σ2 = np(1 – P). Because the calculation of the binomial probabilities is cumbersome for large sample sizes n, the normal approximation to the binomial distribution is widely used. A useful rule of thumb for use of the normal approximation to the binomial distribution is to make sure n is large enough if np ≥ 5 and n(1 – P) ≥ 5. Otherwise, the binomial distribution may be so asymmetric that the normal distribution may not provide a good approximation. Other rules, such as np ≥ 10 and n(1 – P) ≥ 10, or np(1 – P) ≥ 10, are also used in the literature. Because all of these rules are only approximations, for consistency's sake we will use np ≥ 5 and n(1 – P) ≥ 5 to test for largeness of sample size in the normal approximation to the binomial distribution. If the need arises, we could use the more stringent condition np(1 – P) ≥ 10.

Recall that discrete random variables take no values between integers, and their probabilities are concentrated at the integers as shown in Fig. 4.7. However, the normal random variables have zero probability at these integers; they have nonzero probability only over intervals. Because we are approximating a discrete distribution with a continuous distribution, we need to introduce a correction factor for continuity which is explained next.

Correction for continuity for the normal approximation to the binomial distribution

To approximate P(X ≤ a) or P(X > a), the correction for continuity is (a + 0.5), that is,

P(X≤a)=P(Z<(a+0.5)−npnp(1−p) )

and

P(X>a)=P(Z> (a+0.5)−npnp(1−p)).

To approximate P(X ≥ a) or P(X < a), the correction for continuity is (a − 0.5), that is,

P( X≥a)=P(Z>(a−0.5) −npnp(1−p))

and

P (X<a)=P(Z<(a−0.5)−npnp(1−p)).

To approximate P(a ≤ X ≤ b), treat ends of the intervals separately, calculating two distinct z-values according to steps (a) and (b), that is,

P(a≤X≤b)=P((a−0.5)−npnp(1−p)<Z<(b+0.5)−npnp(1−p)).

Use the normal table to obtain the approximate probability of the binomial event.

The shaded area in Fig. 4.8 represents the continuity correction for P(X = i).

Example 4.4.2

A study of parallel interchange ramps revealed that many drivers do not use the entire length of parallel lanes for acceleration, but seek, as soon as possible, a gap in the major stream of traffic to merge. At one site on Interstate Highway 75, 46% of drivers used less than one-third of the lane length available before merging. Suppose we monitor the merging pattern of a random sample of 250 drivers at this site.

What is the probability that fewer than 120 of the drivers will use less than one-third of the acceleration lane length before merging?

What is the probability that more than 225 of the drivers will use less than one-third of the acceleration lane length before merging?

Solution

First we check for adequacy of the sample size:

np=(250)(0.46)=115 andn(1–p)=(250) (1–0.46)=135.

Both are greater than 5. Hence, we can use the normal approximation. Let X be the number of drivers using less than one-third of the lane length available before merging. Then X can be considered to be a binomial random variable. Also,

μ=np=(250)(0.46)=115.0

and

σ=np(1−p)=250(0.46)(0.54)=7.8804.

Thus,

P(X<120)=P(Z<119.5−1157.8804=0.57103)=0.7157, that is, we are approximately 71.57% certain that fewer than 120 drivers will use less than one-third of the acceleration length before merging.

P(X>225)=P(Z>225.5−1157.8804=14.02213)≈0, that is, there is almost no chance that more than 225 drivers will use less than one-third of the acceleration lane length before merging.

Section 5.4 Conditional Distribution, Density and Mass Functions

5.17

For the discrete random variables whose joint PMF is described by the table in Exercise 5.14, find the following conditional PMFs:

PM(m|N=2);

PM(m|N≥2);

PN(n|M≠2).

Consider again the random variables in Exercise 5.11 that are uniformly distributed over an ellipse.

Find the conditional PDFs, fX|Y(x|y) and fY|X(y|x).

Find fX|Y>1(x).

Find fY|{|X| <1}(y)

Recall the random variables of Exercise 5.12 that are uniformly distributed over the region |X| + |Y| ≤ 1.

Find the conditional PDFs, fX|Y(x|y) and fY|X(y|x).

Find the conditional CDFs, FX|Y(x|y) and FY|X(y|x).

Find fX|{Y > 1/2}(x) and FX|{Y > 1/2}(X).

Suppose a pair of random variables (X, Y) is uniformly distributed over a rectangular region, A: x1 < X < X2, y1 < Y < y2. Find the conditional PDF of (X, Y) given the conditioning event (X, Y) ɛ B, where the region B is an arbitrary region completely contained within the rectangle A as shown in the accompanying figure.

2.8 Discrete Random Variables

Suppose we conduct an experiment, E, which has some sample space, S. Furthermore, let ξ be some outcome defined on the sample space, S. It is useful to define functions of the outcome ξ, X = f(ξ). That is, the function f has as its domain all possible outcomes associated with the experiment, E. The range of the function f will depend upon how it maps outcomes to numerical values but in general will be the set of real numbers or some part of the set of real numbers. Formally, we have the following definition.

Definition 2.9: A random variable is a real valued function of the elements of a sample space, S. Given an experiment, E, with sample space, S, the random variable X maps each possible outcome, ξ ∈ S, to a real number X(ξ) as specified by some rule. If the mapping X(ξ) is such that the random variable X takes on a finite or countably infinite number of values, then we refer to X as a discrete random variable; whereas, if the range of X(ξ) is an uncountably infinite number of points, we refer to X as a continuous random variable.

Since X = f(ξ) is a random variable whose numerical value depends on the outcome of an experiment, we cannot describe the random variable by stating its value; rather, we must give it a probabilistic description by stating the probabilities that the variable X takes on a specific value or values (e.g., Pr(X=3) or Pr(X > 8)). For now, we will focus on random variables which take on discrete values and will describe these random variables in terms of probabilities of the form Pr(X=x). In the next chapter when we study continuous random variables, we will find this description to be insufficient and will introduce other probabilistic descriptions as well.

Definition 2.10: The probability mass function (PMF), PX(x), of a random variable, X, is a function that assigns a probability to each possible value of the random variable, X. The probability that the random variable X takes on the specific value x is the value of the probability mass function for x. That is, PX(x) = Pr(X=x). We use the convention that upper case variables represent random variables while lower case variables represent fixed values that the random variable can assume.

Example 2.23

A discrete random variable may be defined for the random experiment of flipping a coin. The sample space of outcomes is S = {H, T}. We could define the random variable X to be X(H) = 0 and X(T) = 1. That is, the sample space H, T is mapped to the set {0, 1} by the random variable X. Assuming a fair coin, the resulting probability mass function is PX(0) = 1/2 and PX(l) = 1/2. Note that the mapping is not unique and we could have just as easily mapped the sample space {H, T} to any other pair of real numbers (e.g., {1,2}).

Example 2.24

Suppose we repeat the experiment of flipping a fair coin n times and observe the sequence of heads and tails. A random variable, Y, could be defined to be the number of times tails occurs in n trials. It turns out that the probability mass function for this random variable is

PY(k)=(nk)(12 )n,k=0,1,…,n.

The details of how this PMF is obtained will be deferred until later in this section.

Example 2.25

Again, let the experiment be the flipping of a coin, and this time we will continue repeating the event until the first time a heads occurs. The random variable Z will represent the number of times until the first occurrence of a heads. In this case, the random variable Z can take on any positive integer value, 1 ≤ Z < ∞. The probability mass function of the random variable Z can be worked out as follows:

Pr(Z=n)=Pr(n-1tailsfollowedby oneheads)=(Pr(T))n-1Pr(H)=(12)n-1(1 2)=2-n.

Hence,

PZ(n) = 2−n, n = 1, 2, 3,….

Example 2.26

In this example, we will estimate the PMF in Example 2.24 via MATLAB simulation using the relative frequency approach. Suppose the experiment consists of tossing the coin n = 10 times and counting the number of tails. We then repeat this experiment a large number of times and count the relative frequency of each number of tails to estimate the PMF. The following MATLAB code can be used to accomplish this. Results of running this code are shown in Figure 2.3.

Try running this code using a larger value for m. You should see more accurate relative frequency estimates as you increase m.

From the preceding examples, it should be clear that the probability mass function associated with a random variable, X, must obey certain properties. First, since PX(x) is a probability it must be non-negative and no greater than 1. Second, if we sum PX(x) over all x, then this is the same as the sum of the probabilities of all outcomes in the sample space, which must be equal to 1. Stated mathematically, we may conclude that

When developing the probability mass function for a random variable, it is useful to check that the PMF satisfies these properties.

In the paragraphs that follow, we list some commonly used discrete random variables, along with their probability mass functions, and some real-world applications in which each might typically be used.

B. Binomial Random Variable

Consider repeating a Bernoulli trial n times, where the outcome of each trial is independent of all others. The Bernoulli trial has a sample space of S = {0,1} and we say that the repeated experiment has a sample space of Sn = { 0, 1 }n, which is referred to as a Cartesian space. That is, outcomes of the repeated trials are represented as n element vectors whose elements are taken from S. Consider, for example, the outcome

(2.35).

The probability of this outcome occurring is

(2.36)Pr(ξk)=Pr(1,1,…,1,0, 0,…,0)=Pr(1)Pr(1)…Pr(1) Pr(0)Pr(0)…Pr(0)=(Pr(1))k(Pr(0))n- k=pk(1-p)n-k.

In fact, the order of the 1's and 0's in the sequence is irrelevant. Any outcome with exactly k 1's and n − k 0's would have the same probability. Now let the random variable X represent the number of times the outcome 1 occurred in the sequence of n trials. This is known as a binomial random variable and takes on integer values from 0 to n. To find the probability mass function of the binomial random variable, let Ak be the set of all outcomes which have exactly k 1's and n − k 0's. Note that all outcomes in this event occur with the same probability. Furthermore, all outcomes in this event are mutually exclusive. Then,

PX (k) = Pr(Ak) = (# of outcomes in Ak)*(probability of each outcome in Ak

(2.37)(nk)pk( 1-p)n-k,k=0,1,2,…,n .

The number of outcomes in the event Ak is just the number of combinations of n objects taken k at a time. Referring to Theorem 2.7, this is the binomial coefficient,

As a check, we verify that this probability mass function is properly normalized:

(2.39)∑ k=0n(nk)pk( 1-p)n-k=(p+1-p)n= 1n=1.

In the above calculation, we have used the binomial expansion

(2.40)(a+b)n =∑k=0n(nk)ak bn-k.

Binomial random variables occur, in practice, any time Bernoulli trials are repeated. For example, in a digital communication system, a packet of n bits may be transmitted and we might be interested in the number of bits in the packet that are received in error. Or, perhaps a bank manager might be interested in the number of tellers that are serving customers at a given point in time. Similarly, a medical technician might want to know how many cells from a blood sample are white and how many are red. In Example 2.23, the coin tossing experiment was repeated n times and the random variable Y represented the number of times tails occurred in the sequence of n tosses. This is a repetition of a Bernoulli trial, and hence the random variable Y should be a binomial random variable with p = 1/2 (assuming the coin is fair).

1.2.3 Moments and Expected Values

If X is a discrete random variable, then its m th moment is given by

(1.6)E[Xm]=∑iximPr{X=xi}

[where the xi are specified in (1.1)], provided that the infinite sum converges absolutely. Where the infinite sum diverges, the moment is said not to exist. If X is a continuous random variable with probability density function f (x), then its mth moment is given by

(1.7)E[Xm] =∫−∞+∞xmf(x)dx,

provided that this integral converges absolutely.

The first moment, corresponding to m = 1, is commonly called the mean or expected value of X and written mX or μX. The m th central moment of X is defined as the m th moment of the random variable X − μX, provided that μX exists. The first central moment is zero. The second central moment is called the variance of X and written σx2 or Var[X]. We have the equivalent formulas Var[X] = E [(X − μ)2] = E [X2] − μ2.

The median of a random variable X is any value v with the property that

Pr{X≥v}≥12and Pr{X≤v}≥12.

If X is a random variable and g is a function, then Y = g (X) is also a random variable. If X is a discrete random variable with possible values x1, x2, …, then the expectation of g (X) is given by

(1.8)E[g(X)]=∑i= 1∞g(xi)Pr{X−xi},

provided that the sum converges absolutely. If X is continuous and has the probability density function fX, then the expected value of g (X) is evaluated from

(1.9)E[g(X)]= ∫g(x)fX(x)dx.

The general formula, covering both the discrete and continuous cases, is

(1.10) E[g(X)]=∫g(x)dFX(x),

where FX is the distribution function of the random variable X. Technically speaking, the integral in (1.10) is a Lebesgue–Stieltjes integral. We do not require knowledge of such integrals in this text, but interpret (1.10) to signify (1.8) when X is a discrete random variable, and to represent (1.9) when X possesses a probability density fX.

Let FY (y) = Pr{Y ≤ y} denote the distribution function for Y = g (X). When X is a discrete random variable, then

E[Y]=∑jyjPr{Y=yj}=∑ig(xi)Pr{X=xi}

if yi = g (xi) and provided that the second sum converges absolutely. In general,

(1.11)E[Y]=∫ydFY(y)=∫g(x)dFX(x).

If X is a discrete random variable, then so is Y = g (X). It may be, however, that X is a continuous random variable, while Y is discrete (the reader should provide an example). Even so, one may compute E [Y] from either form in (1.11) with the same result.

EXAMPLE 4.2a

Consider a random variable X that is equal to 1,2, or 3. If we know that

p(1)12andp(2)=13

then it follows (since p (1) + p (2) + p (3) = 1) that

A graph of p(x) is presented in Figure 4.1.

The cumulative distribution function F can be expressed in terms of p(x) by

If X is a discrete random variable whose set of possible values are x 1, x 2, x 3,…, where x 1 < x 2 < x 3 <…, then its distribution function F is a step function. That is, the value of F is constant in the intervals [xi-1, xi) and then takes a step (or jump) of size p(xi) at xi .

For instance, suppose X has a probability mass function given (as in EXAMPLE 4.2a) by

Then the cumulative distribution function F of X is given by

F(a )={0a<1121≤a<2562≤a<313≤a

This is graphically presented in Figure 4.2.

Whereas the set of possible values of a discrete random variable is a sequence, we often must consider random variables whose set of possible values is an interval. Let X be such a random variable. We say that X is a continuous random variable if there exists a nonnegative function f (x), defined for all real x ɛ (-∞,∞), having the property that for any set B of real numbers

(4.2.1)P{X∈B}=∫Bf(x)dx

The function f (x) is called the probability density function of the random variable X .

In words, Equation 4.2.1 states that the probability that X will be in B may be obtained by integrating the probability density function over the set B. Since X must assume some value, f(x) must satisfy

All probability statements about X can be answered in terms of f (x). For instance, letting B =[a, b ], we obtain from Equation 4.2.1 that

(4.2.2)P{a≤X≤b}=∫abf(x)dx

If we let a = b in the above, then

In words, this equation states that the probability that a continuous random variable will assume any particular value is zero. (See Figure 4.3.)

The relationship between the cumulative distribution F (.) and the probability density f (.) is expressed by

F(a)=P{X∈(-∞,a)}=∫-∞af(x)dx

Differentiating both sides yields

That is, the density is the derivative of the cumulative distribution function. A somewhat more intuitive interpretation of the density function may be obtained from Equation 4.2.2 as follows:

P{a-ɛ2≤X≤a+ɛ2}=∫a-ɛ/2a+ɛ/2f(x)dx≈ɛf (a)

when ε is small. In other words, the probability that X will be contained in an interval of length ε around the point a is approximately ε f (a). From this, we see that f (a) is a measure of how likely it is that the random variable will be near a .