Continuous Random Variables

The previous section on discrete random variables sets us up nicely to consider continuous random variables. Informally, a random variable is continuous if it represents a quantity that is measured (as opposed to counted). For instance, if we were to conduct a sampling of the heights of everyone in the class, we would get a range of values that are not essentially countable (e.g. $183.32 \mathrm{~cm}$ ). However, we may be interested in computing the probability of observing heights greater than $182 \mathrm{~cm}$ and less than $185 \mathrm{~cm}$; this section will introduce the methods for computing such probabilities.

NOTE: There is a slight change in nomenclature between discrete and continuous random variables; instead of probability mass functions (PMFs; $f_{X}(x)$ ) we have probability density functions (PDFs; also $\left.f_{X}(x) !\right)$.

We shall find out that all our definitions for the cumulative distribution function $\left(\mathrm{CDF} ; F_{X}(x)\right)$, expectation $(E[X])$ and variance $(\operatorname{Var}[X])$ are the same, except we replace summations with integrations.

Since we have an infinite number of values $X$ can take on as a continuous random variable, we can no longer computer the probability of observing an exact value (which is equal to zero, as we shall show mathematically). Thus, we need to define the probability over a range of values:

A random variable $X$ is continuous if there exists a function $f_{X}(x)$, the probability density function (PDF) of $X$, with the property:

Integrating the PDF over an interval $(a, b)$ gives the probability that $X$ takes a value within that interval.

If we let $a \rightarrow-\infty$, the probability $F_{X}(a)$ becomes 0 and we can write:

Notice the similarities between the above equation and property 2 for discrete $\operatorname{CDFs}\left(F_{X}(b)=\sum_{x \leq b} f_{X}(x)\right)$, where we have essentially replaced the summation with an integral; this tells us that we can integrate the PDF of $X$ to obtain the CDF. But what about evaluation of the CDF at a single point?

From this we can conclude that the CDF evaluated at a single point is zero; this has important implications, in that the CDF can only be evaluated over an interval of values for random variable $X$. This also leads to an often relaxed interchange between strict inequalities and inequalities in the computation of probabilities (you will see different versions in different texts):

For $a<b$ :

We can redefine the PDF as the derivative of the CDF:

The above definition is what is more commonly provided. Similar to a PMF, a function $f_{X}(x)$ is a PDF for a continuous random variable $X$ if and only iff:

Expectation and Variance for Continuous Random Variables

Analogous to the moments we computed for discrete random variables, we can do the same for continuous RVs:

The expectation, $E[X]$, of a continuous random variable $X$ is defined as:

In direct analogy to the discrete case, the expectation is a weighted integral over the values $X$ can take.

Similarly, we define the expectation of a function $g(X)$ of $X$ to be:

and note that the 'formal' definition of the variance is the same as in the discrete case:

Example: Probability Density Function from 2008 Exam

The continuous random variable $X$ has the density function:

(a) Show that $k=\alpha-1$. Solution:

(b) Calculate the mean of $X$.

Solution: The mean is the expectation of that random variable. Thus we need to compute:

To solve for this we can use integration by parts:

Evaluation of the first term gives us:

Which is equal to zero in both since $\alpha$ is strictly greater than two.

Computing the integral in the second term gives:

Which is equal to: $\frac{-1}{(\alpha-2)}$

Subtracting the two terms gives us $E[X]=\frac{1}{(\alpha-2)}$

(c) Compute the $P(1 \leq X \leq 2)$ for $\alpha=3$. Solution: We can simply integrate the PDF over the range $x=[1,2]$.

Figure 7 shows a plot of the $\operatorname{PDF} f_{X}(x)$ and $\int_{1}^{2} f_{X}(x) d x$ for $\alpha=3$.

Figure 7: Plot of the probability density function $f_{X}(x)=\frac{(\alpha-1)}{(1+x)^{\alpha}}$, for $\alpha=3$. The area under the curve is equal to 0.13889 , which corresponds to the probability of observing $x$ in that interval.

Example: Probabilistic Interpretation of a Wave Function

Let's take a moment to think about where we have seen these operations before in our Chem Eng courses. Recall in your Properties of Matter course that you were provided the wave function, $\Psi$, for the "particle in a box problem" that satisfies the Schrödinger equation and allegedly contains all the dynamical information about the system.

1. General Solution (Review):

Recall the Schrödinger equation for free motion in one direction is given by solutions to the equation:

The general solution to this problem is:

Where the wave function $\Psi_{k}(x)$ can be written more conveniently as:

This solution must satisfy the boundary conditions $\Psi_{k}(x=0)=0$ and $\Psi_{k}(x=L)=0$. Thus the expression simplifies to:

Next we need $\Psi_{k}(x)$ to satisfy the other boundary condition $\Psi_{k}(x=L)=0$, which results in the requirement that:

Solving for $k$ above, the final solution to the wave function for a particle in a box is therefore:

Where we now index the wave function by $n$ (the multiple of $\pi$ ).

2. Normalisation of the Wave Function:

If we focus on the information regarding the location of a particle, then we can use the Born interpretation of the wave function, which states that the probability density function (PDF) for finding a particle as a function of its position $x$ is:

Thus, starting from our solution to the Schrödinger equation $(\Psi(x)$ above), we have to normalise the wave function so it satisfies $\int_{-\infty}^{\infty} f_{X}(x) d x=1$ and can be utilised as a PDF to describe particle position:

This results in the following normalised wave function for a particle in a box of length $L$ (given to us in PoM):

(a) Probability Density Function

(b) Probability Density Maps

Figure 8: Plot of the probability density function for the particle in a box problem described by $f_{X}(x)=$ $\Psi_{n}(x)^{2}=\frac{2}{L} \sin ^{2} \frac{n \pi x}{L}$ for $n=1$ and $n=2$, and $L=1 \mathrm{~nm}$. (a) Notice that for $n=1$, the particle has the highest probability for being found at $x=0.5$ (in the middle of the box). When $n=2$, we see that the particle is more likely to be found at $x=0.25$ and $x=0.75$, with equal probability. As $n$ increases, we will have $n$ maxima in our PDF of equal probability. (b) Top-down representation of PDF functions, where darker shading indicates areas with a higher probability of observing the particle.

Figure 8 shows what the PDF looks like for the particle in a box for $n=1$ and $n=2$ in a box of length $L=1 \mathrm{~nm}$.

Expectation and Variance of the Wave Function

Given the wave function, we can compute the expected position of the particle using our definition of Expectation $(E[X]$; often represented in physics textbooks as $<x>)$ :

As in the previous example, this requires integration by parts and also the trigonometric equation $\sin ^{2} x=\frac{1}{2}(1-\cos 2 x)$

We leave it to the reader to provide that $u v=L$ and $\int v d u=\frac{L}{2}$. Thus:

Thus, the mean particle position is in the middle of the box and is independent of $n$.

In an analogous manner, we can compute $E\left[X^{2}\right]$ to determine the variance of these distributions using $\operatorname{Var}[X]=E\left[X^{2}\right]-E[X]^{2}$.

This expression is slightly more complicated, but again can be analytically computed using integration by parts:

Evaluating the first term is simple: $u v=L^{2}$. But the second term results in:

The latter term requires one more round of integration by parts (using $u=x, d u=1, d v=\sin \frac{2 n \pi x}{L}$, $v=$ $\left.-\frac{L}{2 n \pi} \cos \frac{2 n \pi x}{L}\right)$, which in total results in the terms in the brackets below:

Finally we can compute the variance as:

Although it was quite a bit of work, we can make a number of useful inferences with this analytical expression for $\operatorname{Var}[X]$. For starters, we can see that the variance is always positive, which it should be as it measures the dispersion of our PDF.

Pause and Reflect 1: How does the variance change as a function of $n$ ? Does this make intuitive sense?

Pause and Reflect 2: How do these results for expectation and variance compare to what we would expect for classical expressions?

Hint: The probability density function for classical mechanics would be simply represented as a uniform distribution, which results in the $\operatorname{PDF} f_{X}(x)=\frac{1}{L}$.

Example: Real Data for Continuous RVs - Particle Size Distribution

Finally let's consider an example of a continuous random variable based on actual experimental data. Many of such instances involve experiments where the cummulative distribution function $\left(\mathrm{CDF} ; F_{X}(x)\right)$ can be measured, and then we need to fit the experimental data to some model to compute probabilities of interest. In this example we will use experimental data corresponding to the distribution of particle sizes, which has important applications in various aspects of Chemical Engineering (e.g. reaction engineering, powder processing, etc).

In particle characterisation, we make an approximation that the particles exhibit some sort of a spherical shape, with a equivalent spherical diameter of $X$. Particle size analysis equipment of various sorts (e.g. sieves of different mesh sizes) can provide us with measurements pertaining to the total number of particles found to be below a particular size (or diameter, $X$ ). An example data set from actual measurements is presented below in Figure 9.

This raw data bears a striking resemblance to the shape of the discrete RV CDF for our sum of two dice example (see blue step function in Figure 4)! Indeed, the normalised form of this data represents our CDF for particle sizes $X$. Of course the data presented in Figure 9 is discrete in that it is comprised of only 30 data points; in practice, however, we intuitively know that the actual sizes of the particles in the experiment take on a continuum of values between 0-60 microns.

In order to work with the particle sizes as a continuum, we will need to fit the available experimental CDF data to a mathematical model (this is a topic we defer for now but will explore in further detail in later chapters). In the particle characterisation community, a number of PDF and CDF models have been proposed; one such model, known as the Rosin-Rammler-Sperling-Bennett (RRSB) cumulative function, is presented below:

where $N_{3}(x)$ implies the function models the cumulative mass or volume, $x_{63.2}$ represents the particle size at which $63.2 \%$ of the distribution lies below and $n$ is a constant called the 'uniformity index'.

Figure 9: Raw data for measured particle size distribution. Measurements have been normalised to total mass of the sample. This function represents the CDF for our continuous random variable $X: F_{X}(x)$.

Now say that using the data in Figure 9 we have 'estimated' the parameter values (again, using methods described later in the course) for this RRSB cumulative function to be $x_{63.2}=37.76$ and $n=15.165$. Now we can plot the particle size CDF for all sizes using the RRSB equation to represent our cumulative distribution function, as shown in the red curve in Figure 10.

By contrasting Figures 9 and 10, we can see that the RRSB CDF (red curve in Figure 10) approximates the raw data fairly well, particularly for the diameter range that describes most of the particles (33-41 microns). Since we now have a continuous CDF, we can differentiate this to obtain our probability density function (PDF), $f_{X}(x)=\frac{d}{d x} F_{X}(x)$, as shown by the green curve in Figure 10.

Using the analytical model for our $\operatorname{CDF} F_{X}(x)$, we can ask questions regarding the distribution of particle sizes in our sample. For example, if were interested in quantifying the population of particles having diameters between 30-35 microns, we would simply evaluate the following:

Likewise, we can compute the expectation $E[X]$ and variance $\operatorname{Var}[X]$ for the PDF using the moment functions defined earlier in the section. Taking the first moment of $f_{X}(x)$, we find that $E[X]=36.326$, as shown in Figure 11; note that this value does not correspond to the apex (or maximum value) of $f_{X}(x)$ in Figure 10, but lies slightly to the left, as the PDF has a broader left 'shoulder'.

Figure 10: RRSB cumulative function $\left(F_{X}(x)\right.$, also referred to as $N_{3}(x)$, red curve) using parameter values $x_{63.2}=37.76$ and $n=15.165$ estimated from raw data in Figure 9 . The $\operatorname{PDF}\left(f_{X}(x)\right.$, green curve $)$ can be calculated from the CDF via differentiation: $f_{X}(x)=\frac{d}{d x} F_{X}(x)$. Note that the area under the curve of $f_{X}(x)$ is equal to 1 (via numerical integration using the trapezoid rule).

Lastly, computing the second moment of $f_{X}(x)$ about $E[X]=36.326$ provides $\operatorname{Var}[X]=8.721$, as shown in Figure 12. It is interesting to examine the individual contributions to the variance calculation, which are shown by the green curve in Figure 12. From this, we see a clear bias in particle diameters below the mean value of 36.326 , which tells us that the distribution is asymmetric.

Figure 11: Expectation of $\operatorname{PDF} f_{X}(x)$ by computing the first moment is found to be $E[X]=36.326$. The green curve represents individual contributions to the expectation: $\int_{x-\delta}^{x+\delta} u \cdot f_{X}(u) d u$ for a given $x$. The red curve denotes the integral value up to $x$ : $\int_{0}^{x} u \cdot f_{X}(u) d u$, which when $x=60$ gives us $E[X]$.

Figure 12: Variance of PDF $f_{X}(x)$ by computing the second moment about $E[X]$ is found to be $\operatorname{Var}[X]=$ 8.721. The green curve represents individual contributions to the variance: $\int_{x-\delta}^{x+\delta}(u-E[X])^{2} \cdot f_{X}(u) d u$. The red curve denotes the integral value up to $x: \int_{0}^{x}(u-E[X])^{2} \cdot f_{X}(u) d u$, which when $x=60$ gives us $\operatorname{Var}[X]$. One can see from this plot that the PDF is asymmetric about $E[X]$ (i.e. its mean value).