Random Variables & Probability Distributions

Discrete Random Variables: Univariate and bivariate distributions:

Function/Property	Univariate (X)	Bivariate (X,Y)
Probability Mass Function	$P(X = x) = f_X(x)$	$P(X = x, Y = y) = f_{X,Y}(x, y)$
Cumulative Distribution Function	$F_X(x) = P(X \leq x)$ $F_X(x) = \sum_{u \leq x} f_X(u)$	$F_{X,Y}(x, y) = P(X \leq x, Y \leq y)$ (general) $F_{X,Y}(x, y) = \sum_{u \leq x} \sum_{v \leq y} f_{X,Y}(u, v)$
Expectation	$E[g(X)] = \sum_{\forall x} g(x) \cdot f_X(x)$	$E[g(X, Y)] = \sum_{\forall x} \sum_{\forall y} g(x, y) f_{X,Y}(x, y)$
Conditional Expectation		$E[X\\|Y = y] = \sum_{\forall x} x f_{X\\|Y}(x\\|y)$ (see below for $f_{X\\|Y}(x\\|y)$ )
Variance	$Var[X] = E[X^2] - E[X]^2$
Covariance		$Cov[X,Y] = E[XY] - E[X]E[Y]$
Ch 1 Relationships	$P(X, Y)$	Expressed in terms of PDF: $f_{X, Y}(x, y)$
Conditional Probability	$P(Y\\|X) = \frac{P(X,Y)}{P(X)}$ $P(X\\|Y) = \frac{P(X,Y)}{P(Y)}$	$\begin{aligned} f_{Y \mid X}(y \mid x) & =\frac{f_{X, Y}(x, y)}{f_{X}(x)} \\ f_{X \mid Y}(x \mid y) & =\frac{f_{X, Y}(x, y)}{f_{Y}(y)}\end{aligned}$
Total Law of Probability	$P(X) = \sum_{\forall y} P(X, y)$
Marginalisation		$f_X(x) = \sum_{\forall y} f_{X,Y}(x, y)$ $f_Y(y) = \sum_{\forall x} f_{X,Y}(x, y)$
Independence	$P(X, Y) = P(X)P(Y)$	$f_{X,Y}(x, y) = f_X(x)f_Y(y)$

Continuous Random Variables: Univariate and bivariate distributions:

Function/Property	Univariate $(X)$	Bivariate $(X, Y)$
$\begin{array}{l}\text { Probability Density Function } \\ \qquad \text { PDF }\end{array}$	$P(a<X<b)=\int_{a}^{b} f_{X}(x) d x$	$\begin{array}{c}\qquad P[(X, Y) \in A]=\iint_{A} f_{X, Y}(x, y) d x d y \\ \text { (Note: region } A \text { is not necessarily rectangular) }\end{array}$
$\mathrm{CDF}$	$\begin{aligned} F_{X}(x) & =P(X \leq x) \\ F_{X}(x) & =\int_{-\infty}^{x} f_{X}(u) d u\end{aligned}$	$\begin{array}{c}F_{X, Y}(x, y)=P(X \leq x, Y \leq y) \text { (general) } \\ F_{X, Y}(x, y)=\int_{-\infty}^{y} \int_{-\infty}^{x} f_{X, Y}(u, v) d u d v\end{array}$
Link between PDF & CDF	$f_{X}(x)=\frac{d}{d x} F_{X}(x)$	$f_{X, Y}(x, y)=\frac{\partial^{2}}{\partial x \partial y} F_{X, Y}(x, y)$
$\begin{array}{l}\text { Expectation } \\ \text { Conditional Expectation }\end{array}$	$E[g(X)]=\int_{-\infty}^{\infty} g(x) f_{X}(x) d x$	$\begin{array}{c}E[g(X, Y)]=\int_{-\infty}^{\infty} \int_{-\infty}^{\infty} g(x, y) f_{X, Y}(x, y) d x d y \\ E[X \mid Y=y]=\int_{-\infty}^{\infty} x f_{X \mid Y}(x \mid y) d x \\ \left.\text { (see below for } f_{X \mid Y}(x \mid y)\right)\end{array}$
$\begin{array}{l}\text { Variance } \\ \text { Covariance }\end{array}$	$\operatorname{Var}[X]=E\left[X^{2}\right]-E[X]^{2}$	$\operatorname{Cov}[X, Y]=E[X Y]-E[X] E[Y]$
Ch 1 Relationships	$P(X, Y)$	Expressed in terms of PDF: $f_{X, Y}(x, y)$
Conditional Prob	$\begin{array}{l}P(Y \mid X)=\frac{P(X, Y)}{P(X)} \\ P(X \mid Y)=\frac{P(X, Y)}{P(Y)}\end{array}$	$\begin{aligned} f_{Y \mid X}(y \mid x) & =\frac{f_{X, Y}(x, y)}{f_{X}(x)} \\ f_{X \mid Y}(x \mid y) & =\frac{f_{X, Y}(x, y)}{f_{Y}(y)}\end{aligned}$
$\begin{array}{c}\text { Total Law } \\ \text { Marginalisation }\end{array}$	$P(X)=\sum_{\forall y} P(X, y)$	$\begin{array}{l}f_{X}(x)=\int_{-\infty}^{\infty} f_{X, Y}(x, y) d y \\ f_{Y}(y)=\int_{-\infty}^{\infty} f_{X, Y}(x, y) d x\end{array}$
Independence	$P(X, Y)=P(X) P(Y)$	$f_{X, Y}(x, y)=f_{X}(x) f_{Y}(y)$

Independence Definition of a Random Variable