The Total Law of Probability

Consider a collection of events $A_{1}, \ldots, A_{n}$. These form a partition of $\Omega$ if:

1. they are pairwise mutually exclusive.
2. they are collectively exhaustive.
3. they all have nonzero probability, $P\left(A_{i}\right)>0$ for $i=1, \ldots, n$.

A partition divides $\Omega$ into non-overlapping subsets, like the pieces of a jigsaw puzzle. The Law of Total Probability allows us to use partitions to simplify probability calculations.

Suppose that $A_{1}, \ldots, A_{n}$ is a partition of $\Omega$. For any event $B \subset \Omega$, the Law of Total Probability states that:

We consider separately the probability that $B$ occurs in each possible 'scenario' $\left(A_{k}\right)$, and then combine the resulting conditional probabilities.

Example: Total Law of Probability

Suppose you have three coins. You know that one of the coins, coin 1, is fair and therefore lands heads with the probability 0.5 . The other two coins, coins 2 and 3 , are biased, landing heads with probabilities of 0.4 and 0.8 respectively. Given that your opponent picks one of the three coins at random (with equal probability) and tosses it, without telling you which one it was, what is the probability that it lands heads?

In the problem statement above we are provided with the conditional probabilities of heads, given we know the coin is 1,2 or 3 (events $C_{1}, C_{2}$ and $C_{3}$ respectively):

We do not actually know which of the three coins has been tossed, but we know that all three coins are equally likely:

Since events $C_{1}, C_{2}$ and $C_{3}$ are mutually exclusive (i.e. only one coin is tossed), and collectively exhaustive (at least one of these coins is tossed), we can apply the law of total probability to compute the total probability of a coin landing heads: