General Laws of Probability

As we eluded to at the start of this course, the definition for the probability of an event has undergone considerable evolution over the years. Original definitions arose within the context of gambling, or situations where $n$ possible outcomes were equally likely. If some event $A$ were satisfied by $m$ outcomes, then $P(A)=\frac{m}{n}$ (this is the classical or a priori definition of probability, which we will encounter in subsequent examples).

However, what if outcomes are not equally likely, or occupy an infinite space? We could then resort to an empirical or a posteriori definition of probability, where we repeat an experiment $n$ times; this method can also suffer from limitations (can you think of any?).

Due to these issues, there was a strong desire to define probability as a function rather than a ratio (or limit). Suppose that $A \subseteq \Omega$ is an event, and let $P(A)$ be the probability of $A$. Intuitively, $P(A)$ is a number that indicates how likely $A$ is to occur. The axioms of probability are:

From these axioms, we can deduce some basic properties of probability.

The last property, $P(A \cup B)=P(A)+P(B)-P(A, B)$, is known as the "inclusion-exclusion" formula (or "additive rule", depending on what text you are reading), and can be illustrated using the following Venn diagram:

$P(A, B)$ is included in both $P(A)$ and $P(B)$ and therefore is counted twice when adding $P(A)$ and $P(B)$ together, requiring it to be subtracted afterwards, resulting in $P(A)+P(B)-P(A, B)$ expression.

From this we also have the "union bound", i.e. $P(A \cup B) \leq P(A)+P(B)$, which is a useful inequality when $A$ and $B$ are disjoint events.

Generalisation of the Inclusion-Exclusion Formula:

As the inclusion-exclusion is such a useful tool for solving problems, we take a moment here to show how it generalises to three events:

Example: Inclusion-Exclusion

In an online survey of 100 students, 55 students claimed to have seen the Breaking Bad TV series. The survey has also revealed that 44 students watch Game of Thrones, out of which 20 students watched both shows. What is the probability that a randomly-selected student has seen neither of the two shows?

This problem is closely related to the set operations described in the previous section. The sample space for randomly selecting a student from the survey, $\Omega$, consists of a 100 outcomes each corresponding to a different person. Fifty-five of these outcomes belong to the event of selecting a Breaking Bad fan, meaning that the probability of said event, $P(B)$ to be equal to $\frac{55}{100}$. Similarly, we expect to select a Game of Thrones fan with the probability $P(G)=\frac{44}{100}$. Likewise, we know that a randomly-selected student has watched both TV shows with the probability $P(B, G)=$ $\frac{20}{100}$. Using the inclusion-exclusion principle, we can find the probability that a randomly-selected student has watched either of these shows:

Since we want to get the probability that the student has seen neither, we need to take the complement of this event:

The result becomes intuitively evident when we draw a Venn diagram of survey results: