Hypotheses about Distributions: Goodness-of-Fit Tests

The final application for statistical hypothesis testing that we will consider is whether the data sampled could be modelled by some distribution. For instance, we may be interested to see if the model we came up with is actually representative of a given the data set. The goodness-of-fit test considers the following hypotheses:

Essentially we are testing to see if there is any evidence in the sample that the chosen distribution is a bad fit. Note that evidence for rejecting a particular distributional model does not point to why it fails, and thus offers no guidance for finding an alternative (i.e. better-fitting) model.

Let's illustrate this concept with the help of an example. Suppose that for a particular experiment, we suspect that the number of particles suspended in a dusty gas could be modelled using a Poisson distribution. In order to verify this, we momentarily flash the light onto a microscope field and count the particles seen a couple of times. We record our findings in a following table:

Please note how we have discretised the infinite range of values that $x$ can take into $k=7$ bins. The test statistic for goodness-of-fit tests rely on this discretisation of the probability distribution. Namely, Pearsons chi-squared statistic is defined as:

Where:

$k \quad$ is the number of discrete bins for the distribution (in the example $k=7$ )

$O_{i} \quad$ is the number of observations falling into the $i$-th bin (i.e. the frequency row in the table)

$E_{i} \quad$ is the expected number of observations falling into the particular distribution, given the model

In order to compute the expected counts $\left(E_{i}\right)$, we first fit some distribution of choice to the data. Since we are using Poisson distribution in our example, we only have one parameter to fit: $\lambda$. We therefore compute its MLE estimate:

We can now compute the probabilities associated with each of the bins $P\left(x \in B_{i} \mid \lambda\right)$.

If we were to randomly place the values into the bins, according to the probabilities $p_{i}$ specified in the table we would expect to obtain $n \times p_{i}$ items in each of the bin, where $n$ is the total number of items placed (i.e. $n=143$ in this particular example). We will not prove this, but the result comes from a generalisation of Binomial probability distribution for more than two outcomes.

After computing each of the $E_{i}$ values, we can augment the table:

Note how $E_{i}$ values are fractional, but all still sum to the same $n$. From this table we compute the value for our test statistic $X^{2}$ :

Under the null hypothesis the Pearson's chi-squared test statistic follows a chi-square distribution with $k-p-1$ degrees of freedom, where $k$ is the number of data points and $p$ is the number of parameters in the fitted distribution:

We complete the test by consulting the tables of chi-squared distribution. Note that this test is always a right-tailed test, and therefore we reject the null hypothesis if the test statistic $X^{2}$ is strictly greater than the critical value for a given significance level.

In the dust particles example we used, the critical value for $\chi_{7-1-1}^{2}$ at $\alpha=0.05$ significance level is 11.070. In fact, the $X^{2}=1.9503$ value corresponds to p-value of 0.8569 under the said distribution. Therefore we cannot reject the null hypothesis (and therefore suspect the data can be modelled using Poisson distribution).