Hypothesis testing enables us to make statements about statistical significance. There are direct parallels between our justice system and hypothesis testing. Since most of us know about how our justice system works, I’ll use it as an analogy to explain how statistical hypothesis testing works.
The main idea: We use statistical hypothesis tests to accept or reject a hypothesis that there is no difference between two or more groups. We need to make sure if there is a difference that it is “significant” so that we are confident in our conclusion.
I’ll admit that this is not the most intuitive topic. But there is a similar concept that occurs in our legal system that offers us a useful analogy to help us understand the idea. >
In many settings where statistics are used, we often want to find a difference between two or more things. Some examples
- A drug company wants to see if there is a different result from group A who is given a drug compared to group B who does not take the drug
- Local government wants to know if the level of pesticides found in fish in a lake is different than the level of pesticides in fish in a different nearby lake>
- Fitbit wants to know if people who live in cities walk more on average than people who live in suburbs>
The null hypothesis
The null hypothesis is that there is no difference between the two groups. That is:
- There is no difference between the group taking the drug and the group not taking the drug
- There is no difference between the level of pesticides in fish in the two lakes.
- There is no difference in distanced walked between people living in cities and people living in suburbs
Statistical hypothesis testing are techniques for making conclusions about these differences. We analyze the data using a statistical test. There are several types of tests unique to the problem at hand, such as whether we are estimating a number or proportion, comparing averages or proportions, if sample sizes are same or different and many other scenarios. You may have heard of some of these techniques, including Z-tests, T-tests, and Chi-Squared tests. We will not get into the details here.
These tests let us conclude one of two things. We can either 1) fail to reject the null hypothesis and conclude there is no difference or 2) reject the null hypothesis. If we reject the hypothesis, we conclude the difference is significant.
Similar to our legal system
We said the null hypothesis is that there is no difference. In law there is a similar concept. A verdict about a defendant may be “not guilty”. It is never “innocent”. In the legal system, the beginning hypothesis is that the defendant is not-guilty. This is similar to statistical testing which starts with the hypothesis that two sets are not different. The burden of proof in law is to show beyond a reasonable doubt that the person is different than the hypothesis of not-guilty. In statistics we have to find evidence to show that a set is different. The prosecution must use evidence to prove beyond a reasonable that the hypothesis of being not guilty should be be rejected. If not rejected, the conclusion is that the defendant is not guilty. Note, the person is not innocent. Similarly in statistical testing, we do not accept the null hypothesis, rather we fail to reject the null hypothesis.
|Hypothesis||Legal System||Statistical Testing|
|Null Hypothesis||Not guilty
There is no difference between the defendant and the population.
|No difference between two groups|
there is a difference between the defendant and the population.
|Groups are different|
Decision Errors: Understanding False Positives and False Negatives
What does statistically significant mean?