P-value vs. T-value: what’s the difference?

Mr. Bayes pondering over the difference between these values…

You may have heard of p-values in the context of hypothesis testing, with a vague (and technically incorrect 😉) idea that “If p-value is < 0.05, then the two groups are different”. This post explains the definition of p-value and t-value , their difference in usage and interpretation with examples in Python using cats 😸

xkcd - p-value interpretations

P-value

p-value is the probability of getting a result equal to or more extreme than what was observed, assuming that the Null hypothesis is true. In simple terms, it tells your what the odds are that your observed results could have happened by chance. Since p-value is a probability, its value is always in the range of [0,1] .

Exhibit 1: Sample male British shorthair cat lounging in his habitat

Suppose, that you want to test the hypothesis that the average weight of male British Shorthair (BSH) cats is 6kg(13 lbs). You go to the vet and collect weights of 50 randomly selected male BSH cats, and the result you got is 6.6kg. You ran some statistical tests to compute the p-value and it turned out to be 0.07. How should you interpret p-value = 0.07 ?

Two things to bear in mind since we are running a classical frequentist hypothesis test:

1. The Null hypothesis H0 is assumed to be true. In this case, it means that we assume that the true average weight of male BSH cats is 6kg
2. Imagine that you are computing the probability that the sample mean is ≥ 6.6kg assuming that the hypothesis is correct. This means that we want to calculate:

The value you get from #2 is the p-value . A p-value of 0.07 means that if we repeat our sampling many, many times, each time sampling 50 male BSH cats randomly and computing their average weight, then 7 out of 100 times we can expect to see an average weight ≥ 6.6kg.

What does it mean to have a p-value = 0.07 ? Well, there are 2 possible explanations:

1. Our assumption is incorrect, and a very rare event has taken place due to sampling error…