What does a p-value represent?

What does a p-value represent?

• A. The probability of observing the data (or more extreme) if there were truly no difference.
• B. The probability that the null hypothesis is true.
• C. The probability that the alternative hypothesis is true.
• D. The probability of making a Type I error.

Answer: (A)

The p-value reflects how surprising the observed data would be if there were truly no difference. It’s the probability, calculated from the null distribution, of obtaining results as extreme as or more extreme than what we actually saw, assuming the null hypothesis is true. It does not tell you the probability that the null is true (or false), nor the probability that the alternative is true. It’s also not the Type I error rate by itself—the latter is about how often you’d make a false rejection if you repeated the study many times with the null true. Remember that p-values can be influenced by sample size: a large study can yield a small p-value for a tiny, possibly meaningless difference, while a small study might not reach significance for a meaningful difference.

The p-value reflects how surprising the observed data would be if there were truly no difference. It’s the probability, calculated from the null distribution, of obtaining results as extreme as or more extreme than what we actually saw, assuming the null hypothesis is true. It does not tell you the probability that the null is true (or false), nor the probability that the alternative is true. It’s also not the Type I error rate by itself—the latter is about how often you’d make a false rejection if you repeated the study many times with the null true. Remember that p-values can be influenced by sample size: a large study can yield a small p-value for a tiny, possibly meaningless difference, while a small study might not reach significance for a meaningful difference.