What is a p-value?
A p-value is the probability of getting a result at least as extreme as the one you observed, assuming the null hypothesis is true. The null hypothesis is the default position, for example that a new drug has no effect or that two groups have the same average. A small p-value says your data would be surprising if the null hypothesis were true, which is evidence against it.
Researchers compare the p-value with a significance level, α, chosen before the test. The most common choice is α = 0.05. If p ≤ α the result is called statistically significant and the null hypothesis is rejected; otherwise it is not rejected. A p-value is not the probability that the null hypothesis is true, and a significant result is not proof that an effect is large or important.
From z-score to p-value
Many tests, including the one-sample z-test, the two-proportion test and large-sample tests of means, produce a test statistic that follows the standard normal distribution under the null hypothesis. The p-value is then an area under the normal curve, and which area you need depends on the alternative hypothesis:
| Alternative hypothesis | Test type | P-value |
|---|---|---|
| Parameter is less than the null value | Left-tailed | P(x < Z) |
| Parameter is greater than the null value | Right-tailed | P(x > Z) |
| Parameter is different from the null value | Two-tailed | P(x < −|Z| or x > |Z|) |
The between area, P(−Z < x < Z), is the confidence level that corresponds to a two-tailed cutoff at Z, and the center area is the distance from the mean to Z.
Worked example
A factory claims its bolts average 50 mm. A sample of 64 bolts averages 50.3 mm, and the population standard deviation is 1.2 mm. The test statistic is z = (50.3 − 50) / (1.2 / √64) = 2. For a two-tailed test, the p-value is P(x < −2 or x > 2) = 0.0455. Since 0.0455 < 0.05, the difference is significant at the 5% level. For a one-tailed test of "greater than", the p-value is 0.02275.
From p-value to z-score (critical values)
Working backward gives the critical value of a test, the z-score that marks the edge of the rejection region. Enter the significance level as a two-tailed p-value of 0.05 and the calculator returns z = 1.96. Common critical values:
| α | One-tailed z | Two-tailed z |
|---|---|---|
| 0.10 | 1.282 | 1.645 |
| 0.05 | 1.645 | 1.96 |
| 0.01 | 2.326 | 2.576 |
| 0.001 | 3.09 | 3.291 |
Reading p-values sensibly
- The 0.05 threshold is a convention, not a law of nature. A p-value of 0.049 and one of 0.051 carry nearly the same evidence.
- With very large samples, tiny and unimportant differences can produce very small p-values. Report the effect size and a confidence interval as well.
- Running many tests inflates the chance of false positives. Adjust α (for example with a Bonferroni correction) when you test many hypotheses.
- This calculator uses the normal distribution. For small samples with an estimated standard deviation, a t-distribution gives more accurate p-values.
Type I and Type II errors
Rejecting a true null hypothesis is a Type I error, and its probability is the significance level α. Failing to reject a false null hypothesis is a Type II error. Lowering α from 0.05 to 0.01 reduces false alarms but makes real effects harder to detect unless the sample size grows. The power of a test, one minus the Type II error rate, depends on the sample size, the true effect size and α, and is worth planning before data collection rather than after.
Converting a sample result into a z-score
For a sample mean, z = (x̄ − μ₀) / (σ / √n). For a sample proportion, z = (p̂ − p₀) / √(p₀(1 − p₀) / n). Enter the resulting z-score above to read off the p-value for your test.