Quick answer
P-values test compatibility between observed data and a null model; they do not measure effect importance. Confidence intervals estimate plausible effect ranges and precision. Strong interpretation combines both metrics with study quality, assumptions, and practical impact to avoid misleading conclusions.
Key takeaways
- A p-value is calculated under a statistical model; it is not the probability that the null hypothesis is true.
- A confidence interval displays effect magnitude and uncertainty on the scale used in the analysis.
- Statistical non-significance does not demonstrate equality or absence of an important effect.
- Interpretation must consider design quality, multiplicity, missing data, prior evidence, and practical consequences.
What P-Values Do and Do Not Mean
A p-value is not the probability that the null hypothesis is true. It quantifies how unusual observed data are under model assumptions.
Confidence Intervals Add Context
Intervals reveal effect magnitude and uncertainty, helping distinguish statistically significant but clinically trivial findings.
Decision Quality
Interpret statistical evidence with prior plausibility, design quality, and external validity rather than threshold-only logic.
Practical method
Step-by-Step Workflow
- 1
Name the estimate
State exactly what was estimated—such as a mean difference, odds ratio, risk difference, slope, or hazard ratio—and which groups or units it compares.
- 2
Read the interval
Identify the point estimate, confidence limits, null value, and values representing a clinically or practically important effect.
- 3
Use the p-value narrowly
Describe compatibility with the specified null model without turning a continuous measure into proof, certainty, or a binary discovery label.
- 4
Write a contextual conclusion
Combine magnitude, precision, assumptions, study limitations, multiplicity, and external evidence in plain language.
Worked example
A useful non-significant result
- Scenario
- A treatment yields a risk ratio of 0.82 with a 95% confidence interval from 0.64 to 1.05 and p = 0.12.
- Approach
- Report the estimate and full interval. The data are compatible with a meaningful reduction, little effect, and a small increase; the study does not estimate the effect precisely enough to separate these possibilities.
- Interpretation
- Do not write that the treatment has no effect. Conclude that evidence against the null is limited and that the interval still includes effects that may matter clinically.
Common Mistakes to Avoid
- Calling the p-value the probability that chance caused the result
- Equating p > 0.05 with no effect
- Calling every p < 0.05 important
- Ignoring multiplicity and selective analysis
Frequently Asked Questions
Can a non-significant result still be useful?
Yes. It can inform uncertainty, detect small effects, and guide better-powered follow-up studies.
Why report effect size with p-values?
Effect sizes indicate practical importance, while p-values alone only address statistical compatibility.