Which statistical test should I use?
It depends on three things: whether you're comparing groups or measuring relationships, how many groups or variables are involved, and what type of data you have. Use the following as a starting point. When your data clearly breaks a test's assumptions, such as normality in a small sample, use the non-parametric alternative.
Scroll horizontally to compare aims, tests and alternatives.
Common statistical tests and non-parametric alternatives by research aim| Your aim | Common test | Non-parametric alternative |
|---|
| Compare two independent groups | Independent-samples t-test | Mann–Whitney U test |
|---|
| Compare the same people at two points | Paired-samples t-test | Wilcoxon signed-rank test |
|---|
| Compare three or more groups | One-way ANOVA | Kruskal–Wallis test |
|---|
| Test an association between two categorical variables | Chi-square test of independence | Fisher's exact test, for small samples |
|---|
| Measure a relationship between two continuous variables | Pearson correlation | Spearman correlation |
|---|
| Predict a continuous outcome from several predictors | Multiple regression | None directly; consider transforming the data |
|---|
| Predict a yes-or-no outcome | Logistic regression | Not applicable |
|---|
Can I use parametric tests on Likert-scale data?
A single Likert item is ordinal, so non-parametric tests are the cautious choice for individual items. Scale scores, averaged across several items that measure one construct, are commonly treated as continuous. Norman (2010) argued that parametric tests remain robust for Likert data even when assumptions are bent. Whichever you choose, state and justify it in your methodology.
What does a p-value actually mean?
A p-value is the probability of getting results at least as extreme as yours if there were really no effect. It is not the probability that your hypothesis is true. The 0.05 cut-off is a convention, not a law. A small p-value says an effect is unlikely to be chance, not that it is large or important, so report effect sizes too.
What is an effect size, and why report it?
An effect size shows how large a difference or relationship is, independent of sample size. Common measures include:
- Cohen's d for differences between means.
- r for correlations.
Cohen (1988) suggested 0.2, 0.5 and 0.8 as rough benchmarks for small, medium and large values of d, but what counts as meaningful depends on your field and question. These are not universal cut-offs.
How do I check the assumptions of regression?
Check these five things:
- A linear relationship between predictors and outcome.
- Independent errors.
- Constant variance of residuals (homoscedasticity).
- Roughly normal residuals.
- No severe multicollinearity between predictors.
Use scatterplots and residual plots for the first four and variance inflation factors (VIF) for the last. Values above 5 or 10 are commonly flagged. Report what you checked and what you did about problems.
How should I report statistical results?
Most programmes expect APA style: the test, its statistic, degrees of freedom, p-value and effect size, in a consistent format. Use this exact example:
t(118) = 2.45, p = .016, d = 0.45.
Put detailed output in tables and explain in words what each result means for your hypothesis. Never paste raw software output into the main text.