How to Choose a Statistical Test: A Decision Table With Examples

Choosing the wrong statistical test is one of the most common reasons reviewers question a paper's results. The choice is less mysterious than it looks: four questions about your data narrow it down to one or two options.

In short: to choose a statistical test, answer four questions. (1) Are you comparing groups, looking for a relationship, or predicting an outcome? (2) What type is your outcome variable: continuous, ordinal, categorical, count or time-to-event? (3) How many groups are there, and are the measurements independent or paired? (4) Do the data meet the assumptions of a parametric test? For example, two independent groups with a roughly normal continuous outcome → independent t-test; the same with skewed or ordinal data → Mann–Whitney U.

The 4 questions that decide the test

1. What is your aim? Compare groups (for example treatment vs control), measure a relationship between variables, or predict an outcome from several predictors.
2. What type is the outcome variable?

• Continuous: blood pressure, test score, time in minutes.
• Ordinal: Likert ratings, pain on a 0–10 scale.
• Categorical: yes/no, smoker/non-smoker, three treatment arms.
• Count: number of hospital visits.
• Time-to-event: time until relapse.

1. How many groups, and are they independent or paired? Paired data come from the same participants measured twice, or from matched pairs.
2. Are parametric assumptions met? Mainly approximate normality of the outcome (or residuals), similar variances between groups and independent observations.

Statistical test decision table

Worked examples

• Does a new teaching method improve exam scores? Two independent classes, continuous score, roughly normal → independent t-test. If scores are heavily skewed → Mann–Whitney U.
• Did anxiety scores change after an 8-week programme? Same participants before and after, continuous score → paired t-test, or Wilcoxon signed-rank if differences are not normal.
• Is smoking associated with a diagnosis? Two yes/no variables → chi-square test; if any expected cell count is below 5 → Fisher's exact test.
• Is screen time related to sleep duration? Two continuous variables → Pearson correlation; if the relationship is monotonic but not linear, or data are ordinal → Spearman.
• Which factors predict whether a patient is readmitted? Binary outcome with several predictors → logistic regression.

Check the assumptions before you trust the result

• Normality: look at histograms or Q–Q plots. Formal tests such as Shapiro–Wilk flag tiny deviations in large samples, so do not rely on them alone.
• Equal variances: compare group spreads; Welch's t-test does not assume equal variances and is a safe default for two groups.
• Independence: each observation should come from a different participant unless you use a paired or repeated-measures test.
• Multiple comparisons: when you run many tests, adjust (for example Bonferroni or Holm) or pre-specify the main comparison.

Report an effect size and a confidence interval with every p-value, so readers can judge whether the effect matters, not just whether it is statistically significant.

Can AI choose the statistical test for you?

AI tools can suggest a suitable test from a description of your question and data, explain the assumptions, and draft analysis code. They can also be confidently wrong. Treat the suggestion as a second opinion: check it against the decision table, run the assumption checks yourself, and do not upload identifiable research data to tools that are not approved by your institution.

In this SciSpace session, a PhD statistician tests AI on real analyses and shows how to validate the model it chooses:

Key moments: 12:32 is it safe to upload research data to AI? · 18:12 checking analysis with STROBE, CHAMP and SAMPL checklists · 43:53 validating the statistical model AI chose. See also how AI matches charts and statistical tests to research questions and Spearman vs Pearson in other SciSpace webinars.

Watch: cleaning data and running statistical tests with AI

Choosing the test is one step. This SciSpace webinar covers preparing a dataset, cleaning it and running the tests, with checks on what the AI produces.

Related guides

Frequently asked questions

注意转载授权 · 阅读原文