| Definition | The population-level comparison, relationship, or estimate the study asks about. | How observations, groups, exposures, or treatments were created and measured. | Variable types, group count, dependence, and usable sample information. | The population parameter or model quantity to estimate or test. | The inferential procedure and the conditions that justify it. | The observed effect or relationship and its sampling uncertainty. | A conclusion about evidence under the method and assumptions. | A context-aware conclusion with practical meaning and uncertainty. |
|---|
| Primary decision question | What exactly should the analysis learn? | What comparisons and claims can the design support? | What kind of outcome and observations are present? | Which population claim is being evaluated? | Why is this procedure appropriate for this design? | What values remain reasonably compatible with the evidence? | What does the result support and not support? | What action is defensible, and what could change it? |
|---|
| Purpose | Keep method choice tied to a real inferential target. | Set the boundary for inference and causation. | Narrow the defensible method family. | Make the target explicit before reading output. | Connect computation to valid inference. | Show magnitude and precision, not only threshold crossing. | Prevent probability and causation errors. | Translate analysis without overstating evidence. |
|---|
| Time horizon | Before examining results. | Before or during data collection. | Before model fitting. | Before formal inference. | Before and during analysis. | After fitting the method. | After diagnostics and output review. | At reporting and future review. |
|---|
| Typical information | Problem statement, population, outcome, predictors, and decision context. | Sampling, assignment, timing, pairing, clustering, and measurement. | Codebook, measurement scale, missingness, and observation relationships. | Research question and statistical parameter. | Distribution, residual, variance, expected-count, independence, and linearity checks as applicable. | Point estimate, confidence interval, standard error, and effect size. | Estimate, interval, test result, effect size, assumptions, and design. | Practical threshold, costs, limitations, sensitivity, and stakeholder context. |
|---|
| Common confusion | Starting with a favorite test. | Treating observational and randomized designs as equivalent. | Ignoring paired or repeated observations. | Writing hypotheses about sample statistics. | Selecting by outcome alone. | Reporting only a p-value. | Interpreting a p-value as the probability a hypothesis is true. | Equating statistical significance with importance. |
|---|
| What does not belong | A vague topic with no analyzable target. | A causal claim unsupported by design. | A method that assumes independence when observations are paired. | A hypothesis chosen after seeing the desired result. | A universal test-selection rule. | False precision without an uncertainty measure. | Proof language from a single statistical test. | A guaranteed outcome or unsupported recommendation. |
|---|