Effect Size
A measure of the magnitude of a finding—how big the difference is between conditions, not just whether it exists. Essential for determining practical significance beyond statistical significance.
Definition: A measure of the magnitude of a finding—how big the difference is between conditions, not just whether it exists. Essential for determining practical significance beyond statistical significance.
Effect size quantifies the magnitude of a finding—how much better one version is over another, not just whether the difference exists.
Why It Matters
Statistical significance tells you a difference is probably real. Effect size tells you if it is big enough to matter.
A study might find a statistically significant difference that is too small to justify the cost of implementation. Conversely, a practically meaningful difference might not reach statistical significance in a small sample. You need both pieces of information.
Common Measures
Raw difference: The actual units of measurement (e.g., "12.5 SUS points higher")
Cohen's d: A standardized measure expressing the difference in terms of standard deviations:
- Small effect: d ≈ 0.2
- Medium effect: d ≈ 0.5
- Large effect: d ≈ 0.8
Standardized effect sizes allow comparison across different scales and studies.
Practical Application
When reporting findings, always include both:
- Statistical significance: Is the difference real?
- Effect size: Is the difference big enough to matter?
This practice is central to driving impact—it moves beyond a simple p-value to tell stakeholders whether a difference justifies the cost of development.
Related Terms
Statistical Significance
A determination that an observed result is unlikely to have occurred by random chance alone. Conventionally indicated by a p-value below 0.05, meaning less than 5% probability of the result being a fluke.
P-Value
The probability of observing your data (or something more extreme) if there were truly no effect. Widely used, widely misunderstood, and never sufficient on its own to make a decision.
Quantitative Research
Research focused on numerical measurement with the goal of generalizing findings from a sample to a broader population. Answers 'how much,' 'how many,' and 'how often.'
Mentions in the Knowledge Hub
This term is referenced in the following articles:
Sample Size Calculator — Tool and Explanations
An interactive sample size calculator for UX research, with the statistical foundations explained — from binomial problem discovery to power analysis.
Partnering with Data Science: The Quant-Qual Collaboration
The most powerful insights rarely come from a single source. They emerge from the strategic partnership between UX research and Data Science, fusing deep contextual understanding with patterns identified at massive scale.