A study can fail before the first observation is collected. The failure may not be bias, fraud or bad statistics. It can be more basic: the design never had enough information to answer the question it was built to ask.

Imagine a school wants to know whether a new teaching routine improves an assessment score by at least five points. The research team recruits twenty learners because twenty feels manageable. The study later finds a three-point difference with a wide confidence interval. Was the routine ineffective? Was the sample too small? Was five points the right target? Was the outcome too noisy? The result cannot answer those questions unless the design made its information requirements explicit in advance.

Statistical power and sample size planning are ways of matching the amount of information collected to the inferential job. Power asks how often a specified statistical procedure would detect a specified effect under repeated hypothetical studies. Sample size planning asks how much data is needed to achieve a chosen operating characteristic, precision target or other information goal under stated assumptions.

They are not rituals for producing one magic number. They are model-based planning tools. Their output is only as defensible as the question, effect size, variability, design, analysis, missing-data assumptions and decision rules placed inside them.

This article is a general research-methods explainer. Its education examples are constructed and do not report eduKate student outcomes. Clinical-trial sources are used where they provide strong methodological guidance; they are not clinical recommendations and do not replace specialist medical governance.

The design-to-information loop

QUESTION
→ DEFINE TARGET QUANTITY
→ DEFINE MEANINGFUL EFFECT OR PRECISION
→ CHOOSE DESIGN
→ CHOOSE ANALYSIS
→ STATE VARIABILITY / EVENT / CORRELATION ASSUMPTIONS
→ SET ERROR OR PRECISION TARGETS
→ CALCULATE INFORMATION REQUIREMENT
→ ADJUST FOR CLUSTERING / ATTRITION / MISSINGNESS / MULTIPLICITY
→ TEST SENSITIVITY TO ASSUMPTIONS
→ CHECK FEASIBILITY
→ FREEZE THE PLAN
→ COLLECT
→ ANALYSE
→ REPORT WHAT THE DESIGN COULD AND COULD NOT DETECT

The loop begins with a question and ends with an interpretation bounded by the design. Starting from a convenient sample and asking what can be claimed later reverses the logic.

1. Sample size is not one universal property of a study

“How many participants do I need?” has no answer until the research job is specified. A study estimating a mean to within two points needs a different calculation from a study testing whether two groups differ. A prevalence survey, a cluster trial, a survival analysis and a multilevel longitudinal study can all contain one hundred observations while carrying very different amounts of information.

The sample size belongs to a design-analysis pair. Change the primary outcome, allocation ratio, clustering, follow-up time, variance model or hypothesis and the appropriate information calculation can change as well.

That is why modern reporting guidance asks investigators to disclose the assumptions supporting the calculation. CONSORT 2025 explanation and elaboration specifically requires reporting how sample size was determined, including the primary outcome, assumed group values, target difference, variability, alpha, power, attrition adjustments and software where relevant.

2. Power is conditional, not a permanent badge

For a conventional hypothesis test, statistical power is the probability that the procedure rejects the null hypothesis when a particular alternative model is true. The phrase “the study has 80% power” is incomplete unless the alternative effect, variance, design, alpha level and test are also specified.

A study might have high power to detect a ten-point difference and poor power to detect a two-point difference. It may have adequate power under the expected standard deviation and inadequate power if the outcome proves twice as variable. Power is therefore a function, not a medal.

This point matters after the study too. A non-significant result does not become a proof of no effect merely because the protocol once contained the phrase 80% power. The design had power against a specified alternative. The observed estimate and its uncertainty still need to be interpreted directly.

3. Alpha and beta describe different long-run errors

In a simple null-hypothesis testing framework, alpha controls the long-run Type I error rate under the null model. Beta is the long-run probability of failing to reject when the specified alternative is true. Power is one minus beta.

These error rates are properties of procedures under repeated hypothetical use. They are not posterior probabilities that the null or alternative is true after one result. A p-value of 0.03 does not mean there is a three per cent probability that the null hypothesis is true.

Lowering alpha while holding everything else fixed generally increases the sample size required for the same power. Raising target power also generally increases required sample size. These are trade-offs between information, error control and resources.

4. The effect size must represent a question worth answering

A sample size calculation needs some target difference or effect. The most defensible target is usually not “whatever effect earlier small studies happened to report”. It should be tied to the smallest difference that would matter for the decision, theory or scientific claim.

Daniël Lakens’ open-access article Sample Size Justification emphasises that sample size should be justified in relation to the inferential goal. A priori power is one route, but precision, resource constraints, population coverage and other explicit justifications can be appropriate for different jobs.

If a school would not alter its practice for an average improvement smaller than five points, designing a study only to detect a twenty-point difference answers a less useful question. The resulting sample may be cheap and highly powered for a dramatic effect while being almost uninformative around the actual decision boundary.

5. Standardised effects can help comparison and hide meaning

Standardised effect sizes divide a difference by a measure of variability. They can be useful when scales differ, but they also move the interpretation away from the original units.

A standardised difference of 0.3 can correspond to very different practical changes depending on the outcome scale, population and spread of scores. When a raw-unit effect has clear substantive meaning, preserve it alongside any standardised quantity.

Sample size planning should therefore ask two separate questions: what difference matters in the world, and what statistical representation of that difference is required by the calculation?

6. A simple two-group example

Suppose a fictional study compares two independent groups on a continuous outcome. The smallest important mean difference is five points, and the common standard deviation is expected to be ten points. The standardised difference is therefore 0.5.

Under a conventional two-sided independent-samples t-test with alpha 0.05 and target power 0.80, a familiar approximation gives roughly sixty-four observations per group for an effect of this size. The exact result depends on the chosen test and calculation method. The point is not the number 64. The point is how rapidly the required sample changes when the assumptions change.

Constructed planning comparisons for illustration

Target standardised effect	Approximate information implication	Interpretive lesson
0.8	Relatively smaller sample	Easy to detect only because the target is large
0.5	Moderate sample	Common teaching example
0.2	Much larger sample	Small effects require far more information

Because required sample size grows roughly with the inverse square of the detectable standardised difference in many simple settings, halving the target effect can require about four times as much information. This nonlinear cost is why “we will detect even tiny differences” can imply an enormous design.

7. Variability is part of the information budget

For a fixed raw difference, greater outcome variability reduces the signal-to-noise ratio and usually increases the sample size required. That makes the assumed standard deviation one of the most consequential inputs to a continuous-outcome power calculation.

Do not take it from the most convenient prior paper without checking whether the outcome definition, population, time point and measurement conditions are comparable. A more heterogeneous population may have greater spread. A more reliable instrument may reduce measurement noise. Restricting eligibility may reduce variance while also reducing external validity.

Planning therefore connects directly to Construct Validity and Measurement Models and Measurement Error and Misclassification. Measurement design changes statistical information.

8. Precision planning can be more appropriate than power planning

Not every study is built around a null-hypothesis test. If the primary aim is to estimate a mean, proportion, correlation or treatment effect with useful precision, plan the width of the confidence or credible interval rather than forcing the problem into a detect-versus-not-detect framework.

Suppose a school wants the mean assessment score of a population estimated with a margin of error no larger than two points under specified coverage assumptions. The relevant calculation concerns standard error and interval width. Power against an arbitrary zero-difference null may be irrelevant.

This is one reason Lakens’ sample-size framework is useful: different inferential goals justify different planning methods. The correct question is not “What is the universal minimum N?” but “What amount of information makes this intended inference useful?”

9. Census-like problems use a different logic

Sometimes the population is finite and small enough that measuring almost everyone is feasible. Then a near-census can be more natural than an abstract power calculation. In other settings, a sample is drawn to estimate a population characteristic with a target margin of error.

This route belongs with How Surveys and Sampling Work and How Censuses and Population Statistics Work. Sample size alone cannot repair coverage error, nonresponse or a frame that excludes the people the claim is about.

10. Clustered data reduce the number of independent pieces of information

One hundred learners from one class are not necessarily equivalent to one hundred learners sampled independently across many unrelated contexts. Learners in the same class share teacher, timetable, environment and peer effects. Their outcomes may be correlated.

Positive intraclass correlation creates a design effect: the effective information is smaller than the raw number of individual observations suggests. In simple equal-cluster settings, a common approximation is:

Design effect = 1 + (m − 1) × ICC, where m is cluster size and ICC is the intraclass correlation.

If m = 20 and ICC = 0.05, the design effect is 1 + 19 × 0.05 = 1.95. A nominal sample of 200 individuals can then carry roughly the information of about 103 independent observations for some purposes. This is a constructed illustration, not a universal conversion.

The full problem is richer when cluster sizes vary, treatment is assigned at cluster level or outcomes are repeated through time. See How Clustered and Multilevel Data Work for the canonical treatment of dependence.

11. The number of clusters can matter more than the number of people

In a cluster-randomised study, adding many more people to each of a small number of clusters may provide less information about a group-level treatment effect than adding additional clusters. The reason is that treatment variation occurs between clusters.

Four schools with five hundred learners each are still only four schools for many cluster-level comparisons. A very large individual count should not conceal a weak number of independent assignment units.

This is why published power calculations for cluster designs must state the ICC, expected cluster sizes, number of clusters and analysis assumptions, not merely total N.

12. Unequal allocation usually costs information

For a simple two-group comparison with equal per-person cost and variance, equal allocation is often statistically efficient. Moving to a 2:1 or 3:1 ratio can require a larger total sample to preserve the same power.

Unequal allocation can still be justified by costs, capacity, recruitment, ethics, learning objectives or the need for greater precision in one arm. The choice should be explicit. “More people in the interesting group” is not automatically better design.

13. Binary outcomes depend on baseline risk as well as target difference

For a binary outcome, sample size depends on the expected event proportions and the effect scale. Detecting a change from 50% to 45% is not the same information problem as detecting a change from 5% to 0% even though both differ by five percentage points.

Rare outcomes can require very large samples because few informative events occur. A study may enrol thousands of people and still contain too few events to estimate a stable effect.

Always report the expected baseline rate, target difference or ratio, and the rationale for both. Effect sizes without their baseline can be difficult to interpret.

14. Survival studies are often powered by events, not merely enrolment

In time-to-event analysis, information is strongly related to the number and timing of observed events. A thousand enrolled participants with very few events may provide less information than a smaller cohort with longer follow-up and more observed events.

Planning must therefore combine effect assumptions with expected event incidence, accrual, follow-up, censoring and loss to follow-up. See How Time-to-Event and Survival Analysis Work for the underlying risk-set logic.

15. Repeated measures can add information and dependence at the same time

Measuring the same person multiple times can increase precision about trajectories or within-person change. Those measurements are correlated, so ten measurements on one person are not ten independent people.

Power depends on the covariance structure, number and timing of measurements, missingness pattern and analysis model. A planning calculation that assumes independence can substantially overstate information.

16. Covariate adjustment can increase precision—but should not be treated as free power

Strong baseline predictors can reduce residual variance and improve precision when used appropriately in analysis. This can reduce the sample size needed for a particular target precision or power.

But the gain depends on how predictive the covariates are, whether they are measured reliably and whether the analysis model is specified correctly. Planning around an unrealistically high R-squared can produce an underpowered study.

Covariate adjustment also does not transform an observational association into a causal effect. It is a statistical operation inside a design whose identification assumptions still need to be justified.

17. Attrition is not solved by multiplying N by 1.2 without thought

A common planning move is to calculate the number of completed observations needed and inflate recruitment to allow for expected attrition. If 100 complete observations are needed and 20% are expected to be lost, dividing by 0.8 gives a recruitment target of 125.

This arithmetic is valid only as an accounting step. It does not remove bias caused by informative dropout. If the people lost to follow-up differ systematically in their unobserved outcomes, a larger initial sample can produce more observations while leaving the missing-data problem intact.

The correct companion is How Missing Data Analysis Works: plan prevention, record reasons for missingness, specify the analysis and test sensitivity to assumptions.

18. Noncompliance and contamination change the effect you can detect

If treatment assignment is diluted because participants do not receive the intended intervention or control participants obtain similar support elsewhere, the observed intention-to-treat contrast can be smaller than the effect under perfect adherence.

Planning should therefore distinguish the estimand. Are we estimating the effect of assignment, the effect of receiving treatment under assumptions, or another policy-relevant quantity? The sample size calculation should be aligned with the intended analysis.

19. Multiple outcomes and repeated looks change error control

A design planned around one primary test cannot automatically keep the same Type I error properties after adding ten outcomes and checking the data after every twenty participants.

Multiplicity and sequential monitoring can require adjusted thresholds or specialised designs. These adjustments often change the information requirement. The canonical route is How Multiple Testing and Sequential Analysis Work.

CONSORT 2025 also asks trial reports to explain interim analyses and stopping guidelines. Planning and reporting should describe the same design, not two different stories.

20. Adaptive designs need simulations or design-specific mathematics

When allocation, sample size, stopping or arm selection can change in response to accumulating data, simple fixed-design formulas may no longer describe the operating characteristics.

Simulation can estimate power, false-positive rates, expected sample size and stopping behaviour under many scenarios. But the simulation must reproduce the actual rules, timing and analysis. A beautifully coded simulation of the wrong adaptive algorithm is still the wrong design.

21. Pilot studies are often poor sources for a precise effect-size guess

Small pilots are valuable for recruitment, acceptability, logistics, measurement feasibility and rough variability. They are often unstable estimators of the treatment effect the main study needs to detect.

If a tiny pilot happens to show a large effect, using that estimate unchanged can make the main study too small. Better inputs may come from the smallest effect of interest, a meta-analysis, credible prior evidence or a range of scenarios.

22. “Observed power” after the study usually adds little

Post-hoc power calculated by plugging the observed effect into the original power formula is largely a transformation of the observed test statistic. It does not rescue an inconclusive result or tell us the probability that the null is true.

After the study, report the effect estimate, interval, design and limitations. If the result is too imprecise to answer the question, say so. The useful post-study planning question concerns what information a future study would need, not how to re-label the same uncertainty.

23. A non-significant result can still exclude important effects

Suppose a study estimates a difference of one point with a 95% confidence interval from −1 to 3, and the smallest important benefit was five points. Even without statistical significance, the interval may already exclude the five-point effect the decision-maker cared about.

Conversely, an interval from −12 to 14 tells us very little even if the point estimate is close to zero. Interpretation should focus on which substantively important values remain compatible with the data and assumptions.

24. A significant result can still be too imprecise for use

A study can achieve p < 0.05 while leaving a wide range of practically different effects plausible. If a programme’s cost depends on whether the benefit is two points or twelve points, a binary significance label does not finish the decision.

Planning for precision can therefore complement or replace planning for significance.

25. Overpowered studies can detect trivial effects

With enormous samples, extremely small departures from a null model can become statistically significant. That is not a flaw in the mathematics. It is a reminder that statistical detectability and practical importance are different questions.

Predefine what effect sizes matter. Report estimates in interpretable units. A huge study should increase resolution, not lower the standard for meaningful interpretation.

26. Underpowered studies waste more than statistical opportunity

A weakly informative study consumes time, attention and participant effort while having a high chance of producing an inconclusive result for the question it claims to address.

That does not mean every small study is wasteful. Small studies can be appropriate for feasibility, rare populations, intensive measurement, qualitative inquiry, estimation with honestly wide intervals or cumulative evidence. The problem is a mismatch between small information and a large claim.

27. Feasibility constraints should be admitted, not hidden

Sometimes only thirty observations are realistically available. That can be an honest design fact. The correct response is not to manipulate assumptions until a calculator declares thirty adequate.

Instead, identify the smallest effect detectable with useful power, the precision achievable, or the restricted question the data can support. Lakens explicitly treats resource constraints as one possible sample-size justification when communicated transparently.

28. Sensitivity analysis belongs inside sample size planning

A single planned N can hide enormous dependence on uncertain inputs. Calculate a grid across plausible effect sizes, standard deviations, ICCs, attrition rates and event rates. Show how the target changes.

For example, if a design requires 150 participants when the standard deviation is 10 but 330 when it is 15, the uncertainty about variability is operationally important. That insight may justify collecting better pilot information about variance or selecting a more reliable outcome.

See How Sensitivity Analysis and Robustness Checks Work for the broader method.

29. Simulation is especially valuable for complex designs

When analytical formulas are unavailable or poorly matched to the intended model, simulate the full data-generating and analysis process. Generate many hypothetical datasets under specified scenarios, run the planned analysis and estimate the fraction of runs satisfying the decision rule.

A credible simulation records distributions, dependencies, missingness, clustering, stopping rules, analysis code and random seeds. It should also include deliberately adverse scenarios, not only the assumptions that make the study look efficient.

30. The analysis model and power model should match

Planning a sample size for an independent t-test and later analysing a multilevel model, repeated-measures model or heavily adjusted regression can break the link between the claimed operating characteristics and the actual procedure.

Exact identity is not always necessary if approximations are justified, but the relationship should be documented. The more complex the design, the stronger the case for simulation under the intended analysis.

31. Model uncertainty belongs outside the calculator too

A formula can be exact under assumptions that are only approximate in the world. Outcome distributions may be skewed. Variances may differ. Clusters may be unbalanced. Missingness may depend on outcomes. Effects may vary across people.

Do not confuse exact arithmetic with exact knowledge. The result “N = 186” should often be read as “under these assumptions and this procedure, 186 is the calculated target.” The assumptions deserve at least as much attention as the final integer.

32. Bayesian designs have different planning targets

Bayesian studies may plan sample size around posterior interval width, posterior probability of a clinically or scientifically meaningful effect, predictive probabilities or decision utility. They can also use frequentist operating characteristics as external checks.

The important principle is consistency between the inferential framework and the planning criterion. A Bayesian analysis does not eliminate the need to plan information. It changes the questions used to evaluate adequacy.

33. Equivalence and non-inferiority require different thinking

A conventional superiority study asks whether evidence supports a difference in a specified direction or magnitude. Equivalence and non-inferiority studies ask whether an effect remains within a predefined margin.

The margin is a substantive boundary, not a convenient statistical constant. Sample size depends on that margin, expected effect, variance and chosen error control. A weakly justified margin can make a well-powered study answer the wrong practical question.

34. Prediction models need events and effective complexity, not just total N

When the job is prediction, sample size planning must consider outcome prevalence, number and form of candidate predictors, shrinkage, overfitting and desired calibration or prediction precision. A crude “ten events per variable” heuristic is not a universal design rule.

Prediction is owned by How Forecasting and Prediction Work. This article’s contribution is the planning principle: information requirements depend on model complexity and evaluation goals.

35. Machine-learning train/test splits change the information allocation

Holding out a test set protects evaluation from model tuning but reduces the data available for fitting. Cross-validation reuses observations across folds but creates a different dependence structure in performance estimates.

Do not calculate N as if every observation performs every role independently. Training, tuning and evaluation are distinct information jobs.

36. The smallest effect of interest should survive contact with the real decision

A target difference should be interpretable against cost, burden, alternatives and measurement uncertainty. If a three-point gain changes nothing about the decision, powering the study to detect three points may be scientifically interesting but operationally unhelpful.

Conversely, if a two-point gain would affect thousands of people at negligible cost, even a small average effect can matter. Practical importance is contextual.

37. Value of information provides a deeper stopping question

More data generally reduces sampling uncertainty, but more data is not always worth its cost. If the decision would remain the same across the remaining plausible effect range, additional precision may have low immediate decision value.

How Value of Information Works formalises that question. Power analysis asks whether a design can detect a specified alternative. Value-of-information analysis asks whether resolving uncertainty is expected to improve a decision enough to justify the research.

38. A sample size calculation should be reproducible

Record the formula or software, version, statistical test, alpha, target power, allocation, effect, variance, event rate, correlation, cluster size, attrition and every inflation factor. A future reader should be able to reconstruct the target N from the stated assumptions.

If one parameter came from a prior study, cite it. If it came from expert judgement, say so. If it was chosen as a stress scenario, label it. Numbers without provenance are difficult to challenge and impossible to audit.

39. Preregistration protects the design story

A sample size rationale is most informative when it exists before the outcome is known. Otherwise the target can be retrofitted to the collected N or to the effect that happened to appear.

See How Preregistration and Registered Reports Work for separating planned confirmatory decisions from later exploratory learning.

40. Deviations can be legitimate if they remain visible

Recruitment may be slower than expected. Event rates may differ. A validated variance estimate may become available. A data-monitoring committee may approve a pre-specified sample-size re-estimation.

Changing the design is not automatically misconduct. The key questions are whether the change was authorised, whether it used unblinded outcome information, how it affects error rates and whether the original and revised plans are transparently reported.

41. A worked planning ledger

Planning field	Constructed example	Why it matters
Primary outcome	Score at eight weeks	Defines measurement and time
Smallest important difference	5 points	Defines substantive target
Expected SD	10 points	Defines noise scale
Alpha	0.05 two-sided	Defines Type I error procedure
Power	0.90	Defines Type II error target at specified effect
Allocation	1:1	Affects efficiency
Expected attrition	15%	Affects recruitment target
Analysis	Pre-specified adjusted regression	Should match calculation
Sensitivity	SD 8–14; attrition 5–25%	Shows fragility of N

None of these numbers becomes true by entering it into software. The planning ledger makes their status inspectable.

42. A design can be feasible and still scientifically weak

A research team may be capable of recruiting fifty learners. If the smallest important effect requires five hundred under the proposed design, the feasibility constraint does not make fifty adequate.

Possible responses include changing the outcome to a more precise measure, using a stronger within-person design where justified, pooling sites, narrowing the target question, extending recruitment, or explicitly conducting a feasibility study rather than an efficacy study.

43. A scientifically strong design can still be operationally impossible

The reverse problem also occurs. A power calculation may demand 12,000 participants in a population of 3,000 eligible people. That is not a reason to invent the remaining 9,000. It is evidence that the planned inferential target cannot be achieved through that design in that population.

Research planning is an optimisation under constraints, not a calculator that overrides reality.

44. The design should fail honestly when the assumptions fail

Suppose observed attrition doubles, ICC is much larger than expected and recruitment stops early. The study’s original power statement no longer describes the realised design. The correct response is to report the realised sample, deviations, effect estimates and uncertainty.

Do not preserve the appearance of the planned design after the information architecture has changed.

45. A practical checklist before collecting data

1. Write the population and target quantity.
2. Choose the primary outcome and time point.
3. Define the smallest effect or precision that matters.
4. Specify the design and analysis.
5. Source the expected variance, event rate or correlation.
6. Set and justify alpha, power or interval-width targets.
7. Account for clustering and repeated measures.
8. Plan for missingness and attrition without pretending inflation removes bias.
9. Account for multiplicity or interim looks.
10. Calculate the target under several plausible scenarios.
11. Check recruitment and follow-up feasibility.
12. Record formula, software and version.
13. Pre-specify authorised adaptation rules.
14. Separate the required complete-case information from recruitment inflation.
15. State what the realised design will be unable to exclude if recruitment falls short.

46. A practical checklist after the study

1. Report the planned and realised sample sizes.
2. Report recruitment, exclusions and attrition by group where relevant.
3. Explain deviations from the sample-size plan.
4. Report the effect estimate and uncertainty.
5. Do not use post-hoc observed power as a substitute for the interval.
6. State which important effect sizes remain compatible with the data.
7. Describe whether missingness, clustering or model assumptions reduce effective information.
8. Update future planning with the new evidence without rewriting the original plan.

47. The Library ownership boundary

This article owns the general planning problem: how much information is needed for a quantitative inferential job. It does not replace specialist owners.

• Experimental Design owns randomisation, controls, blocking and causal experimental structure.
• Statistical Inference and Uncertainty owns estimates, confidence intervals, p-values and Bayesian reasoning.
• Clustered and Multilevel Data owns hierarchical dependence.
• Multiple Testing and Sequential Analysis owns multiplicity and repeated looks.
• Time-to-Event and Survival Analysis owns censoring, risk sets and hazards.
• Value of Information owns whether reducing uncertainty is worth the cost.

48. What a learner should remember

There is no meaningful sentence of the form “N = 100 is enough” without a question. Enough to estimate what? Enough to detect which difference? Enough under what variance, clustering, follow-up and analysis? Enough for which decision?

The deepest habit is to treat sample size as an information contract. The researcher promises to collect enough evidence for a defined job, under declared assumptions, and later tells the reader whether that contract was actually fulfilled.

Sources and further reading

Sources were checked for this edition in September 2026. The article is an explanatory synthesis rather than a design calculator. Complex, high-stakes or regulated studies require design-specific statistical and ethical review.

• Hopewell and colleagues, CONSORT 2025 explanation and elaboration: updated guideline for reporting randomised trials, BMJ, 2025.
• Lakens, Sample Size Justification, Collabra: Psychology, 2022.
• CONSORT 2025 statement record, PubMed.
• CONSORT 2025 expanded checklist, Oxford Research Archive.

Continue through eduKate: Experimental Design → Statistical Inference and Uncertainty → Sensitivity Analysis and Robustness Checks → Value of Information → Research Collections Directory.