Sixty learners complete an assessment. Their average score is 80. A report announces that the hundred learners enrolled in the programme achieved an average of 80.
The arithmetic is correct for the sixty observed scores. The sentence about the hundred learners is not established. Forty outcomes are missing, and the reason they are missing may be part of the story the report needs to understand.
Missing data analysis is the discipline of deciding what incomplete observations can support, under which assumptions, and with how much uncertainty. It investigates the process that made information unavailable before choosing whether to recover records, analyse complete cases, weight observations, model incomplete data, impute values or report bounds instead of a single answer. It does not turn guesses into measurements.
This guide concerns research and analytical judgement, not instructions for awarding examination marks or changing official records. Every numerical case below is constructed for explanation; none describes eduKate students, operational results or a published experiment.
Reading route: Start with what a blank means, work through the hundred-learner example, distinguish missingness mechanisms, then examine analysis choices, multiple imputation and the reporting standard. The central question throughout is: who or what disappears when we insist on a complete table?
A blank is a state of knowledge, not a number
A blank cell can mean that a question was not asked, an instrument failed, a person declined to answer, a record arrived late, an answer was illegible, a value was withheld for privacy or the question did not apply. These situations can look identical in a spreadsheet while requiring different treatment.
Consider a library recording the year of publication of donated books. An empty field might mean that the cataloguer has not yet looked, that the title page is missing, that the edition has no printed date, or that the date is disputed. Assigning the current year to all four cases creates a neat column and a false history. The first repair is semantic: preserve what is known about why the value is absent.
Zero is different. Zero loans means a recorded count of no loans during a defined interval. A missing loan count means the count is not available. Not applicable is different again: a digital document may have no physical shelf location. Treating these three states as equivalent can distort totals, denominators and the apparent performance of a service.
A practical record therefore separates the value from its status. The value can remain empty while a companion field records unobserved, not applicable, withheld, pending, invalid or disputed. That is an analytical design recommendation, not a universal coding standard. Its purpose is to prevent later users from having to infer the meaning of silence.
The wider infrastructure belongs to Data Quality and Metadata and Data Lineage. Here the narrower concern is what happens to an inference when the underlying observation is absent.
What the hundred-learner example actually tells us
Return to the fictional assessment. There are one hundred enrolled learners, sixty observed scores and an observed mean of 80. Assume the score scale genuinely runs from 0 to 100 for every learner. The observed scores sum to 4,800. Let the average score among the forty missing learners be x.
Whole-group mean = (4,800 + 40x) ÷ 100 = 48 + 0.4x.
This equation exposes the assumption hidden inside the headline. Reporting 80 for the full group implicitly sets x to 80. The observed data alone do not establish that value. The missing group could have the same average, a lower average or a higher average. The explanation for non-completion determines which possibilities deserve consideration.
| Assumed missing-group mean | Resulting whole-group mean | Interpretation |
|---|---|---|
| 40 | 64 | The observed group substantially overstates the full-group mean. |
| 55 | 70 | The whole group reaches a hypothetical threshold of 70. |
| 80 | 80 | The full-group mean equals the observed mean. |
| 100 | 88 | The missing learners all attain the scale maximum. |
With nothing beyond the score bounds, the whole-group mean lies between 48 and 88. This is an identification interval: a range consistent with the observed totals and stated restrictions. It is not a 95% confidence interval and does not assign equal probability to all values inside it.
The equation also gives a useful tipping point. To support a whole-group mean of at least 70, the missing-group average must be at least 55. Instead of arguing vaguely about whether forty missing assessments are too many, the researcher can ask whether there is evidence that makes an average below 55 implausible.
Perhaps most absences came from a documented scheduling clash unrelated to attainment. Perhaps learners who struggled most were also least likely to return. Perhaps a technical fault affected one class with a distinctive starting level. Those possibilities imply different research tasks. The algebra cannot decide between them; it tells us exactly what needs to be investigated.
Define the question before choosing the repair
The same incomplete dataset can legitimately answer different questions. What was the mean among completers? What was the mean among all enrolled learners? What would the mean have been if every learner had taken the assessment under the same conditions? Did the programme improve learning compared with an alternative? These are not interchangeable.
The first question is descriptive and may be answered directly from the sixty records. The second requires handling the unavailable outcomes. The third introduces a hypothetical completion condition. The fourth adds causal identification: even complete follow-up would not, by itself, establish what the programme caused.
Write the target quantity in words before giving it a technical name. Specify the population, outcome, time point and comparison. This quantity is often called the estimand. Missingness methods should serve it rather than quietly replacing it with whatever the remaining records make easiest to calculate.
A seemingly cautious choice can still change the question. Restricting an analysis to people who attended every session may remove missing outcomes, but it now describes unusually persistent participants. That can be a useful subgroup analysis. It should not be presented as the programme’s result for everyone who began.
Three mechanisms that change the meaning of absence
The standard distinction between missing completely at random, missing at random and missing not at random concerns the relationship between observation and the data, not whether a blank looks irregular on a screen. Stef van Buuren’s explanation of MCAR, MAR and MNAR provides a useful entry to these concepts.
Missing completely at random, or MCAR: the probability of missingness does not depend on the relevant observed or unobserved values. A genuinely random loss of assessment forms could fit this condition. Losing the forms from one particular class would require further thought because class membership may be related to scores.
Missing at random, or MAR: after conditioning on relevant observed information, missingness does not additionally depend on the missing values. It does not mean that nonresponse happened for no reason. An observed starting score or recorded attendance pattern may help account for who is missing.
Missing not at random, or MNAR: dependence on the unavailable values remains after the observed information has been taken into account. Learners with particularly disappointing unrecorded outcomes might be less likely to respond even among learners with the same recorded starting score.
These are modelling conditions, not labels that software can certify merely by inspecting a table. Observed associations may contradict a simple MCAR story. They do not generally prove that MAR holds rather than an MNAR alternative. The missing values are precisely the values we cannot inspect. Investigating the collection process is therefore part of the statistical work, not an optional interview before the real analysis.
The mechanism is relative to what was recorded
Suppose nonresponse depends on internet access. In one dataset internet access was recorded; in another it was not. The same real-world process may be more plausibly handled using an observed-variable model in the first dataset than in the second. Missingness is not simply a permanent property attached to a file.
This makes good preparation valuable. Collecting a small amount of relevant baseline information can be more useful than collecting many irrelevant variables. A record of why a follow-up was missed can help distinguish an administrative delay from refusal, relocation or an outcome-related decision.
However, more recording is not automatically justified. Sensitive details require a legitimate purpose and appropriate safeguards. The analytical desire to explain missingness is not permission to collect everything about a person. Sometimes the responsible result is wider uncertainty, not deeper surveillance.
A missingness review should also distinguish unit nonresponse from item nonresponse. In unit nonresponse, an entire person or record is absent from the response dataset. In item nonresponse, a record exists but some fields are unavailable. A data export containing only responders can conceal the first problem completely unless it is reconciled with an independent list of eligible units.
When complete-case analysis changes the population
Complete-case analysis uses records with all variables required for a particular analysis. It is transparent and can be defensible under appropriate conditions. It can also discard useful information and alter the composition of the group being studied. Sterne and colleagues discuss these trade-offs and the dangers of unexamined fixes in their BMJ paper on multiple imputation.
In a second constructed example, a population contains fifty learners in each of two recorded baseline groups. Suppose their true group means are 80 and 60. The full-population mean is therefore 70. Assume response occurs randomly within each group, but ninety per cent of the first group and fifty per cent of the second group respond.
We then observe forty-five scores with a mean of 80 and twenty-five with a mean of 60. The combined respondent mean is (45 × 80 + 25 × 60) ÷ 70, approximately 72.86. No individual score was altered. The distortion arose because the respondent dataset gave the first group more weight than it had in the target population.
A larger respondent sample with the same response imbalance would not automatically solve the problem. It could make 72.86 more precisely estimated while leaving it wrong for the population mean of 70. Precision and representation are separate questions.
It would also be incorrect to say that complete-case analysis is valid only under MCAR in every conceivable setting. Some analyses remain valid under weaker, analysis-specific conditions. The responsible lesson is narrower: validity must be justified for the target estimand and missingness process, rather than assumed because deletion is easy to explain.
Weighting repairs representation only under an argument
In the two-group example, known response probabilities suggest weights of 1 ÷ 0.9 for first-group respondents and 1 ÷ 0.5 for second-group respondents. The weighted total returns each group to an effective count of fifty, and the weighted mean becomes 70. This is a worked demonstration, not a recommendation to use those weights in real educational data.
Why does it work here? We explicitly assumed that response was random within the recorded groups and that the response probabilities were known. Those assumptions allow the respondents within each group to represent its nonrespondents. If unrecorded outcomes still influence response within either group, the simple weighting argument fails.
Weights also create practical problems. Very low response probabilities imply very large weights, so a few observations can dominate the result. Estimating the probabilities introduces uncertainty. A group with no respondents supplies no observed outcomes to reweight. Weight trimming may improve stability but changes the estimator and can introduce bias; it is not a cost-free repair.
The deeper principle is that weighting rearranges the influence of evidence already present. It does not manufacture direct evidence for an entirely unobserved group. This distinction also matters when moving research to a different population.
Why filling every blank with an average is attractive and risky
Mean substitution appears modest: use the observed average where a value is missing. Yet the completed column now contains invented observations placed exactly at its centre. The apparent spread is reduced, relationships with other variables can change, and an ordinary analysis may treat the inserted numbers as if they had been measured.
In the hundred-learner example, substituting 80 for every missing score produces a mean of 80 because that result has been built into the operation. It provides no new evidence that the missing learners actually averaged 80. The completed spreadsheet has answered the analyst’s assumption, not the original uncertainty.
Replacing unavailable follow-up values with the last recorded value makes a different assumption: that the relevant quantity can be treated as unchanged for the analytical purpose. That may be a defined scenario in a particular problem, but it is not a universally conservative choice. Where outcomes can improve or deteriorate, carrying observations forward can be optimistic in one setting and pessimistic in another.
Interpolating a short gap in a smoothly varying sensor series can be appropriate for a bounded engineering task. Interpolating an absent person’s opinion between two neighbours in a spreadsheet has no comparable justification. The legitimacy of a repair comes from the data-generating process, not the convenience of a software command.
Multiple imputation keeps uncertainty in the calculation
Multiple imputation creates several plausible completed datasets, analyses each and combines the resulting estimates and uncertainty. The observed values remain observed; the imputed values vary between completed datasets. Van Buuren’s three-stage explanation is a useful description of imputation, analysis and pooling.
The purpose is not to discover a missing learner’s exact score. It is to carry uncertainty about missing information into inference about a defined quantity, under an explicit model. A plausible imputed value is still not an observed value. That distinction should survive every export, chart and later reuse.
Imagine three illustrative completed datasets yield estimated means of 70, 72 and 71. Their pooled estimate is 71. Suppose each analysis reports an estimated variance of 4, so the average within-imputation variance is 4. The sample variance of the three point estimates is 1. For the standard scalar pooling setup, the total variance is:
Total variance = within-imputation variance + (1 + 1/m) × between-imputation variance.
Here m is 3, giving 4 + (1 + 1/3) × 1 = 5.33, with a standard error of approximately 2.31. Reporting only the average within-imputation standard error of 2 would omit uncertainty represented by differences between the imputed analyses. The calculation follows the scalar pooling structure documented by the mice developers.
Three datasets are used only to make this arithmetic inspectable. They are not a recommended number for a real analysis. Choosing an adequate number of imputations and constructing confidence intervals require attention to missing information, Monte Carlo error, degrees of freedom and the analysis model.
Crucially, one pools appropriate estimates and uncertainty, not all the completed rows as though the study had suddenly recruited three times as many people. Repetition of an imputation process does not create independent participants.
An imputation model must respect the question
Different variables require different treatment. A binary field cannot take 0.43 as an observed category. A date cannot precede an impossible event. A score bounded by its assessment scale should not acquire physically or institutionally impossible values. Relationships involving groups, time, interactions and nonlinear effects also matter.
The original mice methods paper explains chained-equations imputation and addresses predictor selection, transformations, multilevel structure and diagnostic graphs. It demonstrates why imputation is a model-building task rather than a universal blank-filling function.
For a fictional school study, ignoring class membership might erase genuine differences between classes. Imputing a total independently of its component scores might produce contradictions. Omitting an interaction central to the planned analysis could undermine the very pattern the study is trying to estimate.
The list of variables appropriate for imputation is not identical to the list of causal adjustment variables. An outcome can be useful in a research imputation model, while a future outcome would be unavailable when deploying a prediction system. These are different jobs. Applying one workflow to the other without checking what information exists at decision time risks leakage or misspecification.
Keep the distinction visible: reconstructing a research dataset for retrospective inference is not the same as predicting what a live system will encounter next week. Both can involve missing values, but their information boundaries differ.
Diagnostics can expose problems without certifying truth
Useful checks include whether imputed values respect valid ranges, whether the process behaves stably across iterations, whether relevant relationships remain plausible and whether results change materially across defensible model choices. Inspecting only the final mean misses most of the work.
Observed and imputed distributions need not be identical. In our response-imbalance example, lower-baseline learners were underrepresented among responders. A model that accounts for that difference may appropriately generate an imputed distribution unlike the unweighted observed distribution.
Conversely, visually similar distributions do not prove that a missingness assumption is correct. A smooth-looking output can be generated by an unsuitable model. Diagnostics should be treated as opportunities to detect defects and investigate surprises, not as a certificate that unobserved outcomes have been recovered.
A practical review asks what would have looked implausible before the model was run, whether that occurred, and what additional checks are needed. This avoids inventing a favourable definition of plausibility after seeing the answer.
Sensitivity analysis asks how much the unknown would need to differ
Where an MAR-based analysis is used, it is often useful to explore departures from that assumption. A transparent constructed sensitivity exercise might ask what happens if the missing group’s average is 5, 10 or 20 points lower than the model suggests. These differences must be justified as scenarios; they are not discovered facts.
In the first example, every one-point reduction in the assumed missing-group average lowers the full-group mean by 0.4 points. The sensitivity is easy to inspect because forty per cent of the population is unobserved. If the substantive conclusion depends on staying above 70, the critical missing-group mean is 55.
This is more informative than announcing that the result is robust without saying what was changed. The useful statement is conditional: the conclusion survives the stated range of missing-outcome assumptions, or it reverses once the average missing outcome crosses the stated threshold.
Van Buuren’s discussion of MNAR situations highlights investigating causes and examining alternative scenarios. Our numerical exercise illustrates that logic; it does not supply a validated sensitivity range for any real programme.
Missing outcomes are not every kind of incomplete knowledge
Some problems need a different mathematical structure. A value known to lie below a detection limit is not wholly unknown. An event time may be right-censored because observation ended before the event occurred. A question skipped by design may be outside a respondent’s applicable universe. A suppressed count may carry a confidentiality restriction.
These examples share incomplete information, but ordinary imputation should not be applied indiscriminately. A censored event time carries an inequality and a follow-up process. A structurally inapplicable field may have no meaningful value to recover. Filling either as though it were a lost questionnaire answer can invent the wrong object.
The same caution applies to archives. A missing historical document may never have existed, may have been destroyed or may survive elsewhere. A statistical method cannot decide which history occurred merely because a catalogue has a blank. The missingness framework helps formulate questions, but the evidential work remains domain-specific.
Repair the collection process before perfecting the spreadsheet
The best analytical repair may begin outside analysis. A delayed upload can be reconciled. A duplicated identifier can be corrected against an authoritative record. A form can distinguish refusal from not applicable. A follow-up can be offered through an accessible channel. These changes address the route by which evidence is lost.
For a library, a small audit of uncatalogued books may show that one donation source routinely omits edition details. The useful intervention is then not an increasingly elaborate year-imputation model. It may be a revised intake question and a documented unresolved-date status.
For an educational evaluation, an analysis of completion by class and collection channel may identify a delivery problem. Repairing that problem for the next cycle can improve both participation and interpretation. It should not be used to rewrite the previous cycle as though the missing observations had been collected.
Recovery also has limits. Recontact can impose burden or conflict with consent. An original record may be irretrievable. A later response may concern a different time point. Recovering something is not the same as recovering the intended measurement.
What an honest report preserves
A reader should be able to distinguish the eligible population, the observed population and the analysed population. Report missingness by relevant variable and stage, not only as a single headline percentage. Explain what is known about the process and which explanations remain uncertain.
State the target quantity, the chosen method, the reasons for choosing it and the assumptions on which it depends. For imputation, identify the model structure and pooling approach sufficiently for an informed reader to understand the analysis. For weighting, explain the response model and how extreme weights were handled. For bounds, say which restrictions created the bounds.
Keep observed and imputed information distinguishable in preserved data and documentation. An analyst should not have to reconstruct this distinction from an old email. Preserve the unaltered source data, transformation history and the edition of the analysis used in the published report.
Finally, report what changes under plausible alternatives. A limitation is useful when it identifies the vulnerable conclusion and what would strengthen it. A generic sentence saying that missing data may have affected results gives the reader much less information.
A better version of the opening report
For our first constructed dataset, a defensible initial statement would be: “Sixty of one hundred enrolled learners completed the assessment. Their observed mean was 80. The mean for all enrolled learners cannot be identified from these results alone. On a 0–100 scale, it would be at least 70 only if the forty missing learners averaged at least 55.”
That statement is longer than “the programme achieved 80”. It is also more useful. It states a known result, identifies the population actually observed, exposes the missing part and converts uncertainty into a concrete research question.
If further evidence supports an analysis under a stated missingness model, the report can add the model-based estimate and its uncertainty. The new estimate should supplement the observation record, not erase it. Later readers need to know which parts came from measurement and which came from an inferential bridge.
The educational habit is simple: before asking how to fill the blank, ask what the blank means, whose experience it removes and what assumption would be needed to cross it. A library becomes more trustworthy when its unknowns remain visible enough to be repaired.
Sources, scope and further reading
This is an explanatory synthesis with original hypothetical calculations. It is not a systematic review, a software validation or a prescription for any particular clinical, examination or administrative dataset. Source pages were checked for this edition on 5 September 2026. The cited methods literature supplies the conceptual background; the learner and library scenarios are illustrative applications.
- Stef van Buuren, Flexible Imputation of Missing Data: concepts of MCAR, MAR and MNAR.
- Stef van Buuren, Multiple imputation in a nutshell.
- Sterne and colleagues, Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls, BMJ, 2009.
- Van Buuren and Groothuis-Oudshoorn, mice: Multivariate Imputation by Chained Equations in R, Journal of Statistical Software, 2011.
- mice developer documentation, combining estimates by pooling rules and scalar pooling.
Continue through the Library: Use Surveys and Sampling for who enters a dataset, Statistical Inference and Uncertainty for estimation, and Causal Inference for what an intervention can be said to cause. Return to the Research Collections Directory for the wider route.