Deleting the slowest trial can reverse an experiment’s conclusion without improving the experiment. Before removing a result, establish whether it is a recording error, a documented breach of the intended procedure or a genuine outcome of the system being tested. Keep the original record, apply a justified rule consistently, and show when the conclusion depends on the exclusion. An inconvenient value is a reason to investigate, not permission to erase it.
The worked comparison below is fictional. It is designed to make the consequences of exclusion visible, not to establish that one real teaching method, device or experimental procedure is better than another.
One deleted number changes the winner
Kai Kai’s project compares two layouts for sorting the same small set of counters. The outcome is elapsed completion time in seconds. The five recorded times for layout A are 8, 9, 10, 11 and 42. For layout B, they are 12, 13, 14, 15 and 16.
Layout A’s total is eighty seconds, so its mean is sixteen. Layout B’s total is seventy seconds, so its mean is fourteen. On the observed mean-time measure, B is faster by two seconds.
The forty-two-second trial looks untidy. Someone calls it an outlier and removes it. A’s retained times now total thirty-eight seconds across four trials, giving a mean of 9.5 seconds. The edited comparison makes A look faster by 4.5 seconds.
No new trial has occurred. The apparent winner changed because the inclusion rule changed. This does not establish that the forty-two-second value must remain in every possible analysis. It establishes that the decision about that value carries the conclusion and needs an explanation independent of the desired winner.
“Unusual” describes a relationship, not a cause
Forty-two is far from the other recorded A times. That observation makes it worth checking. It does not identify why it happened.
Perhaps a digit was copied incorrectly. Perhaps an unrelated interruption occurred. Perhaps the sorting layout occasionally jams or makes a counter difficult to retrieve. These explanations would have different consequences for the analysis.
The NIST guidance on detecting outliers distinguishes erroneous data from unusual observations that may reflect variation or something scientifically informative. It also cautions that outlier identification depends on the underlying statistical assumptions. An apparent extreme is not automatically a bad measurement.
In this five-trial illustration, we have not established a formal outlier classification. Calling the longest trial a potential anomaly is more careful than announcing that a statistical test has proved it invalid.
Begin with a factual description: “Trial A5 took forty-two seconds, substantially longer than the other four A trials.” Keep the explanation open until the record supplies more evidence.
Decide what the experiment is trying to measure
Suppose the question is average completion time during ordinary use of each layout. If A occasionally becomes difficult to operate, that difficulty may belong to the outcome the project needs to measure.
Suppose instead the question is completion time when a specified independent interruption does not occur. A trial affected by that interruption may not answer the intended question, provided the restriction is justified and consistently applied.
These are different targets. The same long duration can be relevant to one and outside the scope of another. The target should not be changed only after a disappointing result appears.
Write the question before deciding the fate of A5. Does the project include stalls, retries and recoveries as part of normal performance? Does timing begin before setup or after it? What counts as completion? Without those definitions, “remove the bad trial” has no stable meaning.
A result cannot be judged fit for purpose until the purpose is clear. This is an experimental-design question before it becomes an averaging question.
A verified transcription error deserves correction
Imagine a separate version of the example in which the original collection sheet clearly records twelve seconds for A5, while the spreadsheet contains forty-two. The discrepancy can be checked against the preserved source.
Correcting the spreadsheet to twelve is then a repair of a known transcription error. A’s corrected times are 8, 9, 10, 11 and 12, with a total of fifty and a mean of ten seconds.
That is not the same as deleting forty-two because it spoils the mean. The corrected value comes from an independent record, not from estimating where the point ought to sit among the others.
Keep a note of the original entry, the verified source and the correction. Do not silently rewrite the collection sheet to match the spreadsheet or leave later collaborators unsure which version was checked.
If the handwriting is ambiguous or the source has been lost, the conclusion must remain more limited. “Twelve would fit the pattern” does not establish that twelve was observed. Plausibility can suggest a question; it cannot substitute for the missing record.
A real jam may be the most relevant result
Return to the original forty-two-second observation. Suppose the notes say that counters became trapped in one part of layout A and had to be retrieved before sorting could finish. No unrelated interruption was recorded.
If the project concerns ordinary end-to-end completion, removing the stalled trial can conceal exactly the behaviour a user needs to understand. The system is not defined only by its smoothest runs.
A fair report might say that A completed four of the five observed trials in eight to eleven seconds, but one took forty-two after a documented snag. B’s observed times were twelve to sixteen seconds. This reports a useful difference in the pattern without declaring one layout universally superior.
The next design question could concern the snag: was it related to the layout, the materials, handling or another condition? Further investigation is needed. The five numbers alone cannot identify the cause.
The important repair is preserving the event long enough to learn from it. Calling it noise prematurely would turn a possible design clue into an invisible omission.
An invalid trial can still belong in the record
Now imagine a trial in which the task was interrupted by an unrelated announcement and everyone stopped working while the timer continued. If the defined experiment excludes such interruptions, that trial may be unsuitable for the intended completion-time comparison.
Keep the record of what happened. Mark its analysis status and the reason rather than erasing the trial from history. An excluded observation can remain part of the collection record without entering a particular summary.
The distinction matters when someone later asks why the sample sizes differ or whether one layout was disrupted more often. A missing row supplies no answer. A documented status explains the analytical boundary.
Apply the same criterion wherever it occurs. If an unrelated interruption invalidates an A trial, the equivalent interruption cannot be ignored in B merely because B’s result already supports the expected conclusion.
A good exclusion rule identifies a relevant condition. It does not identify an inconvenient answer and invent a condition afterwards to make its removal sound methodological.
The fastest and the typical are not the average
The A times have a median of ten seconds; the B times have a median of fourteen. Thus A has the lower observed median even while B has the lower observed arithmetic mean.
There is no arithmetic contradiction. The median identifies the middle ordered observation. The mean allocates the total time across the included trials. The long A trial affects the mean strongly but does not change its middle value in this example.
Choosing the median can be useful when the question concerns a typical position in the observed distribution. It does not make A5 disappear, explain its cause or establish that its delay is unimportant to every user.
The fastest observed A time is eight; B’s is twelve. A comparison of best times answers another question again. It cannot stand in for expected performance across ordinary attempts without further reasoning.
Name the measure in the conclusion. “Lower observed median” is more precise than “better”, and “lower observed mean” is not equivalent to “wins on every relevant feature”.
A deadline question can prefer a different result
Suppose a separate, explicitly stated performance question asks how many observed trials finished within twelve seconds, including exactly twelve. A meets that condition in four of five trials. B meets it in one of five.
The observed proportions are eighty percent and twenty percent. On that particular threshold measure, A performs better in this small dataset despite having the larger mean time.
That does not guarantee the same proportions on future trials. Nor does it justify choosing twelve seconds after examining the values simply because that threshold makes A look attractive. The threshold needs a reason related to the intended task.
The comparison shows why deleting the slowest trial is an inadequate substitute for deciding what performance means. A user may care about typical speed, total workload, severe delays or meeting a threshold. The evidence should be organised around that question, not around a search for a flattering number.
One layout may involve a trade-off rather than a single universal rank. Preserving all the observations makes that trade-off easier to see.
Deleting one trial from each group is not automatic fairness
Someone might try to balance the edit by removing the slowest observation from both layouts. A then averages 9.5 seconds. B’s retained values 12, 13, 14 and 15 average 13.5.
The rule is symmetrical, but symmetry alone does not make it appropriate. The comparison now concerns each group’s fastest four observed trials, not all five. A genuine difficult event in either group has been excluded by design.
A deliberately chosen trimmed summary can have a legitimate analytical purpose. It should be named, justified and used with awareness of what it leaves out. It is not a universal repair for inconvenient data, especially in a tiny exploratory set.
Equal sample sizes are not the only meaning of a fair comparison. If one trial was genuinely invalid and the others were valid, deleting another valid trial simply to make the counts match sacrifices information without necessarily improving validity.
Explain the rule’s connection to the question. Do not confuse “applied equally” with “answers the right question”.
Show the sensitivity without choosing by appearance
A useful sensitivity display for the original example reports A’s mean as sixteen with all five recorded trials and 9.5 after excluding A5. B’s five-trial mean remains fourteen. The direction of the observed mean comparison changes.
This makes the fragile conclusion visible. It does not tell the reader which analysis is correct. That still depends on the status of A5 and the intended outcome definition.
Avoid reporting only the preferred result and placing the conflicting calculation somewhere no reader is likely to notice. When an exclusion determines the main conclusion, the dependency deserves a clear explanation alongside that conclusion.
The Office of Research Integrity’s educational material on data analysis identifies exclusions, missing values and transformations as integrity concerns and emphasises a sufficient record of how data were handled. For a school project, a short transparent note can carry that responsibility without a complex research system.
The useful wording is conditional: “The observed mean advantage reverses when A5 is excluded; its exclusion therefore requires a justified procedural basis.”
A fixed threshold is not a universal truth machine
A calculator, graphing package or online tutorial may flag values using a numerical rule. Such a flag can help direct attention. It does not establish that the underlying observation was recorded incorrectly.
NIST’s outlier discussion notes the role of distributional assumptions and distinguishes flagging, accommodating and formally identifying unusual observations. A method intended for one setting should not be applied mechanically to a different one.
For a school investigation with very few trials, the first useful checks are often the raw entry, units, timing rule and recorded conditions. Adding an impressive statistical label before inspecting those basics can make uncertainty less visible rather than reduce it.
Do not claim that every observation within a numerical cutoff is valid either. A wrong value can fall near the centre of a dataset, while a genuine result can be extreme. Distance from the others is a property of the number’s position, not a complete account of its history.
Repetition should investigate a condition, not hunt for agreement
Repeating an appropriate low-risk trial can help examine whether the slow event recurs. Record what is being repeated, what stays comparable and any change to the method.
Do not keep replacing A5 with new trials until one falls into the preferred range. That procedure selects results for agreeing with the expected answer and can hide how often the unwanted outcome occurs.
If the layout is repaired after a snag, the new runs belong to the revised layout. They may provide valuable evidence about that revision, but they should not silently replace the earlier runs as though the design never changed.
The existing guide to repeating investigations explains the difference between a comparable repeat and a changed method. The question here is which record a later trial is allowed to support.
Use teacher-approved activities and safe materials. There is no need to recreate a dangerous failure or impose stress on a participant simply to make an outlier explanation more convincing.
A trial that never finishes is not a zero-second success
Suppose another run stops after thirty seconds because the task has not finished. Recording zero would make it look like instantaneous completion, the opposite of what happened.
Recording thirty as an exact completion time would also misstate the observation. What is known is that completion had not occurred by the stopping point, together with whatever stopping rule was used.
Keep that status explicit. A restricted demonstration can report the unfinished trial separately and explain why its result is not directly comparable to completed durations. More formal analysis of incomplete event times requires appropriate methods and expertise.
Removing all unfinished runs and averaging only successful completions changes the question to completion time among the runs that finished. That may be informative, but it omits failure to complete from the headline unless the report carries both pieces of information.
The same logic connects with missing-data handling: the absence of an exact outcome is not permission to insert the number that makes the calculation easiest.
Preserve order and context before sorting the times
The original values may contain information about when the long trial occurred. If all early runs were slow and later ones faster, learning or a changing setup might matter. Sorting the times numerically can make that sequence harder to inspect unless trial order has been preserved.
Retain the trial identifier and original collection order. Keep any relevant condition notes attached to the record. Reorganising the display should not detach a time from the event that produced it.
Also ask whether A and B were tested under comparable circumstances. A mean difference cannot by itself isolate the layout’s effect if materials, operator practice or other conditions changed with the layout. Exclusion review does not replace experimental-design review.
The investigation therefore has two separate questions: are the records valid for the stated comparison, and does that comparison justify the causal explanation being proposed? A satisfactory answer to the first does not automatically settle the second.
Protect the context before calculating. It may be the information that tells you whether the unusual trial is an error, a condition change or an important outcome.
Ask whether the same rule would reject a pleasing result
Imagine a remarkably fast trial rather than a slow one. The timer was started after the task had already begun, and the resulting duration makes the preferred layout look excellent. Would the group keep it because the number supports its hopes?
A rule concerned with timing validity should reject that measurement for the same kind of procedural reason that would invalidate a disappointing duration. The direction of the result should not decide whether the rule applies.
Where practical, review the procedural notes before calculating which layout wins. Another reader can also check whether the stated criterion was applied consistently. These steps do not make a small project perfectly unbiased, but they make result-driven exceptions harder to hide.
If a new problem is discovered after collection, it can still require a justified change. Describe what was discovered, why the earlier rule was inadequate and how the revised rule affects every relevant trial. Calling a criterion preplanned when it was invented later would conceal information the reader needs.
Keep an exclusion record small enough to maintain
For each trial under review, retain its identifier, recorded value, relevant condition note, decision and reason. Distinguish corrected, excluded from this analysis, retained and unresolved. These are analytical statuses, not descriptions of whether the person collecting the data did a good job.
For a corrected transcription, point to the source that established the correct entry. For a procedural exclusion, identify the condition that failed and the outcome definition it affects. For an unresolved extreme, explain what was checked and what remains unknown.
The record should also identify which report or chart uses the decision. Otherwise a corrected source can coexist with an older presentation that still shows the preferred but unsupported ranking. Follow the change through the outputs that depended on it.
This need not become a bureaucratic exercise. In a five-trial project, a short note beside the affected row may be enough. The purpose is that another person can understand the decision without having to trust a recollection such as “we removed that one for a good reason”.
Write a conclusion that leaves the uncertainty usable
A defensible description of the original invented dataset is: “Layout A had a lower median time, but its forty-two-second trial raised its five-trial mean above B’s. We have not established that the long trial was invalid. The mean-based ranking depends on whether that observation is included.”
The next sentence should identify the needed investigation: “We will inspect the recorded conditions and timing process, then use a consistent rule justified by the intended outcome.” This is more useful than a generic admission that the experiment had limitations.
If a verified recording error is found, explain the correction. If a genuine snag is found, keep it in the relevant performance account. If its origin cannot be determined, state that uncertainty instead of making the preferred rank decide the classification.
The Learning Manual on residuals and anomalous data provides the wider model-checking route. This worked case adds a specific warning: one discretionary deletion can carry the whole comparison.
The inconvenient result is part of the investigation
Alicia can make the final presentation clearer without making every trial look tidy. She can show the actual observations, explain the summary chosen and identify which conclusion changes when A5 is excluded.
That is a stronger project than a clean chart whose exclusions cannot be reconstructed. The point of an experiment is not to make the expected pattern survive. It is to learn what the observations and method can genuinely support.
Continue through the Science Article Directory for investigation and evidence, and the Mathematics Article Directory for means, medians and data interpretation.
Before deleting the slowest trial, ask what you would be deleting from the explanation. Sometimes it is a verified mistake. Sometimes it is the problem the experiment was supposed to help you discover.
Browse connected guides in the Parent Learning Support Directory.