Good classification does not require pretending to know more than the evidence allows.
Reality is often incomplete when a decision must be made. A fossil is fragmentary. A photograph has no caption. A medical test is equivocal. A historical source is disputed. A machine-learning model assigns two categories almost equal probability. A newly discovered object does not fit the existing taxonomy. A record is old enough that the original context has disappeared.
The weakest classification systems react by forcing a confident label anyway.
The stronger systems preserve uncertainty as information.
Quick answer: how should uncertainty be represented in classification?
- Unknown: the relevant fact is not known.
- Unclassified: the fact may be known, but the scheme has not yet been applied.
- Uncertain: evidence exists but does not support a confident assignment.
- Probable: one category is currently better supported than alternatives.
- Possible: a category remains plausible but is not the leading explanation.
- Disputed: credible sources support competing assignments.
- Provisional: a temporary classification is being used pending better evidence.
- Not applicable: the dimension does not meaningfully apply.
- Outside scope: the item belongs beyond the current system boundary.
The broader framework is How to Categorise Anything. This article focuses on the part many systems try to hide: what to do when the answer is not fully known.
1. Uncertainty is not the same as ignorance
Ignorance means relevant knowledge is absent. Uncertainty means evidence exists but does not support one fully secure conclusion.
Those states need different responses. Ignorance may require data collection. Uncertainty may require comparison, probability, expert review or waiting for the system to evolve.
2. A blank field is not enough
A blank can mean unknown, unavailable, not applicable, withheld, not yet reviewed or lost.
Collapsing those meanings into one empty value destroys useful information and makes later audit difficult.
3. Unknown should be explicit
“Unknown” is a legitimate knowledge state.
An honest unknown prevents a system from creating false precision merely because a database field requires a value.
4. Unclassified is different from unknown
A record can contain all the evidence required for classification and still remain unclassified because no classifier has reviewed it yet.
Workflow state should not be confused with knowledge state.
5. Not applicable protects meaning
A mountain does not have an employee count. A physical specimen may not have a publication language. A person may not have a document format.
“Not applicable” prevents nonsensical dimensions from being treated as missing data.
6. Provisional classification is useful
Operations sometimes need a temporary answer before the evidence is complete.
A provisional label says: use this category for now, but keep the classification open to revision.
This is better than either freezing the workflow or pretending the temporary answer is permanent truth.
7. Probable classification needs a basis
“Probably Category A” is meaningful only if the system knows why A is favoured.
The basis may be statistical evidence, expert judgement, multiple converging features or a formal scoring rule.
Confidence without an evidence model becomes decoration.
8. Possibility should not be mistaken for probability
An item can possibly belong to several categories even when one is much better supported.
Store candidates and ranking where useful rather than treating every plausible option as equally likely.
9. Disputed classification is a different state again
Dispute does not necessarily mean lack of evidence. It can mean credible authorities interpret the evidence differently.
A historical event, archaeological object, medical diagnosis or legal category may carry competing classifications with strong arguments on several sides.
A mature system can preserve both claims with provenance.
10. Confidence should attach to the classification claim
Confidence is not a property of the object itself. It is a property of the claim that the object belongs to a category under a given evidence state.
This distinction matters when new evidence arrives.
11. Confidence scales need definitions
Low, medium and high confidence sound clear until different users interpret them differently.
Define what each level means operationally. If numerical probabilities are used, define how they are calibrated and what evidence produces them.
12. Probability is not always available
Some domains support quantitative probabilities. Others rely on qualitative expert confidence.
Do not invent numbers merely because numbers look more scientific.
13. Evidence quality should be represented separately
A high-confidence classification based on poor evidence may indicate overconfidence.
Store evidence quality, source reliability and classification confidence as distinguishable dimensions where the stakes justify it.
14. Provenance turns uncertainty into auditable knowledge
Record who assigned the category, when, from which evidence, under which taxonomy version and using which rule.
Without provenance, later reviewers cannot tell whether uncertainty came from weak evidence, old definitions or classifier disagreement.
15. Uncertainty can be local to one dimension
An object may be securely classified by type while its date remains uncertain.
A document may definitely be a letter, probably date from the 1930s, possibly originate in Singapore and have disputed authorship.
This is one reason faceted classification is powerful: uncertainty can attach to the coordinate that is uncertain rather than contaminating the entire object.
16. Multiple candidates can coexist
A classifier may return:
- Category A — leading candidate;
- Category B — plausible alternative;
- Category C — weak alternative.
Keeping alternatives is valuable when the cost of premature closure is high.
17. Thresholds convert uncertainty into action
Operational systems often need a rule such as: auto-accept above one threshold, send to review within an intermediate band and reject or hold below another threshold.
The threshold is an action policy, not the underlying truth of the classification.
18. Error costs should shape thresholds
If false inclusion is expensive, require stronger evidence before assigning the category. If false exclusion is more dangerous, use a more sensitive threshold and send ambiguous cases for review.
The right threshold depends on consequence.
19. High-stakes categories need review paths
Where classification affects rights, safety, treatment, access or significant resources, uncertain cases should have a clear human review or appeal mechanism.
Automation can help triage. It should not make uncertainty invisible.
20. “Other” should not absorb uncertainty
An item that could be A or B is not necessarily “Other”.
“Other” means the item does not fit named categories. Uncertainty means the evidence does not currently let us choose confidently among categories.
21. New phenomena can create structural uncertainty
Sometimes the problem is not evidence about the item. The taxonomy itself is missing the right category.
Repeated low-confidence classifications can therefore be a sensor for schema change.
22. Boundary cases deserve a named review state
Some items sit exactly where the category definition changes.
Marking boundary cases makes them available for future taxonomy testing instead of allowing them to disappear into ordinary records.
23. Disagreement between classifiers is information
If independent classifiers repeatedly disagree on the same branch, the category definitions may be unstable.
The testing framework in How to Test a Classification System treats disagreement as a diagnostic signal rather than merely a performance failure.
24. Uncertainty should be time-stamped
A classification can be uncertain today and secure next year.
Store the date of the evidence state so later systems do not treat yesterday’s uncertainty as permanent.
25. Reclassification is normal
When new evidence changes the best-supported category, update the current classification while preserving the previous claim in history.
Revision is not evidence that the earlier system failed. It may be evidence that the system learned.
26. World change and knowledge change are different
A record can change category because the object changed or because our understanding changed.
These events should be distinguished. One is a change in the world. The other is a change in the model of the world.
27. Schema change is different again
A category can change because the classification system itself was revised.
That requires versioning and crosswalks so the record remains interpretable across taxonomy versions.
28. Uncertainty can propagate
If a derived category depends on an uncertain upstream claim, the downstream conclusion should not become magically certain.
Inference systems need rules for carrying uncertainty through chains of reasoning.
29. Ontologies can preserve uncertain claims
The ontology model described in How Ontologies Work can separate an entity from a claim about that entity and attach provenance, confidence and competing claims to the relationship.
30. AI classification needs calibrated uncertainty
Machine classifiers often output confidence scores, but scores are useful only when their meaning is tested against real outcomes.
A model that says 90% confidence should be correct roughly 90% of the time across comparable cases if the score is well calibrated.
31. Confidence ranking can support human review
Low-confidence cases can be routed to experts while high-confidence routine cases are automated.
This converts uncertainty from a weakness into a workflow signal.
32. Active learning can target uncertain cases
In machine-learning workflows, the most uncertain or disagreement-heavy cases can be selected for human labelling.
Those reviewed examples then improve the classifier or reveal that the taxonomy itself needs repair.
33. Uncertainty has several sources
- Measurement uncertainty: observations are noisy.
- Source uncertainty: evidence may be unreliable.
- Boundary uncertainty: category definitions overlap.
- Model uncertainty: classifier lacks enough experience.
- World uncertainty: the phenomenon itself is variable or stochastic.
- Schema uncertainty: the taxonomy may be incomplete.
Different sources require different repairs.
34. Classify the uncertainty itself
For difficult systems, uncertainty deserves its own taxonomy.
Knowing whether uncertainty comes from missing data, conflicting evidence, weak definitions or novel cases determines the next action.
35. A practical uncertainty record
- item ID;
- dimension or classification claim;
- candidate category;
- confidence;
- uncertainty type;
- evidence used;
- source quality;
- alternative candidates;
- classifier;
- taxonomy version;
- review status;
- review date.
36. A practical uncertainty workflow
- Collect evidence.
- Identify candidate categories.
- Apply written criteria.
- Record confidence.
- Record evidence quality.
- Preserve alternatives if relevant.
- Route high-impact ambiguous cases to review.
- Store provenance.
- Set a review condition or date.
- Reclassify when evidence or schema changes.
37. Uncertainty should improve the taxonomy
Aggregate uncertain cases by category and cause.
If the same boundary repeatedly creates uncertainty, rewrite the criteria. If novel cases cluster together, consider a new category. If one source creates most uncertainty, improve the evidence pipeline.
38. Do not reward false certainty
Systems often reward completed forms, decisive answers and high automation rates.
If those metrics punish “unknown” and “needs review”, classifiers will fill gaps with unjustified confidence.
Measure calibration and correction quality, not only decisiveness.
39. Uncertainty is compatible with action
Knowing less than everything does not mean doing nothing.
Operations can use precautionary rules, provisional routes, review thresholds and reversible decisions while preserving the uncertainty underneath.
40. The deeper idea
Classification is often presented as certainty: this belongs here; that belongs there.
Real knowledge is more interesting. It contains confidence, disagreement, missing evidence, changing definitions and revision.
A classification system becomes more trustworthy when it can say not only what it thinks, but how sure it is and why.
Final answer
Categorise uncertain cases without hiding the uncertainty. Distinguish unknown, unclassified, probable, disputed, provisional, not applicable and outside scope. Attach confidence to claims, not objects. Preserve evidence and provenance. Route consequential ambiguity to review. Reclassify when evidence improves or the taxonomy changes.
The goal is not to force every object into a confident box. It is to build a system that remains useful while knowledge is still becoming clearer.