A model can produce a fluent answer and still be uncertain. The difficult engineering problem is not merely estimating that uncertainty, but representing it in a form that can guide whether the system answers, hedges, retrieves more evidence, asks a question or abstains.
Uncertainty representation is therefore a bridge between internal probability, evidence state and external decision policy.
This article continues the eduKateSingapore Representation and Tokenisation series. It follows directly from Context Sufficiency: once we know what evidence is present, how should the system represent what remains uncertain?
The Uncertainty Route
MODEL / EVIDENCE STATE → UNCERTAINTY SIGNALS → CALIBRATION → RISK ESTIMATE → ANSWER / HEDGE / RETRIEVE / CLARIFY / ABSTAIN → OUTCOME → RECALIBRATION
1. Probability Is Not the Same as Confidence
A next-token probability describes preference among immediate token continuations.
User-facing confidence concerns whether a meaningful claim or answer is correct. Those are different objects.
2. Token Probability Lives at the Surface Generation Layer
A model can assign high probability to fluent wording around a false claim because the phrase is linguistically predictable.
Fluency probability should not be mistaken for factual reliability.
3. Sequence Probability Is Still Wording-Dependent
Two paraphrases expressing the same claim can have different token probabilities.
Professional decisions need uncertainty over meaning, not merely over phrasing.
4. Semantic Uncertainty Groups Equivalent Answers
If several sampled answers use different words but express the same conclusion, their disagreement is smaller than the raw strings suggest.
Uncertainty should often be measured over semantic states rather than exact token sequences.
5. Disagreement Across Samples Is One Uncertainty Signal
If repeated generations produce different substantive answers, the model is revealing instability.
Self-consistency can therefore provide evidence about uncertainty, though repeated agreement is not proof of correctness.
6. Retrieval Compatibility Is Another Signal
An answer supported cleanly by retrieved evidence is in a stronger state than one contradicting or exceeding that evidence.
Uncertainty can combine model-internal and evidence-external signals.
7. Tool Verification Can Add Operational Evidence
Code execution, calculator output, database lookup or formal validation can confirm or reject parts of an answer.
Verifier results should update confidence rather than remain detached from it.
8. Calibration Connects Confidence to Observed Accuracy
A calibrated system that reports 80% confidence should be correct about 80% of the time across comparable cases.
Without calibration, a confidence number can be persuasive decoration.
9. Confidence Needs a Reference Class
“80% confident” only becomes meaningful when evaluated over a defined distribution of tasks and outcomes.
Calibration can break under distribution shift.
10. Accuracy and Calibration Are Different Properties
A highly accurate model can be overconfident on its remaining errors. A less accurate model can still report probabilities that track its actual success rate more faithfully.
Capability and uncertainty quality need separate evaluation.
11. Confidence Can Be Claim-Level
A long answer can contain one well-supported claim and another speculative one.
One confidence score for the whole paragraph hides this internal variation.
12. Atomic Claims Make Uncertainty Easier to Audit
Breaking a response into factual units allows each claim to be checked against evidence, tools or known references.
Fine-grained uncertainty representation supports selective repair.
13. Ambiguity Is Different From Ignorance
A question can have several plausible interpretations even when the model knows the facts.
Clarification is often better than low-confidence guessing.
14. Evidence Absence Is Different From Model Uncertainty
The system can be internally confident from prior knowledge while current retrieved context remains insufficient.
Evidence sufficiency and model confidence should remain separate state variables.
15. Contradictory Evidence Is a Third Uncertainty Source
Two authoritative sources can disagree because of timing, methodology or definition.
The correct output may be a contested state rather than a single low-confidence number.
16. Abstention Converts Uncertainty Into Action
An uncertainty score alone changes nothing. A selective system uses a policy: answer above some reliability threshold and abstain, retrieve or escalate below it.
This is selective prediction.
17. Abstention Trades Coverage for Risk
A system that answers every question has maximum coverage and may accumulate more errors.
A stricter threshold reduces coverage while improving the reliability of accepted answers.
18. Risk–Coverage Curves Make the Trade-Off Visible
Measure how error rate changes as the system answers a larger fraction of cases.
This is more informative than reporting one arbitrary confidence threshold.
19. Risk-Controlled Refusal Is Stronger Than Heuristic Hedging
Recent work on uncertainty-aware abstention treats refusal thresholds as statistical control problems rather than stylistic decisions.
The goal is not to sound cautious. It is to limit error among the answers the system chooses to provide.
20. Confidence Intervals Can Represent Uncertainty Around the Policy
A finite calibration set only estimates future risk.
Confidence bounds can express how certain the system is that its accepted-answer error remains below a target level.
21. One Threshold Should Not Govern Every Domain Automatically
A brainstorming assistant can tolerate more uncertainty than a medication-checking or payment system.
Risk appetite is part of deployment context.
22. High-Stakes Tasks Need Lower Accepted Risk
The system may require stronger evidence, tool verification or human review before answering decisively.
Uncertainty policy should reflect consequence.
23. Linguistic Hedging Is a User-Facing Representation
Words such as “likely”, “possibly” and “uncertain” convert internal doubt into language.
These phrases are useful only if their meaning is calibrated and consistent.
24. Vague Hedging Can Hide Rather Than Clarify
“It may be…” can be used for 51% confidence or 95% confidence unless the system defines its language.
Human-readable uncertainty needs semantic discipline.
25. Answer Sets Can Represent Several Plausible States
Instead of forcing one answer, a system can return a small set of plausible interpretations or outcomes.
This preserves ambiguity more faithfully than a fabricated single point estimate.
26. Specificity and Factuality Trade Off
A very specific claim carries more information and can be easier to falsify. A broader claim may be more reliably supported.
Uncertainty-aware generation should narrow specificity when evidence does not justify detail.
27. Printed Numerical Confidence Can Be Misleading
A model generating “92% confident” in prose has not necessarily computed a calibrated 0.92 probability.
User-facing numbers require an explicit validated mapping from signals to probability.
28. Semantic Calibration Is More Useful Than Wording Calibration
Professional use cares whether the underlying diagnosis, hypothesis or operational state is correct—not whether one phrase was the most probable continuation.
Recent research increasingly treats this as a mapping from language outputs to meaningful states that can be tested on held-out data.
29. Uncertainty Should Update With New Evidence
A database result, source document or user clarification can reduce ambiguity.
The uncertainty representation should be dynamic rather than attached permanently to the first draft.
30. Contradictory Evidence Can Increase Uncertainty
New information should not always make the model more confident.
A mature system can become less certain after encountering a credible conflict.
31. Memory Quality Affects Confidence Quality
Compressed or stale memory can produce confident answers from distorted premises.
32. Context Sufficiency Should Gate Confidence
If a question requires external evidence and the necessary premise is absent, high model confidence from prior knowledge should not erase the evidence gap.
Reliability requires both adequate evidence and competent reasoning.
33. Calibration Needs Continuous Monitoring
Model upgrades, retrieval changes and new user populations can shift confidence reliability.
Calibration is an operating process, not a one-time certificate.
34. The Uncertainty Representation Audit
- What object is uncertain: token, answer, claim, state or action?
- Which uncertainty signals are available?
- Are signals semantic or merely lexical?
- Is confidence calibrated on held-out data?
- What distribution defines that calibration?
- Can ambiguity be separated from ignorance?
- Can insufficient evidence be separated from model uncertainty?
- What risk threshold governs answering?
- What coverage is retained at that risk?
- Should uncertainty trigger retrieval, clarification, tool use or abstention?
- How are contradictory sources represented?
- Can user-facing confidence language be interpreted consistently?
- How is calibration monitored after deployment changes?
35. What Students Should Remember
- Next-token probability is not the same as answer confidence.
- Calibration connects stated confidence to observed correctness.
- Semantic disagreement can reveal uncertainty hidden by wording variation.
- Evidence insufficiency and model uncertainty are different states.
- Abstention trades coverage for lower accepted risk.
- High-stakes systems should use stricter risk thresholds.
- Confidence should update when new evidence arrives.
36. The Deep Principle
Uncertainty becomes useful only when it changes behaviour.
A trustworthy system does not merely know an answer or fail to know it. It represents how fragile the answer is, what evidence would change it, and when the correct action is to stop speaking and seek more information.