A number is a mathematical object. A written numeral is a representation of that number. A tokenizer receives the written representation, not the abstract quantity itself.
This distinction explains why digit strings can be split in ways that look unnatural to a mathematician. The tokenizer is optimising representation of surface text under a finite vocabulary. Place value, magnitude, units and arithmetic structure must be learned or supplied above that surface segmentation.
This article continues the eduKateSingapore Representation and Tokenisation series.
The Numerical Representation Route
QUANTITY / DATE / MEASUREMENT → WRITTEN NUMERAL + SYMBOLS → NORMALISATION / LOCALE → TOKENISATION → TOKEN IDS → CONTEXTUAL MODEL → MATHEMATICAL / TEMPORAL INTERPRETATION → OUTPUT STRING OR TOOL ACTION → VERIFICATION
1. A Numeral Is Not the Number
The quantity twelve can be written as 12, XII, twelve, 0b1100 or another representation. These forms can refer to the same underlying quantity while producing completely different token sequences.
Numerical reasoning begins after representation.
2. Digit Strings Are Surface Sequences
A text tokenizer sees characters or byte-derived patterns such as 2, 02, 2026 or 20 + 26. The learned segmentation depends on corpus frequency and vocabulary design.
There is no universal rule that one human number equals one model token.
3. Common Numbers Can Become Compact
Frequently occurring years, percentages or number fragments can receive dedicated token identities because they appear often in the tokenizer corpus.
Compactness reflects surface frequency, not mathematical importance.
4. Nearby Numbers Can Tokenise Differently
2024, 2025 and 2026 can be segmented differently under one vocabulary if different substrings became learned tokens.
Numerical closeness does not imply token-level closeness.
5. Place Value Is Not Guaranteed by Token Boundaries
A tokenizer may split 12500 as 12 + 500, 125 + 00 or another sequence. None of these segmentations is guaranteed to align with thousands, hundreds, tens and ones.
The model must learn place-value relationships above the surface cut.
6. Leading Zeros Change Identity Without Changing Magnitude in Some Tasks
007 and 7 can represent the same numeric magnitude but different identifiers. In account codes, serial numbers or filenames, the zeros may be essential.
Numerical normalisation should therefore distinguish quantity from identifier.
7. Decimal Points Carry Structure
3.14 contains punctuation that changes numerical interpretation. A tokenizer can split the digits and point into several pieces, but the model still needs to reconstruct the decimal relation.
Punctuation is structural, not decorative.
8. Decimal Separators Are Locale-Dependent
Some locales use a comma where others use a point. 1,5 can mean one and a half in one convention while comma elsewhere may mark a thousands grouping.
Tokenisation preserves the symbols; locale context supplies the intended semantics.
9. Thousands Separators Are Also Locale-Dependent
1,234, 1 234 and 1.234 can represent the same quantity under different conventions.
A normaliser must not rewrite them without knowing the locale.
10. Negative Signs Are Semantically Large, Visually Small
The difference between 5 and -5 is one character but changes the number’s sign completely.
Token size does not measure conceptual consequence.
11. Minus, Hyphen and Dash Are Different Characters
A document can contain a mathematical minus sign, an ASCII hyphen-minus or a typographic dash. They may look similar but have different code points and uses.
Aggressive punctuation normalisation can corrupt mathematics.
12. Fractions Have Several Surface Forms
One half can appear as 1/2, 0.5, 50%, the single character ½ or words.
Equivalent mathematical values do not imply equivalent token sequences.
13. Percentages Combine Number and Unit-Like Meaning
20% represents a ratio scaled by one hundred. The percent sign may be its own token or part of a larger token, but the model must interpret the composite expression.
The semantic unit is larger than the digit string.
14. Scientific Notation Adds Syntax
6.02e23 or 6.02 × 10^23 represents a value through a notation system with mantissa, base and exponent.
The tokenizer sees a surface sequence; mathematical parsing reconstructs the notation’s structure.
15. Exponents Require Structural Parsing
10^2 is not the same operation as the sequence of symbols read independently. The caret or superscript notation establishes exponentiation.
See Mathematical Tokenisation.
16. Units Change What a Number Means
5, 5 m, 5 kg and 5 s share a numeral but represent different physical quantities.
A numerical system that strips units preserves magnitude while destroying dimension.
17. Unit Symbols Are Compact Semantic Tokens
m can mean metre, minute in some informal contexts, or a variable in mathematics. Unit interpretation depends on domain and neighbouring structure.
Context disambiguates symbols just as it disambiguates words.
18. Compound Units Have Grammar
m/s, kg·m/s² and similar expressions combine units through multiplication, division and powers.
Correct reasoning requires parsing the unit expression, not merely recognising the characters.
19. Currency Symbols Add Jurisdiction and Unit
$100 is ambiguous without currency context. It could mean Singapore dollars, US dollars or another dollar-denominated currency.
A currency token does not fully identify the monetary unit.
20. Currency Codes Are Safer Machine Identifiers
Explicit codes such as SGD or USD reduce ambiguity. A system can preserve the original symbol for display while storing the canonical currency separately.
This is entity-style canonicalisation applied to units.
21. Dates Are Structured Numbers, Not Mere Numerals
04/09/2026 contains day, month and year fields under one convention, but the field order is ambiguous internationally.
A tokenizer can reconstruct the string perfectly and still leave the date unresolved.
22. ISO-Style Dates Reduce Ambiguity
Representations such as 2026-09-05 use an explicit year-month-day order and support lexical sorting more naturally.
A canonical date representation improves machine interoperability even if users see another display format.
23. Time Zones Are Part of Temporal Meaning
09:00 is incomplete when coordination across locations matters. Local time, UTC offset and time-zone rules can change which real instant is meant.
Time needs metadata beyond the digits.
24. Timestamps Should Preserve Their Zone Contract
A machine timestamp can use an offset-aware standard representation while the user interface renders local time separately.
Separate storage identity from display convenience.
25. Phone Numbers Are Identifiers, Not Quantities
A phone number should not be added, averaged or stripped of leading zeros as if it were a magnitude.
Its digits encode identity and routing rather than quantity.
26. Postal Codes and Account Numbers Behave the Same Way
They may consist entirely of digits while functioning as categorical identifiers.
Numeric appearance does not determine semantic type.
27. Serial Numbers Need Exact Copy Fidelity
A model can “understand” that a string is a serial number while reproducing one digit incorrectly.
For exact identity, copy from the authoritative source rather than regenerate from probabilistic memory.
28. Numerical Tokenisation Interacts With Arithmetic
If operands span several tokens, the model must coordinate those pieces before applying learned arithmetic relationships. Different token boundaries can therefore change the learning path.
Tokenisation is not the whole explanation of arithmetic capability, but it is part of the interface through which arithmetic is learned.
29. Carrying and Borrowing Are Structural Operations
Column arithmetic relies on place alignment. Text token boundaries need not preserve that alignment explicitly.
External calculators and symbolic tools can represent arithmetic in a structure better matched to the operation.
30. Tool Use Can Bypass Weak Numerical Representation
A language model can parse a user’s intent and send exact operands to a calculator or code tool. The tool then performs arithmetic under formal numerical semantics.
Strong systems route tasks to representations designed for them.
31. Floating-Point Numbers Add Another Representation Layer
A decimal string such as 0.1 can be parsed into a binary floating-point representation that may not equal the decimal value exactly.
The pipeline becomes text representation → numeric parsing → machine-number representation.
32. Exact Decimal Arithmetic May Need Decimal Types
Financial calculations often require decimal arithmetic or integer minor units to avoid binary floating-point surprises.
The correct numeric representation depends on the receiver’s precision requirements.
33. Very Large Integers Challenge Fixed Numeric Types
A text string can represent an arbitrarily long integer while a machine numeric type may overflow. Big-integer libraries solve this with a different internal representation.
Text coverage and computational numeric range are distinct concerns.
34. Significant Figures Carry Scientific Meaning
2.0 and 2.000 can represent the same nominal quantity while communicating different measurement precision.
A normaliser that collapses trailing zeros can erase evidence about significant figures.
35. Uncertainty Should Be Represented With the Value
A measurement such as 10.2 ± 0.3 cm contains central estimate, uncertainty and unit. Returning only 10.2 creates false precision.
Numerical fidelity includes uncertainty structure.
36. Ranges Are Not Single Numbers
5–10 kg represents an interval. Treating the dash as punctuation to ignore destroys the range relation.
Surface symbols encode mathematical grammar.
37. Tables Provide Numerical Context
A cell containing 42 is uninterpretable without its row label, column heading and units. Flattening a table can separate the number from its schema.
Document structure must survive tokenisation.
38. Numerical Search Needs Typed Representations
Lexical search for “100” cannot naturally answer “values greater than 100”. Search systems should parse typed numeric fields so range and sort operations use numerical semantics.
Text tokens are useful for discovery; typed values are better for numerical operations.
39. Numerical Evaluation Needs Separate Tests
Benchmark exact copying, arithmetic, date interpretation, currency conversion, units, scientific notation and long identifiers separately from ordinary language fluency.
A model can write beautifully and still fail one digit.
40. The Numerical Tokenisation Audit
- Is the string a quantity or an identifier?
- What locale governs decimal and grouping separators?
- Are sign characters normalized safely?
- How are fractions and percentages represented?
- How is scientific notation parsed?
- Are units preserved and canonicalized separately?
- Are currencies explicit?
- Are dates resolved with a known format?
- Are timestamps zone-aware?
- Are leading zeros identity-critical?
- What precision and significant figures must survive?
- How is uncertainty represented?
- Should arithmetic be routed to a formal tool?
- Can exact strings return to source evidence?
41. What Students Should Remember
- A numeral is a representation of a number.
- Token boundaries do not guarantee place-value boundaries.
- Dates, units and currencies add semantics around digits.
- Numeric-looking strings can be identifiers rather than quantities.
- Locale matters for separators and date formats.
- Exact-copy tasks need stronger controls than ordinary prose.
- Formal tools use representations better suited to exact arithmetic.
42. The Deep Principle
Numbers reveal the gap between surface representation and abstract structure especially clearly. A language model receives symbols first and must reconstruct quantity, unit, time or identity from those symbols and their context.
Digits are tokens in text, but quantities live in a mathematical system. Reliable numerical reasoning begins when the representation preserves the bridge between the two.