A time series is a sequence of measurements indexed by time. Tokenising it means deciding how continuous change should be divided into model-ready units without erasing the temporal patterns the receiver needs.
The source may be temperature, heart rate, electricity demand, market prices, traffic flow, machine vibration or student response latency. The representation problem is always similar: sample the changing world, preserve time, choose a granularity, compress where useful and keep enough structure for forecasting, anomaly detection or explanation.
This article continues the eduKateSingapore Representation and Tokenisation series and follows Event Tokenisation.
The Time-Series Route
CHANGING WORLD → SENSOR / OBSERVATION → TIMESTAMPED SAMPLES → CLEANING / ALIGNMENT → RESAMPLING → WINDOWS / PATCHES / BINS → SYMBOLS OR CONTINUOUS EMBEDDINGS → CONTEXTUAL MODEL → FORECAST / CLASSIFICATION / ANOMALY → WORLD RETURN
1. A Time Series Is Already a Discretised World
Temperature changes continuously, but a sensor may record one value every minute. Financial prices change through transactions, but an analyst may store one closing price per day.
The sampling interval is the first tokenisation decision.
2. Sampling Rate Determines What Can Be Seen
Slow sampling can miss short spikes. Fast sampling preserves detail but creates more data and noise.
The right rate depends on the fastest meaningful change the system needs to detect.
3. Uniform Sampling Simplifies Modelling
Many models assume values occur at regular intervals. Real sensors, logs and human observations often arrive irregularly.
Resampling can create a regular grid, but interpolation then introduces derived values that were never directly observed.
4. Interpolation Is a Representation Hypothesis
Filling a missing value by linear interpolation assumes the underlying process changed smoothly between neighbours.
That may be reasonable for temperature and wrong for event-driven financial prices.
5. Missingness Has Meaning
A missing sensor value can mean device failure, network loss, planned shutdown or “not applicable”. Treating all missingness as zero destroys information.
Time-series representations should preserve missingness explicitly.
6. Time Gaps Are Data
Two equal values five seconds apart and five days apart do not represent the same process state.
Irregular-time models often need elapsed time as an input feature.
7. Windows Turn Many Samples Into One Working Unit
A model can group 60 one-minute samples into one hourly window. That window becomes a higher-level token-like unit.
Windowing trades local resolution for shorter sequences.
8. Window Length Controls Temporal Granularity
Short windows preserve rapid change. Long windows capture broader trends but can smooth over brief events.
No window size is universally correct.
9. Sliding Windows Preserve More Context
Overlapping windows share samples so a pattern near one boundary also appears in another.
The cost is repeated data and correlated training examples.
10. Non-Overlapping Windows Are Cheaper but More Boundary-Sensitive
A spike occurring exactly between two windows can be split across both.
Boundary placement can therefore influence detectability.
11. Patches Are Time-Series Tokens
Recent Transformer approaches often group consecutive values into patches and project each patch into an embedding. This is conceptually similar to image patches or text chunks.
The model processes one patch representation rather than every raw sample independently.
12. Patch Size Trades Detail Against Sequence Length
A patch of 16 samples creates fewer model positions than single-sample tokenisation. But abrupt local changes can be compressed inside the patch.
Patch size is a cost-versus-resolution decision.
13. Quantisation Converts Continuous Values Into Symbols
A temperature can be mapped into bins such as very_low, low, normal, high and very_high.
The sequence becomes symbolic and easier to model discretely, but fine numerical differences disappear.
14. Equal-Width Bins Are Simple but Distribution-Blind
Dividing a numeric range into equal intervals gives intuitive thresholds but can leave some bins almost empty and others crowded.
Representation efficiency depends on the source distribution.
15. Quantile Bins Balance Frequency
Quantile binning creates categories containing roughly similar numbers of observations.
The bins become statistically balanced while losing fixed physical width.
16. Domain Thresholds Can Be More Meaningful Than Statistical Bins
Clinical, engineering or educational thresholds can encode known operational significance.
A domain-aware token boundary can be more useful than a purely statistical one when the receiver acts on those categories.
17. Symbolic Aggregate Approximation Turns Shapes Into Strings
Classical symbolic time-series methods can normalise a series, aggregate local windows and map values into a finite alphabet.
This converts continuous numerical trajectories into symbolic sequences suitable for indexing and pattern matching.
18. Symbolic Strings Enable Motif Search
Repeated subsequences can represent recurring operational motifs such as daily demand cycles or machine vibration patterns.
Tokenisation makes those motifs searchable.
19. Motifs Are Higher-Level Tokens
A motif can compress many lower-level windows into one recognisable pattern class.
Granularity moves from samples to recurring shapes.
20. Trend Is a Coarse Representation
Rising, falling or stable can summarise hundreds of samples.
Such labels are useful for explanation but discard local detail.
21. Seasonality Is Structure Across Repeated Time Positions
Daily, weekly and yearly cycles create predictable relationships between distant points.
Models need calendar or periodic-position information to represent those patterns efficiently.
22. Calendar Features Add External Time Structure
Hour of day, weekday, holiday and school term can explain recurring variation not visible from the raw numeric sequence alone.
Context enriches the token stream.
23. Normalisation Changes What Variation Looks Large
Standardising values can make series with different scales comparable.
But if absolute magnitude matters, normalization parameters must remain available so outputs can return to physical units.
24. Per-Series Normalisation Can Hide Population Differences
Normalising each sensor independently emphasises shape while removing baseline differences.
That can help forecasting and hurt tasks where baseline level is the signal.
25. Multivariate Series Add Feature Dimensions
A machine may record temperature, pressure, vibration and current simultaneously.
The tokenisation question becomes both temporal and cross-variable: should one time step contain all channels, or should channels be tokenised separately?
26. Channel Tokens Can Preserve Variable Identity
Variable IDs or channel embeddings tell the model whether a value came from temperature, pressure or another sensor.
Source type is part of the representation.
27. Cross-Channel Relationships Can Be More Important Than Individual Values
A machine fault may be indicated by pressure rising while flow falls.
Contextual modelling must connect several variables at the same or nearby times.
28. Events and Time Series Interact
A maintenance event can explain a sudden shift in vibration. A medication event can explain a physiological change.
Continuous series and discrete event tokens are complementary representations.
29. Event Alignment Needs Exact Time
If event timestamps and sensor timestamps use different clocks or time zones, causal analysis can fail.
Temporal interoperability is a representation contract.
30. Anomalies Are Relative to an Expected Representation
A value is anomalous only relative to a baseline, season, peer group or learned pattern.
Anomaly tokens therefore depend on context, not absolute magnitude alone.
31. Point Anomalies Are One Granularity
One sample can be unusual.
But a perfectly ordinary value can be anomalous when it appears in the wrong sequence.
32. Collective Anomalies Are Sequence Tokens
A run of values may form an abnormal pattern even when no individual value is extreme.
Higher-level windows reveal structure invisible at point granularity.
33. Forecasting Requires a Prediction Horizon
Predicting one minute ahead and one year ahead are different representation jobs.
Input window length should reflect the history relevant to the chosen horizon.
34. Longer Histories Can Add Noise
Feeding years of irrelevant history can waste context and dilute recent regime changes.
More temporal context is not automatically better.
35. Regime Changes Break Historical Assumptions
A market, machine or population can enter a new operating regime.
Tokens learned from old patterns may become misleading if the world has changed.
36. Time-Series Models Need Freshness Awareness
The same historical pattern can have different predictive value before and after a policy, equipment or behaviour change.
Temporal provenance matters.
37. Sensor Calibration Is Upstream Representation Quality
A drifting sensor can produce a smooth, model-friendly series that is physically wrong.
Model performance cannot compensate reliably for uncalibrated measurement.
38. Sampling Jitter Can Introduce False Dynamics
Irregular observation timing can make a steady process appear noisy when treated as uniformly sampled.
Time metadata must survive preprocessing.
39. Time-Series Compression Needs Source Return
If a symbolic motif flags a problem, analysts should be able to return to the raw measurements that produced it.
Compression should not orphan the evidence.
40. Education Has Time Series Too
Response time, score progression, attendance and error frequency form sequences across lessons.
One assessment score is a snapshot; longitudinal representation reveals trajectory.
41. Improvement Is a Temporal Pattern, Not One Value
A student moving from 40 to 60 to 75 reveals a different state from a student scoring 75 once with no history.
Time gives meaning to measurement change.
42. The Time-Series Tokenisation Audit
- What physical or behavioural process is measured?
- What sampling rate preserves meaningful change?
- Are timestamps regular or irregular?
- How is missingness represented?
- What interpolation assumptions are made?
- What window or patch size is used?
- Are windows overlapping?
- Is continuous value quantised into bins or symbols?
- What normalization preserves or removes absolute scale?
- How are multiple variables identified?
- Are event tokens aligned to the same time basis?
- What seasonal or calendar structure is added?
- How are anomalies defined?
- What forecast horizon is required?
- Can derived tokens return to raw observations?
43. What Students Should Remember
- Sampling is the first time-series discretisation.
- Windows and patches create higher-level temporal tokens.
- Quantisation can turn continuous values into symbols.
- Missingness and time gaps are data.
- Seasonality and calendar context matter.
- Events complement continuous measurements.
- Anomalies depend on expected context.
- Compression should preserve a route to raw evidence.
44. The Deep Principle
A time series is a river of change made finite by clocks, samples and windows. Tokenisation decides how much of that flow one model unit should represent.
Temporal tokens are useful when they compress change without erasing the timing, scale and sequence relationships that make the change meaningful.