Reality unfolds continuously. Event systems make that continuous world computable by deciding when something has happened, what changed, who or what was involved, and where one state ends and another begins.
Event tokenisation is the representation move that converts ongoing activity into discrete, addressable events. A door opens. A payment settles. A student submits an answer. A machine exceeds a temperature threshold. A train enters a station. A user clicks a link. Each can become one event token inside a larger sequence.
This article continues the eduKateSingapore Representation and Tokenisation series.
The Event Representation Route
CONTINUOUS WORLD → OBSERVATION → CHANGE DETECTION → EVENT BOUNDARY → EVENT TYPE → ACTOR / OBJECT / TIME / PLACE → STATE TRANSITION → EVENT LOG → SEQUENCE / GRAPH / MODEL → DECISION → WORLD RETURN
1. An Event Is a Chosen Boundary
The world does not naturally arrive with database rows. An event system creates a boundary around activity and says: this occurrence is one thing worth recording.
That boundary can be physical, administrative, technical or analytical.
2. Events Usually Represent Change
Many useful events describe a change from one state to another: unpaid → paid, closed → open, absent → present, queued → running.
The event is meaningful because it marks a transition.
3. States and Events Are Different Representations
A state describes what is true at a moment. An event describes what happened or changed. “The door is open” is a state. “The door opened at 09:02” is an event.
Systems often need both.
4. Event Boundaries Can Be Explicit
A payment processor receives a settlement message. A database receives an insert. A button click generates a browser event. The source system already declares that something happened.
Explicit events are easier to tokenise because the producer supplies the boundary.
5. Event Boundaries Can Also Be Inferred
A sensor stream has no built-in declaration saying “overheating began here.” The event must be inferred from thresholds, change points, patterns or learned detectors.
Inferred events carry model uncertainty.
6. Thresholds Create Artificial Event Lines
If a machine temperature above 80°C triggers an alert, 79.9°C and 80.1°C fall on opposite sides of an event boundary even though the physical states are almost identical.
Administrative thresholds turn continuous values into discrete categories.
7. Hysteresis Can Prevent Boundary Chatter
When a value oscillates around one threshold, a system may emit repeated on/off events. Hysteresis uses different entry and exit thresholds to stabilise the representation.
Boundary design affects event quality.
8. Time Is Part of Event Identity
An event without a timestamp is often incomplete. Time distinguishes repeated occurrences and enables ordering, duration and causal analysis.
Timestamps should preserve timezone or a canonical time basis when systems cross locations.
9. Event Time and Processing Time Are Different
An event can occur at 09:00 and be received by a server at 09:03. Distributed systems therefore distinguish when something happened from when the platform observed or processed it.
Collapsing those times can distort sequence order.
10. Late Events Break Simple Ordering
Networks delay messages. Devices go offline. Logs arrive in batches. A later-received event can have an earlier event-time.
Event streams need explicit ordering semantics.
11. Duplicate Events Are Common
Retries and network delivery can cause the same event to appear more than once.
Stable event IDs and idempotent processing protect downstream state from duplicate representation.
12. Event IDs Are Not Event Types
payment_settled can be an event type. evt_938274 can identify one occurrence of that type.
Type and instance identity are separate dimensions.
13. An Event Type Is a Category Token
Types compress many occurrences into one reusable class: login, purchase, error, door_open, lesson_started.
The type vocabulary therefore functions like a semantic tokenizer over activity.
14. Event Schemas Define What Survives
An event can carry actor, object, amount, timestamp, device, location, source and metadata.
If a field is omitted, downstream systems may never recover it.
15. Event Payloads Need Typed Fields
An amount should not be stored only as a free-text string if numerical operations matter. A timestamp should not be untyped prose if chronological order matters.
Structured event representation preserves semantics beyond surface tokens.
16. Actors and Objects Should Be Canonicalised
The same student, customer or machine may appear under several labels. Canonical IDs let different events point back to one entity.
This connects event tokenisation to Entity Tokenisation.
17. Event Logs Create Sequences
Once events are discrete and timestamped, they can be ordered into a sequence: login → browse → add_to_cart → checkout → payment.
The sequence becomes a model-ready representation of behaviour.
18. Sequence Order Can Reveal Process
The same set of events in a different order can imply a different workflow.
Order is therefore part of event meaning.
19. Duration Requires Paired Boundaries
Some events are better represented as intervals: lesson started at 15:00 and ended at 16:30.
Start-only events cannot represent duration without another boundary.
20. Start and End Events Can Become One Interval Token
A higher-level representation can combine matching start/end events into one session object with duration.
Granularity changes from individual transitions to one bounded activity.
21. Sessions Are Event Chunks
Web analytics often groups many clicks into one session. Manufacturing systems group machine states into runs. Education systems group interactions into lessons.
Sessionisation is event chunking.
22. Session Boundaries Are Heuristics
A 30-minute inactivity rule may define one web session, but human intent does not necessarily reset exactly at that boundary.
Operational segmentation should not be mistaken for psychological truth.
23. Event Streams Can Represent Workflows
Business processes such as order fulfilment, admissions, support cases and logistics can be reconstructed from event sequences.
Process mining treats event logs as evidence of how work actually flows.
24. Missing Events Distort Process Reconstruction
If one system never emits a cancellation event, downstream analysis can falsely infer that an order simply disappeared.
Completeness is part of event fidelity.
25. Event Taxonomies Need Stable Meaning
If lesson_complete changes from “teacher ended the lesson” to “student passed all tasks” without versioning, historical sequences become incomparable.
Event semantics must be versioned.
26. Schema Evolution Is a Representation Migration
Adding, renaming or changing event fields can break consumers and analytics.
Events should carry schema versions or compatible contracts.
27. Event Sourcing Treats Events as Primary Record
In event-sourced systems, current state is derived by replaying historical events rather than stored as the only authoritative record.
The event sequence becomes organisational memory.
28. Replaying Events Reconstructs State
If the event log is complete and deterministic rules are stable, the system can rebuild current state from the sequence.
This is detokenisation at an operational level: discrete history returns to state.
29. Snapshots Compress Long Histories
Replaying millions of events can be expensive. Systems can store periodic state snapshots and replay only later events.
Snapshots trade provenance resolution for computational efficiency unless the full log remains preserved.
30. Causality Is Not the Same as Sequence
Event A occurring before event B does not prove A caused B.
Chronology provides evidence, not causal identity.
31. Distributed Events Can Be Concurrent
Two systems can emit events without one globally authoritative clock. Their relationship may be concurrent rather than strictly ordered.
Distributed representations sometimes need partial order rather than one timeline.
32. Correlation IDs Link Related Events
One transaction can generate events across several services. A correlation or trace ID groups them into one distributed operation.
This adds graph structure above the event stream.
33. Event Graphs Can Represent Branching Processes
A workflow can branch, merge and retry. A simple linear sequence may hide those relationships.
Graphs can preserve causal and dependency structure more faithfully.
34. Behaviour Models Treat Events as Tokens
Recommendation, fraud detection and user-modelling systems can feed event type IDs into sequence models much like text tokens.
The vocabulary contains actions instead of subwords.
35. Time Gaps Need Representation Too
login → purchase after 10 seconds may mean something different from login → purchase after three months.
Time intervals can be encoded as features, buckets or separate tokens.
36. Event Frequency Can Dominate Models
Very common low-value events can overwhelm rare high-value events in training data.
Sampling and weighting may be required so representation frequency does not become importance by accident.
37. Rare Events Matter Disproportionately in Risk Systems
Fraud, equipment failure and safety incidents are rare but critical.
A tokenizer or model optimised only around common event frequency can underrepresent the events that matter most.
38. Event Compression Can Hide Detail
Several low-level sensor changes can be compressed into one “machine_fault” event. That improves interpretability while discarding the raw pattern.
Keep a route back to source telemetry when diagnosis requires it.
39. Human Events Are Interpretive
“The meeting began” can mean the scheduled start time, the moment the first person arrived or the moment substantive discussion started.
Social event boundaries often depend on convention and purpose.
40. Education Is Full of Event Tokens
Student opened question, requested hint, submitted answer, corrected mistake and mastered concept can form a learning event sequence.
The sequence can reveal process that a final score alone hides.
41. Final Scores Are State Summaries, Not Event Histories
Two students can receive the same score after very different learning paths.
Event histories preserve temporal process at higher resolution.
42. The Event Tokenisation Audit
- What continuous activity is being observed?
- What rule declares that an event occurred?
- Is the event explicit or inferred?
- What event type vocabulary exists?
- What stable event ID identifies the occurrence?
- What actor and object IDs are attached?
- What timestamp, timezone and processing time are preserved?
- Can duplicate events be detected?
- Can late events be ordered correctly?
- What state transition does the event represent?
- How are sessions or intervals created?
- What schema version defines the payload?
- Can event history reconstruct state?
- Are rare high-impact events preserved?
- Can every derived event return to source evidence?
43. What Students Should Remember
- Events are discrete representations of ongoing activity.
- An event often marks a state transition.
- Event time and processing time are different.
- Stable IDs prevent duplicates and connect related occurrences.
- Event types form a semantic vocabulary.
- Sessions are higher-level event chunks.
- Sequence does not prove causality.
- Event logs can reconstruct process and state.
44. The Deep Principle
An event is a decision that something in a continuous world deserves one discrete address. That address makes history searchable, modelable and replayable—but only if the event boundary preserves the change the receiver actually cares about.
Event tokenisation turns “something happened” into a stable machine statement about what changed, when it changed and what the world became afterward.