A model that acts in a changing environment needs more than a snapshot. It needs a representation of what state the world is in now, how that state came to be, and what is likely to happen after the next action.
World-state representation is the process of turning partial observations and action history into a latent state that supports prediction, planning and control.
This article completes the current eduKateSingapore Representation and Tokenisation mechanism batch by connecting representation to predictive dynamics.
The World-State Route
OBSERVATION_t + ACTION HISTORY → ENCODER / BELIEF UPDATE → LATENT STATE_t → DYNAMICS MODEL + ACTION_t → PREDICTED LATENT STATE_t+1 → REWARD / OUTCOME / OBSERVATION PREDICTION → PLANNING OR POLICY → ACTION → NEW WORLD OBSERVATION → STATE UPDATE
1. Observation Is Not State
A camera frame shows what is visible from one viewpoint at one moment. It does not reveal hidden objects, future motion, internal machine variables or everything that happened previously.
World state is a richer representation than current observation.
2. State Summarises What Matters for Future Prediction
In control and reinforcement learning, a useful state representation contains enough information from history to predict future outcomes under candidate actions.
The representation is judged by predictive sufficiency rather than visual completeness.
3. Partial Observability Creates Belief State
When the agent cannot observe the full environment directly, it must infer hidden state from current observations and past experience.
The result is often called a belief state: a representation of what the world could currently be, given available evidence.
4. Belief Is Not Certainty
Several hidden states can remain plausible under the same observation history.
A strong world-state representation preserves uncertainty rather than collapsing prematurely to one imagined reality.
5. Actions Are Part of State Inference
If an agent remembers that it moved left, opened a door or applied force, those actions help explain the current observation.
State estimation therefore uses both what was seen and what the agent did.
6. Dynamics Models Predict State Change
A dynamics model estimates how the current latent state will evolve under an action.
This gives the agent a predictive simulator inside representation space.
7. Prediction Can Happen in Latent Space
The model does not always need to generate a full next image or sensor reading to plan effectively.
It can predict a compact latent state containing features relevant to reward and control.
8. DreamerV3 Plans Through Learned World Dynamics
DreamerV3 learns a model of the environment and improves behaviour by imagining future trajectories in latent state space across many domains.
The core representation lesson is that useful planning can occur over learned internal state rather than raw pixels alone.
9. Representation Models Encode Observation History
Dreamer-style systems combine previous latent state, previous action and current observation to infer the new state.
This is a recurrent world-state update.
10. Transition Models Predict Without Seeing the Future Observation
Given current state and action, the transition model predicts what state should come next.
The gap between predicted and observation-updated state reveals model error.
11. Prediction Error Is a World-Model Diagnostic
If predicted state repeatedly diverges from actual subsequent observations, the internal dynamics are missing something important.
World-state fidelity should therefore be tested through future prediction, not only reconstruction.
12. Reconstruction Is Helpful and Incomplete
A latent state can reconstruct what the camera sees and still omit hidden causal variables needed for planning.
Visual fidelity does not guarantee control sufficiency.
13. Reward Prediction Adds Task Relevance
World models often predict reward or task outcome from latent state.
This encourages the representation to preserve distinctions that matter for action quality.
14. Task-Relevant State Can Ignore Visually Salient Detail
The colour of a wall may dominate pixels and have no effect on navigation.
A compact state can discard irrelevant appearance while preserving geometry and hazards.
15. This Is Useful Compression and a Source of Blindness
A feature irrelevant to the training reward can later become important under a new task.
Task-shaped state is not universal world truth.
16. World State Is Different From Memory
Memory stores information about past events or evidence. World state represents the system’s current predictive belief about the environment.
The two interact but serve different jobs.
17. State Can Be Built From Memory
An agent may retrieve past observations or maps to refine current belief.
Memory supplies historical evidence; state integrates it into a current predictive representation.
18. World State Is Different From Event Logs
Event logs record what happened. World state answers what is likely true now after those events.
Replay can reconstruct state only when the event representation and transition rules are sufficiently complete.
19. State Is Different From 3D Scene Representation
A 3D map describes geometry. World state can additionally contain velocity, hidden intent, machine mode, inventory, causal variables and uncertainty.
Geometry is one possible component of state, not the whole state.
20. Objects Can Be State Variables
Object-centric world models can represent identity, position, velocity and relations for persistent entities.
This can make state more interpretable than one monolithic latent vector.
21. Object Identity Must Persist Through Time
If two cars exchange latent identities after crossing, predictions about their trajectories become incoherent.
Entity persistence is part of world-state fidelity.
22. Relations Can Be State Variables Too
Inside, touching, holding, following and connected_to can matter as much as object attributes.
Relational state naturally connects world models to graph representation.
23. State Should Separate Observed From Inferred
“Door is open” may be directly visible. “Person is behind the door” may be inferred from earlier observation.
Provenance inside state should distinguish measurement from belief.
24. Uncertainty Belongs Inside Belief State
A partially observed environment can support several plausible locations or object configurations.
Representing one best guess alone can create brittle planning.
25. Multimodal Observations Must Be Fused
Vision, audio, proprioception, text instructions and map data can all inform current state.
World-state representation is therefore often multimodal even when action space is not.
26. Sensor Disagreement Should Increase Uncertainty
If camera and map disagree, the state estimator should not silently average them into false certainty.
Conflict is itself informative state.
27. Planning Uses Counterfactual Future States
The agent can imagine several action sequences and compare predicted outcomes without executing each one physically.
World models turn latent state into a sandbox for counterfactual reasoning.
28. Counterfactual Accuracy Is Harder Than One-Step Prediction
Small state-transition errors compound over imagined steps.
Long-horizon planning therefore tests whether latent dynamics remain coherent beyond immediate next-state prediction.
29. Model Bias Can Be Exploited by the Planner
A planning algorithm may discover action sequences that look good under the learned world model and fail in the real environment.
The planner can optimise prediction error rather than real success if the model has blind spots.
30. World Return Is Mandatory
Imagined trajectories should be checked against actual environment transitions.
The real world is the final validator of a world model.
31. Online Updating Can Repair Model Error
New observations reveal where predictions failed.
World-state models can use those errors to refine representation and dynamics over time.
32. Distribution Shift Changes the World the Model Thinks It Knows
A robot trained in one room, a market model trained in one regime or a game agent trained under one rule set can encounter states outside training experience.
Unfamiliar state should trigger uncertainty rather than automatic confidence.
33. Language Models Can Encode World-Like State Without Being Full Dynamic World Models
Research has found internal representations in language models correlated with space, time and game state.
Such representations are ingredients of world modelling, but static correlation alone does not establish action-conditioned predictive dynamics.
34. Predictive Dynamics Is the Stronger Test
A world-state representation should support the question: if the agent takes action A now, what state should follow?
That is more demanding than recognising the current scene.
35. World-State Models Need Intervention Tests
Change one action while holding prior state fixed and inspect whether the predicted future changes appropriately.
Action sensitivity tests whether the model represents dynamics rather than passive sequence correlation alone.
36. State Representation Audit
- What environment is being represented?
- What observations are directly available?
- What relevant state is hidden?
- How are past actions incorporated?
- What latent variables carry current belief?
- Does the state preserve uncertainty?
- Can objects and relations maintain identity through time?
- How are observed and inferred features distinguished?
- What transition model predicts the next state?
- What reward or outcome signals shape representation?
- How accurate are one-step and long-horizon predictions?
- Can a planner exploit model errors?
- How does the model detect unfamiliar states?
- Can predictions be checked against actual world transitions?
37. What Students Should Remember
- Observation is not the same as world state.
- Belief state represents what is likely true under partial observability.
- Actions help determine how state changes.
- World models predict latent future states for planning.
- DreamerV3 is a prominent world-model system using imagined latent trajectories.
- State, memory, event logs and 3D maps are related but distinct representations.
- World models must be tested against real future observations and interventions.
38. The Deep Principle
A world-state representation is useful when it carries enough of the hidden present to make the future predictable.
The deepest test of a world model is not whether it can describe what it has seen. It is whether, from a compact state and a possible action, it can anticipate what the world will do next—and admit uncertainty when it cannot.