Research question
How can a model keep track of an environment as actions change it? A platform game makes this question concrete: movement changes the character’s relationship to obstacles, surfaces, and the surrounding scene.
Our investigation connects representation learning with the wider goal of persistent interactive worlds.
Starting point
We implemented JEPA on a trained Super Mario model. The linked maxmill/aq-mario repository is the base-model reference for this work.
The experiment focuses on the relationship between the current world, the action taken, and the representation of what comes next.
Areas of investigation
- Action conditioning. How user input influences the next state.
- Temporal consistency. How the environment holds together across successive predictions.
- Spatial memory. What a representation retains about the surrounding world.
- Latent representations. Which information supports prediction and ongoing interaction.
Why this matters
An interactive experience needs continuity across actions. For our broader platform, this means studying how models can carry useful state forward while the user keeps exploring.
Mario provides a setting for investigating these questions before extending the same research direction to more open-ended environments.
Research status
This page describes the investigation and its starting point. Quantitative results and a separate implementation release are not included here; the resource below points to the base model.