Research question
Can a compact model preserve the controllable character as it predicts an environment? A visually plausible scene is not enough if the agent disappears.
Approach
The system combines a tokenizer with a dynamics model. Spatially structured latents preserve the character’s location; classification over a 21-color palette keeps reconstruction sharp.
What the experiment found
Whole-frame metrics can improve while character reconstruction deteriorates. The agent occupies just 1.3% of the observation, making agent-specific evaluation essential.
Reported results
- Total parameters
- 4.4M
- Tokenizer agent accuracy
- 0.9994
- Rollout agent accuracy
- 0.885
- M2 GPU throughput
- 101 fps
Reported by the repository. Throughput depends on hardware; these figures describe this pixel-art environment, not general-world performance.
Play & reproduce
The repository includes checkpoints and a local play script. Follow its setup instructions, then run:
python src/algo/play_local.py --tok artifacts/tokenizer.pt --dyn artifacts/dynamics.pt