Self-Improvement
Task completion gain (percentage points) over the Gemma 4 31B base agent, per game. Everything above the line helped; everything below it hurt. Hover a marker for the underlying rates, and click a method in the legend to show or hide it.
No approach consistently improves performance across all game environments.
Qualitatively, we observe that the world model is unable to predict the next observation with sufficient accuracy to serve as a useful tool.
The skill discovery approach performs even worse, with catastrophic collapses on PokémonCrystal (36.0% → 4.0%), PokémonRed (38.8% → 8.2%) and ZeldaOracleOfSeasons (40.0% → 14.0%). The performance degradation is always greater for test-only games, suggesting an overfitting to the training distribution. We hypothesize the extent of degradation is caused by an inability to accurately judge task completion during practice, a known failure scenario for skill acquisition methods.
The only method that achieves partial success is our novel utilization of curiosity-based exploration to write game-specific documentation. Here, task completion increases by as much as 22% on select games like BombermanPocket (72.0% → 88.0%), and the boost applies to both games seen during training time and unseen test games like SwordOfHope2 (66.0% → 72.0%). However, even this approach leads to considerable performance degradation on some games (DejaVu1, −29%; DejaVu2, −25%; PokémonCrystal, −17%).
The failure of these standard implementations establishes GameBoyWorlds-Execution as a challenging benchmark and calls for further research on acquiring short-horizon task execution capabilities via self-improvement.