The Gemini Plays Pokémon agent reads game RAM and screenshots, turns actions into button presses, and uses pathfinder and boulder-puzzle sub-agents.
The Gemini Plays Pokémon agent reads game RAM and screenshots, turns actions into button presses, and uses pathfinder and boulder-puzzle sub-agents.
An independent developer, Joel Zhang, built a harness that gives Gemini 2.5 Pro a subset of Game Boy RAM as text, overlaid on a screenshot. Gemini picks actions that become button presses. The harness summarizes history every 100 turns and runs a critic instance every 25 turns. It also provides pathfinder and boulder-puzzle sub-agents, which are prompted Gemini instances.
The agent beat Pokémon Blue in 813 hours on run 1, then in 406.5 hours on a fully autonomous second run. It looped on long contexts, hallucinated items from other games, and handled raw pixels poorly.
The model struggled to read raw screen pixels, so the RAM-to-text translation was needed. Behavior also degraded once context grew well past 100k tokens.
Help me build an agent that plays [game, e.g. Pokémon Blue] in an emulator at [emulator path or setup]. It should read the game state from [RAM or screenshots], decide on an action, and send it as button presses, with a simple helper for [subtask, e.g. pathfinding]. You're done when it can play for [number] steps on its own and log what it decided at each step.
Claude Code got an old DirectX game running on modern hardware, decompiled much of it, built a headless parallel runner, and now writes full level guides with screenshots.
Make pulls contracts from Gmail, Gemini extracts the term and notice period, and a Sheets register emails the owner 60, 30 and 7 days before each deadline.
A self-hosted n8n job reads your calendar, unread email and Notion tasks at 7am and sends one short brief of what matters today.