affinefoundation/memorygym repository overview

⁠MemoryGym

MemoryGym is an affinetes/OpenEnv-compatible benchmark environment for evaluating real agent memory behavior. The official image runs an affent-based memory agent in --memory-only mode and measures whether a model can build, update, and use persistent memory across document, correction, and question events.

MemoryGym controls evaluation fairness by replaying memory mutations against the benchmark write budget and preserving only accepted memory state for later turns. Non-memory tools are disabled, and user-profile memory is cleared so models cannot bypass the workspace memory budget.

⁠Run with af eval

af is the CLI provided by the affine-cortex project:

https://github.com/affine-cortex/affine-cortex

Install affine-cortex, then run MemoryGym with the official Docker image:

pip install affine-cortex
af eval --env memory --model <MODEL>

Example:

af eval --env memory --model qwen3.6-27b

The environment uses the official image:

affinefoundation/memorygym:latest

Pass the model provider credentials required by your af setup. For the default DashScope judge path, set:

export DASHSCOPE_API_KEY=...

⁠What is evaluated

MemoryGym generates synthetic but grounded memory streams:

• document events containing more facts than the write budget can store
• correction events requiring memory updates
• question events requiring answers from persistent memory

Outputs include score, error status, stored entity counts, write usage, per-axis metrics, answer details, and the flattened interaction trace.