The AgentGym environment provides interactive agent evaluation across multiple benchmark tasks.
2.5K
The AgentGym environment provides interactive agent evaluation across multiple benchmark tasks. Each task evaluates an agent's ability to interact with different environments and complete specific objectives through multi-turn conversations.
import affinetes as af
import asyncio
import os
async def main():
# Load environment from Docker Hub (example: sciworld)
env = af.load_env(
image="bignickeye/agentgym:sciworld-v2",
env_vars={"CHUTES_API_KEY": os.getenv("CHUTES_API_KEY")}
)
# Evaluate on a specific task
result = await env.evaluate(
model="deepseek-ai/DeepSeek-V3",
base_url="https://llm.chutes.ai/v1",
task_id=10,
max_round=30
)
print(f"Score: {result['score']}")
print(f"Success: {result['success']}")
await env.cleanup()
asyncio.run(main())
Content type
Image
Digest
sha256:c531e612e…
Size
466.8 MB
Last updated
10 months ago
docker pull affinefoundation/agentgym:textcraft