Sign inSign up

affinefoundation/agentgym

By affinefoundation

Updated 10 months ago

The AgentGym environment provides interactive agent evaluation across multiple benchmark tasks.

Image
Machine learning & AI
0

2.5K

affinefoundation/agentgym repository overview

AgentGym Environment

Task Description

The AgentGym environment provides interactive agent evaluation across multiple benchmark tasks. Each task evaluates an agent's ability to interact with different environments and complete specific objectives through multi-turn conversations.

Supported Tasks
  • webshop: E-commerce shopping tasks
  • alfworld: Household task completion in text-based environments
  • babyai: Grid-world navigation and instruction following
  • sciworld: Scientific reasoning and experimentation
  • textcraft: Text-based crafting game

How to Use with Affinetes

Option 1: Pull Pre-built Docker Image
import affinetes as af
import asyncio
import os

async def main():
    # Load environment from Docker Hub (example: sciworld)
    env = af.load_env(
        image="bignickeye/agentgym:sciworld-v2",
        env_vars={"CHUTES_API_KEY": os.getenv("CHUTES_API_KEY")}
    )
    
    # Evaluate on a specific task
    result = await env.evaluate(
        model="deepseek-ai/DeepSeek-V3",
        base_url="https://llm.chutes.ai/v1",
        task_id=10,
        max_round=30
    )
    
    print(f"Score: {result['score']}")
    print(f"Success: {result['success']}")
    
    await env.cleanup()

asyncio.run(main())

Tag summary

Content type

Image

Digest

sha256:c531e612e

Size

466.8 MB

Last updated

10 months ago

docker pull affinefoundation/agentgym:textcraft