Sign inSign up

affinefoundation/cde

By affinefoundation

Updated 8 months ago

he Code (CDE) environment evaluates language models on code generation tasks

Image
Machine learning & AI
0

2.0K

affinefoundation/cde repository overview

Code Environment (CDE)

Task Description

The Code (CDE) environment evaluates language models on code generation tasks from the Intellect-3-RL dataset. Models are presented with programming problems and must generate correct Python code that passes all test cases.

How to Use with Affinetes

Option 1: Pull Pre-built Docker Image
import affinetes as af
import asyncio
import os

async def main():
    # Load environment from Docker Hub
    env = af.load_env(
        image="affinefoundation/cde:pi",
        env_vars={"CHUTES_API_KEY": os.getenv("CHUTES_API_KEY")}
    )
    
    # Evaluate on a code generation task
    result = await env.evaluate(
        model="deepseek-ai/DeepSeek-V3",
        base_url="https://llm.chutes.ai/v1",
        task_id=42,  # Optional: index into dataset
     
    )
    
    print(f"Score: {result['score']}")
    print(f"Success: {result['success']}")
    
    await env.cleanup()

asyncio.run(main())
Option 2: Build from Source
import affinetes as af
import asyncio
import os

async def main():
    # Build image from local environment directory
    image_tag = af.build_image_from_env(
        env_path="environments/primeintellect/cde",
        image_tag="cde:pi"
    )
    
    # Load environment
    env = af.load_env(
        image=image_tag,
        env_vars={"CHUTES_API_KEY": os.getenv("CHUTES_API_KEY")}
    )
    
    # Evaluate on a code generation task
    result = await env.evaluate(
        model="deepseek-ai/DeepSeek-V3",
        base_url="https://llm.chutes.ai/v1",
        task_id=42
    )
    
    print(f"Score: {result['score']}")
    
    await env.cleanup()

asyncio.run(main())

Tag summary

Content type

Image

Digest

sha256:2cc008cbd

Size

1.9 GB

Last updated

8 months ago

docker pull affinefoundation/cde:print