affinefoundation/liveweb-arena repository overview

⁠LiveWeb Arena

Real-time web evaluation framework for LLM browser agents.

⁠Quick Start

# Install
pip install -e .
playwright install chromium

# Configure
cp .env.example .env
# Edit .env with your API_KEY

# Run evaluation
python eval.py --seed 42 --verbose

⁠Usage

# Basic evaluation
python eval.py --seed 42

# Specific template
python eval.py --templates weather/current_weather --seed 42

# Multi-task evaluation
python eval.py --num-tasks 3 --seed 42

# Deterministic task by ID
python eval.py --task-id 100001 --seed 42

# View all templates
python eval.py --show-registry

⁠Options

⁠Templates

⁠Weather (wttr.in) - 6 templates

location_name, time_of_day, multi_day, current_weather, astronomy, weather_comparison

⁠Stooq (stooq.com) - 7 templates

stooq_price, stooq_comparison, stooq_ranking, stooq_sector_analysis, stooq_currency, stooq_volatility, stooq_range_position

⁠CoinGecko (coingecko.com) - 8 templates

coingecko_price, coingecko_volume, coingecko_comparison, coingecko_rank, coingecko_top_movers, coingecko_supply, coingecko_ath, coingecko_performance

⁠Taostats (taostats.io) - 10 templates

taostats_subnet_info, taostats_comparison, taostats_analysis, taostats_ranking, taostats_price_change, taostats_threshold, taostats_multi_condition, taostats_delta, taostats_range_count, taostats_percentage

⁠Hybrid (cross-source) - 3 templates

hybrid_top_performer, hybrid_ranking, hybrid_conditional_branch

⁠Environment Variables

⁠Output

Results saved to eval/<timestamp>.json:

{
  "score": 1.0,
  "success": true,
  "extra": {
    "seed": 42,
    "answer_details": [...],
    "conversation": [...]
  }
}

⁠License

MIT