CLI for executing SQL queries from files against ClickHouse clusters
935
A CLI tool that executes SQL queries from a file against a ClickHouse cluster, running queries in parallel with round-robin distribution across nodes, checkpointing progress in Valkey, and providing a rich terminal UI showing both batch-level and query-level progress.
Run directly without installing using uv:
uvx clickhouse-query-runner queries.sql
Or install as a persistent tool:
uv tool install clickhouse-query-runner
clickhouse-query-runner queries.sql
Build the image:
docker build -t clickhouse-query-runner .
Run with environment variables and a local SQL file:
docker run --rm \
-e CLICKHOUSE_HOST=clickhouse.example.com \
-e CLICKHOUSE_USER=default \
-e CLICKHOUSE_PASSWORD=secret \
-e CLICKHOUSE_DATABASE=mydb \
-v $(pwd)/queries.sql:/app/queries.sql \
clickhouse-query-runner /app/queries.sql
Pass additional options after the image name:
docker run --rm \
-e CLICKHOUSE_HOST=node1.example.com,node2.example.com \
-e CLICKHOUSE_USER=default \
-e CLICKHOUSE_PASSWORD=secret \
-e CLICKHOUSE_DATABASE=mydb \
-e VALKEY_URL=redis://valkey:6379/0 \
-v $(pwd)/queries.sql:/app/queries.sql \
clickhouse-query-runner --concurrency 4 /app/queries.sql
To use with docker compose, add the service to your compose.yaml:
services:
query-runner:
build: .
environment:
CLICKHOUSE_HOST: clickhouse
CLICKHOUSE_USER: default
CLICKHOUSE_PASSWORD: secret
CLICKHOUSE_DATABASE: mydb
VALKEY_URL: redis://valkey:6379/0
volumes:
- ./queries.sql:/app/queries.sql
command: ["/app/queries.sql"]
depends_on:
- clickhouse
- valkey
git clone https://github.com/gmr/clickhouse-query-runner.git
cd clickhouse-query-runner
uv sync --group dev
To run the tool during development:
uv run clickhouse-query-runner queries.sql
# Set connection environment variables
export CLICKHOUSE_HOST=clickhouse.example.com
export CLICKHOUSE_USER=default
export CLICKHOUSE_PASSWORD=secret
export CLICKHOUSE_DATABASE=mydb
# Run queries from a file
uvx clickhouse-query-runner queries.sql
# With explicit options
uvx clickhouse-query-runner \
--host node1.example.com,node2.example.com \
--concurrency 4 \
--valkey-url redis://valkey:6379/0 \
queries.sql
| Option | Env Var | Default | Description |
|---|---|---|---|
--host | CLICKHOUSE_HOST | (required) | ClickHouse hostname(s), comma-separated |
--port | CLICKHOUSE_PORT | 9440 | ClickHouse server port |
--database | CLICKHOUSE_DATABASE | (required) | Database name |
--user | CLICKHOUSE_USER | (required) | Username |
--password | CLICKHOUSE_PASSWORD | (required) | Password |
--secure | CLICKHOUSE_SECURE | true | Use secure connection |
--concurrency | 2 | Max parallel queries | |
--run-id | (auto) | Override run ID | |
--valkey-url | VALKEY_URL | redis://localhost:6379/0 | Valkey URL |
--checkpoint-ttl | 604800 | Checkpoint TTL in seconds | |
--poll-interval | 0.5 | Progress poll interval | |
--cancel-on-failure | false | Cancel in-flight on failure | |
--dry-run | false | Parse without executing | |
--reset | false | Clear checkpoints and exit | |
--verbose | false | Debug logging |
system.processes for per-query progresssrc/clickhouse_query_runner/
├── __init__.py # Package initialization
├── cli.py # Entry point, arg parsing
├── runner.py # Core async execution engine
├── checkpoint.py # Valkey checkpoint management
├── parser.py # SQL file parsing
├── progress.py # Rich progress display
└── settings.py # Pydantic settings model
uv run ruff check src/
uv run ruff format --check src/
BSD 3-Clause License. See LICENSE for details.
Content type
Image
Digest
sha256:9d15ca7e9…
Size
71.4 MB
Last updated
7 months ago
docker pull gavinmroy/clickhouse-query-runner