# Router Gauntlet (benchmarks.arnao.ai, Oct 6 2026)

Routers vs fixed models on freshly generated, script-graded tasks.

- `gen_tasks.py [seed]`: builds `tasks.json` (prompts + gold answers / hidden tests). Default seed 20261006. Use a NEW seed for every future run, because these tasks are now public.
- `grade.py`: deterministic graders. Code answers run in `python:3-alpine` with no network, a read-only filesystem, all capabilities dropped, user nobody, and a 20 s limit.
- `run.py [cap$]`: every contestant x task x reps through OpenRouter (`OPENROUTER_API_KEY` env), with a hard spend cap. Resumable via `results.jsonl`.
- `summarize.py` -> `summary.json`; `render.py` -> the section in `../index.html` (uses `findings.json`, hand-written).

Run 1: 256 calls, $3.95 billed (matches the OpenRouter key usage delta).