Skip to content
Arena Docs

CLI Reference

The rogue-arena-inspect command comes with the inspect-rogue-arena package. It reads ROGUE_ARENA_BASE_URL and ROGUE_ARENA_TOKEN from the environment. --base-url and --token override them, but keep the token in the environment, since a flag shows up in the process list.

whoami

Section titled “whoami”
Terminal window
rogue-arena-inspect whoami

Prints, as JSON, the user and organization the token belongs to, its label, and its last 4 characters.

evals

Section titled “evals”
Terminal window
rogue-arena-inspect evals
rogue-arena-inspect evals --json

Lists the evals the token can run: each eval’s name, version, number of hops, scoring mode and ID. It also shows whether a trial can start now:

  • ready now (N warm): warm copies are waiting.
  • on demand, ~X min: a trial builds a fresh copy first.
  • unknown: readiness wasn’t reported. The command still works.

--json prints the raw list.

launch

Section titled “launch”
Terminal window
rogue-arena-inspect launch --eval <id> --model <provider>/<model>                   # the built-in example task
rogue-arena-inspect launch --eval <id> --task my_task.py --model <provider>/<model> # your own task
rogue-arena-inspect launch --team team.yaml --eval <id>                             # a multi-agent team
rogue-arena-inspect launch --eval <id> --model <provider>/<model> --max-runtime 1800 # time limit per trial

launch runs inspect eval for you. It finds the example task that ships with the package, sets the eval, and keeps your token out of the command line.

  • --eval takes an eval ID, or a name that matches exactly one eval.
  • --task runs your own task file instead of the example. See Custom tasks and scoring.
  • --team runs a team file. See Multi-agent teams.
  • --max-runtime sets each trial’s time limit in seconds, from 60 to 86400. The default is one hour.
  • Anything after -- goes straight to inspect eval, for example -- --limit 1.

runs

Section titled “runs”
Terminal window
rogue-arena-inspect runs
rogue-arena-inspect runs --json

Lists the token’s recent runs.

run

Section titled “run”
Terminal window
rogue-arena-inspect run <id>           # status, score and result per hop
rogue-arena-inspect run <id> --watch   # follow the run live until it finishes
rogue-arena-inspect run <id> --chat    # a team run's chat transcript

Takes a run ID (or a trial ID). For each trial it prints the score, each hop as pass or fail, and why the trial ended, such as reason=timeout (ran out of wall clock).

  • --watch prints new commands, hops and status changes as they happen, then the final result. --interval sets how often it checks, in seconds (default 3).
  • --chat prints a team run’s chat, one line per message.
  • --json prints the raw objects.

Long commands

Section titled “Long commands”

Commands up to 50 seconds run in one request. A command with a longer timeout runs in the background, and the package checks on it until it finishes. A background command can run for up to 4 hours. Rogue Arena stops it, along with everything it started, at its timeout or when the trial ends.

Rogue Arena returns at most the last 10 MiB of a command’s output, and an upload can be at most 10 MiB.