harmstack flags.
Global and shared flags
These flags apply to theharmstack root command and harmstack init.
Authentication and endpoint targeting
string
Your Harmstack API key for account and job APIs.Env fallback:
HARMSTACK_API_KEYExample:string
URL of the model API endpoint to benchmark.Example:
string
Bearer token for your target model endpoint. Required when using
--consentandskip.Env fallback: TARGET_MODEL_API_KEYExample:string
default:"openai"
API shape for your target endpoint.Accepted values:
openaiopenai_responsesgeminiraw
string
Model name used in requests. Ignored when
--provider=raw.Example:Benchmark selection and run behavior
integer[]
Benchmark IDs to run. Repeat the flag or pass a comma-separated list.Examples:
integer[]
Number of human-annotated unit tests per benchmark job (1 to 10). Must align with
--benchmark-id order and length.Example:boolean
default:"false"
Skip interactive prompts and run non-interactively. Recommended for CI and scripting.Example:
string[]
Optional HTTP headers added to every request to your model endpoint. Repeat as needed.Example:
boolean
default:"false"
Run the Haystack benchmarking flow directly from the root
harmstack command.Example:harmstack compare-jobs flags
string
UUID of the first job.Example:
string
UUID of the second job.Example:
harmstack list-jobs flags
string
default:"table"
Output format.Accepted values:
table, csvExample:integer
default:"10"
Maximum jobs to return.Example:
string
default:"completed"
Status filter.Accepted values:
completed, failed, allExample:harmstack show-job flags
string
UUID of the job to inspect.Examples:
harmstack stats flags
integer
default:"30"
Number of recent completed jobs to include in aggregate calculations.Example:
string
Date filter in
YYYY-MM-DD format.Example: