Use this endpoint to retrieve all benchmarks available to your account. Each benchmark defines a specific evaluation suite you can run against your model endpoint by submitting a job.
Endpoint GET https://api.harmstack.com/v0/benchmarks
Authentication Request No query parameters are required.
curl https://api.harmstack.com/v0/benchmarks \ --header "Authorization: Bearer YOUR_API_KEY"
Response Returns 200 OK with a JSON object containing a benchmarks array.
Array of benchmark objects available to your account. Unique numeric identifier for the benchmark. Use this value as benchmark_id when submitting jobs.
Internal identifier for the benchmark publisher.
Internal name of the benchmark (e.g. "haystack-v1").
Human-readable name of the benchmark (e.g. "Haystack Medical QA v1").
Version number of the benchmark.
ID of the associated benchmark property configuration, if set.
Credit cost per benchmark unit submitted.
Currency code for the price (e.g. "USD").
ISO 8601 timestamp of when the benchmark was created.
ISO 8601 timestamp of the last update.
{ "benchmarks" : [ { "id" : 1 , "merchant_id" : 3 , "name" : "haystack-medical-v1" , "display_name" : "Haystack Medical QA v1" , "version" : 1 , "benchmark_property_id" : 12 , "price_per_benchmark_unit" : 1.0 , "currency" : "USD" , "created_at" : "2024-01-15T10:00:00Z" , "updated_at" : "2024-06-01T08:30:00Z" } ] }
Copy the id from a benchmark object and pass it as benchmark_id when
submitting a job via POST /v0/jobs.
