> ## Documentation Index
> Fetch the complete documentation index at: https://docs.harmstack.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Jobs

> Submit, list, and inspect benchmarking jobs.

Jobs are the core resource in Harmstack. A job runs a specific benchmark against your model endpoint and produces a score. You can submit a single job, submit multiple jobs in one batch, list your past jobs, and retrieve a specific job's status and results.

All endpoints require authentication.

<ParamField header="Authorization" type="string" required>
  `Bearer YOUR_API_KEY`
</ParamField>

***

## Submit a job

```
POST https://api.harmstack.com/v0/jobs
```

Submit a single benchmarking job. The job is queued immediately and begins processing asynchronously. Poll `GET /v0/jobs/{id}` to check progress and retrieve results.

### Request body

<ParamField body="benchmark_id" type="number" required>
  The ID of the benchmark to run. Retrieve available benchmark IDs from `GET /v0/benchmarks`.
</ParamField>

<ParamField body="endpoint_url" type="string" required>
  The URL of your model's chat completions endpoint (e.g. `https://your-model.example.com/v1/chat/completions`).
</ParamField>

<ParamField body="api_key" type="string" required>
  The API key used to authenticate requests to your model endpoint.
</ParamField>

<ParamField body="provider" type="string">
  The API shape of your model endpoint. Accepted values: `openai`, `openai_responses`, `gemini`, `raw`. Defaults to `openai`.
</ParamField>

<ParamField body="model" type="string">
  The model name or identifier (e.g. `"gpt-4o"`, `"claude-3-5-sonnet"`). Optional — used for logging.
</ParamField>

<ParamField body="headers" type="object">
  Additional HTTP headers to include with every request to your model endpoint. Provide as key-value pairs.
</ParamField>

<ParamField body="benchmark_count" type="number" default="1">
  Number of benchmark units to run. Must be between `1` and `10`. Each unit costs one credit.
</ParamField>

<ParamField body="seed" type="number">
  Optional random seed for reproducible unit sampling.
</ParamField>

### Example request

```bash theme={null}
curl --request POST \
  --url https://api.harmstack.com/v0/jobs \
  --header "Authorization: Bearer YOUR_API_KEY" \
  --header "Content-Type: application/json" \
  --data '{
    "benchmark_id": 1,
    "endpoint_url": "https://your-model.example.com/v1/chat/completions",
    "api_key": "sk-your-model-api-key",
    "provider": "openai",
    "model": "gpt-4o",
    "benchmark_count": 3
  }'
```

### Response

Returns `202 Accepted`.

<ResponseField name="job_id" type="string" required>
  UUID of the created job. Use this to poll for results.
</ResponseField>

<ResponseField name="status" type="string" required>
  Initial status of the job. Always `"pending"` on creation.
</ResponseField>

<ResponseField name="message" type="string" required>
  A human-readable message with the polling URL.
</ResponseField>

```json theme={null}
{
  "job_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "status": "pending",
  "message": "job submitted; poll GET /jobs/a1b2c3d4-e5f6-7890-abcd-ef1234567890 for progress"
}
```

<Note>
  Submitting a job immediately deducts credits equal to `benchmark_count` from
  your account balance. Ensure you have sufficient credits before submitting —
  check your balance with `GET /v0/me`.
</Note>

***

## Submit a batch of jobs

```
POST https://api.harmstack.com/v0/jobs/batch
```

Submit multiple benchmarking jobs in a single request. All jobs in the batch share the same model endpoint configuration. Credits for all jobs in the batch are deducted atomically — the entire batch is rejected if your balance is insufficient.

### Request body

<ParamField body="endpoint_url" type="string" required>
  The URL of your model's chat completions endpoint.
</ParamField>

<ParamField body="api_key" type="string" required>
  The API key used to authenticate requests to your model endpoint.
</ParamField>

<ParamField body="provider" type="string">
  The API shape of your model endpoint. Accepted values: `openai`, `openai_responses`, `gemini`, `raw`. Defaults to `openai`.
</ParamField>

<ParamField body="model" type="string">
  The model name or identifier.
</ParamField>

<ParamField body="headers" type="object">
  Additional HTTP headers to include with every request to your model endpoint.
</ParamField>

<ParamField body="seed" type="number">
  Optional random seed for reproducible unit sampling across all jobs in the batch.
</ParamField>

<ParamField body="jobs" type="object[]" required>
  Array of job definitions. Each item specifies which benchmark to run and how many units.

  <Expandable title="job item fields">
    <ParamField body="jobs[].benchmark_id" type="number" required>
      The ID of the benchmark to run for this job.
    </ParamField>

    <ParamField body="jobs[].benchmark_count" type="number" required>
      Number of benchmark units for this job. Must be between `1` and `10`.
    </ParamField>
  </Expandable>
</ParamField>

### Example request

```bash theme={null}
curl --request POST \
  --url https://api.harmstack.com/v0/jobs/batch \
  --header "Authorization: Bearer YOUR_API_KEY" \
  --header "Content-Type: application/json" \
  --data '{
    "endpoint_url": "https://your-model.example.com/v1/chat/completions",
    "api_key": "sk-your-model-api-key",
    "provider": "openai",
    "model": "gpt-4o",
    "jobs": [
      { "benchmark_id": 1, "benchmark_count": 5 },
      { "benchmark_id": 2, "benchmark_count": 3 }
    ]
  }'
```

### Response

Returns `202 Accepted`.

<ResponseField name="batch_id" type="string" required>
  UUID identifying the batch.
</ResponseField>

<ResponseField name="job_ids" type="string[]" required>
  Array of UUIDs for each created job. Poll `GET /v0/jobs/{id}` for each to retrieve results.
</ResponseField>

<ResponseField name="status" type="string" required>
  Initial status of all jobs in the batch. Always `"pending"` on creation.
</ResponseField>

<ResponseField name="message" type="string" required>
  A human-readable message describing how to poll for results.
</ResponseField>

```json theme={null}
{
  "batch_id": "f9e8d7c6-b5a4-3210-fedc-ba9876543210",
  "job_ids": [
    "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
    "b2c3d4e5-f6a7-8901-bcde-f12345678901"
  ],
  "status": "pending",
  "message": "job batch submitted; poll GET /jobs/{id} for each job for progress"
}
```

***

## List jobs

```
GET https://api.harmstack.com/v0/jobs
```

Returns a list of jobs for your account. By default returns your 10 most recent completed jobs.

### Query parameters

<ParamField query="status" type="string" default="completed">
  Filter jobs by status. Accepted values: `pending`, `running`, `completed`, `failed`.
</ParamField>

<ParamField query="limit" type="number" default="10">
  Maximum number of jobs to return.
</ParamField>

### Example request

```bash theme={null}
curl "https://api.harmstack.com/v0/jobs?status=completed&limit=5" \
  --header "Authorization: Bearer YOUR_API_KEY"
```

### Response

Returns `200 OK` with a `jobs` array.

<ResponseField name="jobs" type="object[]" required>
  Array of job objects.

  <Expandable title="job fields">
    <ResponseField name="job_id" type="string" required>
      UUID of the job.
    </ResponseField>

    <ResponseField name="status" type="string" required>
      Current job status: `pending`, `running`, `completed`, or `failed`.
    </ResponseField>

    <ResponseField name="module" type="string" required>
      The evaluation module used (e.g. `"Haystack"`).
    </ResponseField>

    <ResponseField name="benchmark_id" type="number" required>
      ID of the benchmark that was run.
    </ResponseField>

    <ResponseField name="benchmark_name" type="string" required>
      Name of the benchmark that was run.
    </ResponseField>

    <ResponseField name="endpoint_url" type="string" required>
      The model endpoint URL that was evaluated.
    </ResponseField>

    <ResponseField name="needle_annnoted_count" type="number" required>
      Number of annotated needle prompts used in the evaluation.
    </ResponseField>

    <ResponseField name="hay_non_annotated_count" type="number" required>
      Number of non-annotated hay prompts used in the evaluation.
    </ResponseField>

    <ResponseField name="total_prompts" type="number" required>
      Total number of prompts sent to your model endpoint.
    </ResponseField>

    <ResponseField name="created_at" type="string" required>
      ISO 8601 timestamp of when the job was created.
    </ResponseField>

    <ResponseField name="completed_at" type="string">
      ISO 8601 timestamp of when the job completed. `null` if still in progress.
    </ResponseField>

    <ResponseField name="passed_count" type="number">
      Number of evaluation units your model passed. Present when job is completed.
    </ResponseField>

    <ResponseField name="failed_count" type="number">
      Number of evaluation units your model failed. Present when job is completed.
    </ResponseField>

    <ResponseField name="score_pct" type="number">
      Your model's score as a percentage (0–100). Present when job is completed.
    </ResponseField>

    <ResponseField name="total_scored" type="number">
      Total number of units that were scored. Present when job is completed.
    </ResponseField>
  </Expandable>
</ResponseField>

```json theme={null}
{
  "jobs": [
    {
      "job_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
      "status": "completed",
      "module": "Haystack",
      "benchmark_id": 1,
      "benchmark_name": "haystack-medical-v1",
      "endpoint_url": "https://your-model.example.com/v1/chat/completions",
      "needle_annnoted_count": 3,
      "hay_non_annotated_count": 30,
      "total_prompts": 33,
      "created_at": "2024-06-01T12:00:00Z",
      "completed_at": "2024-06-01T12:05:42Z",
      "passed_count": 2,
      "failed_count": 1,
      "score_pct": 66.67,
      "total_scored": 3
    }
  ]
}
```

***

## Get a job

```
GET https://api.harmstack.com/v0/jobs/{id}
```

Retrieve the full status and results for a specific job. Poll this endpoint after submitting a job to track progress and retrieve your score.

### Path parameters

<ParamField path="id" type="string" required>
  The UUID of the job, returned from `POST /v0/jobs` or `POST /v0/jobs/batch`.
</ParamField>

### Example request

```bash theme={null}
curl https://api.harmstack.com/v0/jobs/a1b2c3d4-e5f6-7890-abcd-ef1234567890 \
  --header "Authorization: Bearer YOUR_API_KEY"
```

### Response

Returns `200 OK`. Fields are the same as in the list response, with additional real-time progress fields available while the job is running.

<ResponseField name="job_id" type="string" required>
  UUID of the job.
</ResponseField>

<ResponseField name="status" type="string" required>
  Current job status: `pending`, `running`, `completed`, or `failed`.
</ResponseField>

<ResponseField name="module" type="string" required>
  The evaluation module used.
</ResponseField>

<ResponseField name="benchmark_id" type="number" required>
  ID of the benchmark that was run.
</ResponseField>

<ResponseField name="benchmark_name" type="string" required>
  Name of the benchmark that was run.
</ResponseField>

<ResponseField name="endpoint_url" type="string" required>
  The model endpoint URL that was evaluated.
</ResponseField>

<ResponseField name="needle_annnoted_count" type="number" required>
  Number of annotated needle prompts used.
</ResponseField>

<ResponseField name="hay_non_annotated_count" type="number" required>
  Number of non-annotated hay prompts used.
</ResponseField>

<ResponseField name="total_prompts" type="number" required>
  Total number of prompts sent to your model endpoint.
</ResponseField>

<ResponseField name="price" type="number" required>
  Credits deducted for this job.
</ResponseField>

<ResponseField name="created_at" type="string" required>
  ISO 8601 timestamp of when the job was created.
</ResponseField>

<ResponseField name="completed_at" type="string">
  ISO 8601 timestamp of when the job completed. `null` if still in progress.
</ResponseField>

<ResponseField name="passed_count" type="number">
  Number of evaluation units your model passed. Present when job is completed.
</ResponseField>

<ResponseField name="failed_count" type="number">
  Number of evaluation units your model failed. Present when job is completed.
</ResponseField>

<ResponseField name="score_pct" type="number">
  Your model's score as a percentage (0–100). Present when job is completed.
</ResponseField>

<ResponseField name="total_scored" type="number">
  Total number of units scored. Present when job is completed.
</ResponseField>

<ResponseField name="current" type="number">
  Number of prompts processed so far. Present while the job is running.
</ResponseField>

<ResponseField name="total" type="number">
  Total number of prompts to process. Present while the job is running.
</ResponseField>

<ResponseField name="message" type="string">
  A progress message from the runner. Present while the job is running.
</ResponseField>

```json theme={null}
{
  "job_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "status": "completed",
  "module": "Haystack",
  "benchmark_id": 1,
  "benchmark_name": "haystack-medical-v1",
  "endpoint_url": "https://your-model.example.com/v1/chat/completions",
  "needle_annnoted_count": 3,
  "hay_non_annotated_count": 30,
  "total_prompts": 33,
  "price": 3.0,
  "created_at": "2024-06-01T12:00:00Z",
  "completed_at": "2024-06-01T12:05:42Z",
  "passed_count": 2,
  "failed_count": 1,
  "score_pct": 66.67,
  "total_scored": 3
}
```

<Tip>
  When a job is still in progress (`status: "running"`), the response also
  includes `current`, `total`, and optionally `message` fields so you can track
  real-time progress.
</Tip>