Skip to main content
POST
Start a run

Authorizations

Authorization
string
header
required

Send Authorization: Bearer YOUR_API_KEY. The key resolves to a single organization.

Headers

Idempotency-Key
string

Optional key, unique per organization (1-255 printable ASCII characters, e.g. a UUID). Resending a request with the same key returns the original result instead of creating a second one; reusing it for a different request is a 409 idempotency_key_reused.

Required string length: 1 - 255

Path Parameters

id
string<uuid>
required

Benchmark id.

Body

application/json
candidates
object[]
required

What the run compares, in display order.

Required array length: 1 - 10 elements
estimateToken
string
required

The estimateToken from the estimate, unchanged.

Required string length: 1 - 8192
repeats
integer

Repeats per case (LLM, 1-3). STT always 1.

Required range: 1 <= x <= 3
versionNumber
integer

Published version to run. Default: the current published version (or the draft when nothing is published).

Required range: x >= 1
draft
boolean

Run the unpublished draft instead of a published version.

Response

Successful response.

A run: what it compares, its state, progress and cost.

id
string
required
benchmarkId
string
required
number
number
required
status
enum<string>
required

awaiting_approval: waiting for an owner (over the spend cap). rejected: an owner rejected it or the request expired.

Available options:
awaiting_approval,
queued,
running,
completed,
failed,
cancelled,
rejected
cancelRequested
boolean
required
trigger
enum<string>
required
Available options:
ui,
api,
mcp
versionNumber
number | null
required
isDraftRun
boolean
required
repeats
number
required
totalCells
number
required
completedCells
number
required
passedCells
number
required
failedCells
number
required
erroredCells
number
required
skippedCells
number
required
candidates
object[]
required
results
object[]
required
estimatedCostUsd
number
required

USD.

costCapUsd
number | null
required

USD.

actualCostUsd
number | null
required

USD.

caseSetHash
string | null
required
failureReason
string | null
required
createdBy
string
required
createdByUser
object
required

An organization member.

createdAt
string
required
startedAt
string | null
required
finishedAt
string | null
required
rescoredAt
string | null
required
approval
object | null
required

An owner approval request for a run over the spend cap or threshold.

publishedAsVersionNumber
number | null

Draft runs only: the version later published from the draft this run ran on (it was the draft at the time). Null while that draft is still open, and for runs of a published version.

{key}
any