Skip to main content
PATCH
Update a benchmark

Authorizations

Authorization
string
header
required

Send Authorization: Bearer YOUR_API_KEY. The key resolves to a single organization.

Path Parameters

id
string<uuid>
required

Benchmark id.

Body

application/json
name
string
Required string length: 1 - 120
description
string | null
Maximum string length: 500
language
string | null
Required string length: 2 - 16
agentIds
string<uuid>[]

Agents the benchmark is assigned to (1-20). Single-prompt agents only in v1.

Required array length: 1 - 20 elements
settings
object
scoring
object

Scoring settings, versioned through the draft.

Response

Successful response.

One benchmark with its versions, Judges, groups and case counts (runs: the runs endpoints).

id
string
required
name
string
required
description
string | null
required
kind
enum<string>
required
Available options:
llm,
stt
agents
object[]
required
createdAt
string
required

ISO 8601 timestamp.

updatedAt
string
required

ISO 8601 timestamp.

currentVersion
integer | null
required
hasDraftChanges
boolean
required
versions
object[]
required

Newest first, the draft (if any) first.

checkGroups
object[]
required
judges
object[]
required
caseCounts
object
required
draftCaseCount
integer | null

Cases the open draft would publish; null when the benchmark has no draft.

archivedAt
string | null

ISO 8601 timestamp.

settings
object

Benchmark-level run defaults. Not versioned.

language
string | null
scoring
object
validatorKinds
enum<string>[]
Available options:
judge,
code,
builtin