> ## Documentation Index > Fetch the complete documentation index at: https://docs.zelto.ai/llms.txt > Use this file to discover all available pages before exploring further. # Add cases > Creates manual cases in the draft. Audio cases are created as drafts until their clip is uploaded (clip uploads ship with speech-to-text). ## OpenAPI ````yaml /docs-openapi.json post /v1/benchmarks/{id}/cases openapi: 3.1.0 info: title: Zelto REST API version: v1 description: >- Programmatic access to your Zelto organization. Every endpoint is scoped to the organization the API key belongs to. Mint a key at Settings → Integrations → API integration. servers: - url: https://api.zelto.ai description: Production security: - bearerAuth: [] tags: - name: Monitors description: Filtered, asynchronous exports of stored monitor results. - name: Agents description: Voice and text agents normalised through Zelto's ingestion pipeline. - name: Conversations description: >- Phone calls, chats, and voicemails — plus their transcripts and recordings. - name: Customer Metrics description: >- Customer-provided evaluations attached to a conversation (requires metrics enabled). - name: Reviews description: Human review state for conversations. - name: Buckets description: Reviewer routing — group conversations for batched review. - name: Companies description: Clients or brands your agents work for, and their logos. - name: Findings description: >- Structured issues discovered during analysis, with optional conversation attachments and comments. - name: Experiments description: >- Prospective measurement of two externally routed prompt versions across independent monitor outcomes. - name: Organizations description: >- Headless org provisioning. Requires an admin/service API key (not a read-only org-scoped customer key). - name: AI providers description: >- Your organization's own (bring-your-own) AI provider keys. Monitors can run on them instead of Zelto's built-in models. Owner or admin keys only; the secret is write-only. - name: Benchmarks description: >- Repeatable evaluations of agent prompts, models and speech-to-text engines over a fixed set of cases, scored by Judges. paths: /v1/benchmarks/{id}/cases: post: tags: - Benchmarks summary: Add cases description: >- Creates manual cases in the draft. Audio cases are created as drafts until their clip is uploaded (clip uploads ship with speech-to-text). operationId: createBenchmarkCases parameters: - name: id in: path required: true description: Benchmark id. schema: type: string format: uuid - $ref: '#/components/parameters/BenchmarkIdempotencyKey' requestBody: required: true content: application/json: schema: type: object properties: cases: minItems: 1 maxItems: 100 type: array items: type: object properties: type: type: string enum: - turn - conversation - audio title: type: string minLength: 1 maxLength: 160 tags: description: User tags (max 20); normalized to lowercase-kebab. maxItems: 20 type: array items: type: string maxLength: 200 checkGroupId: anyOf: - type: string format: uuid - type: 'null' language: anyOf: - type: string minLength: 2 maxLength: 16 - type: 'null' agentVersionId: description: >- The agent version whose prompt and tools the case was written against. Null = prompt as recorded. anyOf: - type: string format: uuid - type: 'null' agentId: description: >- The benchmark agent the case was written for (shown on the case). Default: the agent of `agentVersionId`. type: string format: uuid draft: description: Draft cases are saved but never run until confirmed. type: boolean turns: description: >- turn: the conversation so far, ending with the caller turn the agent answers. maxItems: 200 type: array items: $ref: '#/components/schemas/BenchmarkTurn' variables: type: object propertyNames: type: string minLength: 1 maxLength: 100 additionalProperties: type: string maxLength: 2000 expected: description: 'turn: the expected outcome.' anyOf: - type: string maxLength: 20000 - type: 'null' replyExpectation: $ref: '#/components/schemas/BenchmarkReplyExpectation' toolExpectation: $ref: '#/components/schemas/BenchmarkToolExpectation' tools: maxItems: 50 type: array items: $ref: '#/components/schemas/BenchmarkToolDef' toolMocks: maxItems: 50 type: array items: $ref: '#/components/schemas/BenchmarkToolMock' conversation: description: 'conversation: the simulated caller.' type: object properties: persona: type: string minLength: 1 maxLength: 4000 goal: type: string minLength: 1 maxLength: 4000 openingLine: anyOf: - type: string maxLength: 4000 - type: 'null' maxTurns: type: integer minimum: 1 maximum: 30 required: - persona - goal referenceTranscript: description: 'audio: the reference.' anyOf: - type: string maxLength: 20000 - type: 'null' keyTerms: maxItems: 50 type: array items: type: string minLength: 1 maxLength: 200 referenceVerified: description: >- audio: true marks the reference transcript as checked by a person; false flags it as unverified (it still runs). type: boolean skippedJudgeIds: maxItems: 100 type: array items: type: string format: uuid judgeParamOverrides: type: object propertyNames: type: string minLength: 1 maxLength: 100 additionalProperties: type: object propertyNames: type: string minLength: 1 maxLength: 100 additionalProperties: anyOf: - type: number - type: string maxLength: 500 required: - type - title description: Up to 100 cases per request. required: - cases responses: '201': description: Successful response. content: application/json: schema: $ref: '#/components/schemas/BenchmarkCaseBatchResult' headers: Idempotent-Replayed: description: >- `true` when this response replays an earlier request with the same Idempotency-Key. schema: type: string enum: - 'true' '400': $ref: '#/components/responses/ValidationFailed' '401': $ref: '#/components/responses/Unauthorized' '403': description: >- `plan_limit`: This would exceed a Benchmarks limit on the organization's plan. content: application/json: schema: $ref: '#/components/schemas/BenchmarkError' '404': $ref: '#/components/responses/NotFound' '409': description: >- `idempotency_key_reused`: This Idempotency-Key was already used for a different request. content: application/json: schema: $ref: '#/components/schemas/BenchmarkError' components: parameters: BenchmarkIdempotencyKey: name: Idempotency-Key in: header required: false description: >- Optional key, unique per organization (1-255 printable ASCII characters, e.g. a UUID). Resending a request with the same key returns the original result instead of creating a second one; reusing it for a different request is a 409 `idempotency_key_reused`. schema: type: string minLength: 1 maxLength: 255 schemas: BenchmarkTurn: type: object properties: role: type: string enum: - caller - agent - tool text: type: string maxLength: 20000 toolCall: type: object properties: name: type: string minLength: 1 maxLength: 100 pattern: ^[A-Za-z0-9_.:-]+$ arguments: type: object propertyNames: type: string additionalProperties: {} required: - name - arguments toolResult: type: object properties: mocked: type: boolean content: type: string maxLength: 20000 required: - mocked - content nodeId: type: - string - 'null' required: - role - text description: One turn of a conversation. BenchmarkReplyExpectation: type: object properties: check: type: string enum: - reference - different - none text: anyOf: - type: string maxLength: 20000 - type: 'null' idealReply: anyOf: - type: string maxLength: 20000 - type: 'null' required: - check description: How the spoken reply of a turn case is checked. BenchmarkToolExpectation: type: object properties: expect: type: string enum: - call - no_call - any calls: maxItems: 10 type: array items: type: object properties: tool: type: string minLength: 1 maxLength: 100 pattern: ^[A-Za-z0-9_.:-]+$ arguments: type: object propertyNames: type: string minLength: 1 maxLength: 100 additionalProperties: oneOf: - type: object properties: op: type: string const: equals value: {} required: - op - value - type: object properties: op: type: string const: one_of values: minItems: 1 maxItems: 50 type: array items: {} required: - op - values - type: object properties: op: type: string const: regex pattern: type: string minLength: 1 maxLength: 500 required: - op - pattern - type: object properties: op: type: string const: number_range min: type: number max: type: number required: - op - type: object properties: op: type: string const: present required: - op - type: object properties: op: type: string const: semantic description: type: string minLength: 1 maxLength: 500 required: - op - description - type: object properties: op: type: string const: ignore required: - op required: - tool - arguments ordered: type: boolean forbiddenTools: maxItems: 50 type: array items: type: string minLength: 1 maxLength: 100 pattern: ^[A-Za-z0-9_.:-]+$ noExtraArguments: type: boolean replyAt: type: integer minimum: 0 maximum: 10 required: - expect description: Which tool call(s) a turn case expects. BenchmarkToolDef: type: object properties: name: type: string minLength: 1 maxLength: 100 pattern: ^[A-Za-z0-9_.:-]+$ description: type: string maxLength: 2000 parameters: type: object properties: type: type: string const: object properties: type: object propertyNames: type: string additionalProperties: type: object properties: type: type: string enum: - string - number - integer - boolean enum: type: array items: {} description: type: string required: - type required: type: array items: type: string additionalProperties: type: boolean required: - type - properties required: - name - parameters description: A tool schema the candidate is given. BenchmarkToolMock: type: object properties: tool: type: string minLength: 1 maxLength: 100 pattern: ^[A-Za-z0-9_.:-]+$ match: type: object propertyNames: type: string additionalProperties: oneOf: - type: object properties: op: type: string const: equals value: {} required: - op - value - type: object properties: op: type: string const: one_of values: minItems: 1 maxItems: 50 type: array items: {} required: - op - values - type: object properties: op: type: string const: regex pattern: type: string minLength: 1 maxLength: 500 required: - op - pattern - type: object properties: op: type: string const: number_range min: type: number max: type: number required: - op - type: object properties: op: type: string const: present required: - op - type: object properties: op: type: string const: semantic description: type: string minLength: 1 maxLength: 500 required: - op - description - type: object properties: op: type: string const: ignore required: - op response: type: string maxLength: 20000 required: - tool - response description: A canned result for a tool call (tools are never executed in v1). BenchmarkCaseBatchResult: type: object properties: cases: type: array items: $ref: '#/components/schemas/BenchmarkCase' skipped: type: array items: type: object properties: conversationId: type: string traceId: type: string reason: type: string required: - reason dryRun: type: boolean warning: type: string required: - cases description: Cases created by one request (or, with dryRun, the cases that would be). BenchmarkError: type: object required: - error - code - message properties: error: type: string enum: - validation_failed - invalid_parameter - approval_required - plan_limit - phi_mode_blocked - owner_required - not_entitled - forbidden - not_found - payload_too_large - estimate_stale - pointer_moved - no_draft_changes - conflict - idempotency_key_reused - rate_limited - model_unavailable description: Stable error code (same value as `code`). code: type: string enum: - validation_failed - invalid_parameter - approval_required - plan_limit - phi_mode_blocked - owner_required - not_entitled - forbidden - not_found - payload_too_large - estimate_stale - pointer_moved - no_draft_changes - conflict - idempotency_key_reused - rate_limited - model_unavailable description: Stable, machine-readable cause; match on this. message: type: string description: A plain-English sentence saying what is wrong. hint: type: string description: The next step to try. details: type: object additionalProperties: true description: >- Code-specific context, for example `approvalRequestId` with `approval_required` or `cap` with `plan_limit`. BenchmarkCase: type: object properties: id: type: string revision: type: integer type: type: string enum: - turn - conversation - audio title: type: string checkGroupId: type: - string - 'null' source: type: string enum: - production - trace - manual - red_team - api - mcp - upload sourceConversationId: type: - string - 'null' sourceAgent: anyOf: - $ref: '#/components/schemas/BenchmarkAgentRef' - type: 'null' sourceAgentVersionLabel: description: The agent prompt version the case was written against, e.g. "v7". type: string tags: type: array items: type: string language: type: - string - 'null' retired: type: boolean draft: type: boolean referenceUnverified: description: >- audio: the reference transcript came from the call and nobody has checked it yet. The case still runs. type: boolean unpublished: type: boolean preview: type: string skippedJudgeIds: type: array items: type: string judgeParamOverrides: type: object propertyNames: type: string minLength: 1 maxLength: 100 additionalProperties: type: object propertyNames: type: string minLength: 1 maxLength: 100 additionalProperties: anyOf: - type: number - type: string maxLength: 500 addedInVersion: anyOf: - type: integer - type: 'null' retiredInVersion: anyOf: - type: integer - type: 'null' toolExpectation: $ref: '#/components/schemas/BenchmarkToolExpectation' replyExpectation: $ref: '#/components/schemas/BenchmarkReplyExpectation' traceSource: type: object properties: kind: type: string const: trace traceId: type: string spanId: type: - string - 'null' turns: type: string enum: - transcript - span_messages required: - kind - traceId - spanId - turns status: type: string enum: - ready - draft - preparing - failed contentPurged: type: boolean contentDeleted: description: >- "Delete all call content" blanked this case: content and Judge reasoning are gone, the title is a neutral "Case N · content deleted", scores stay. type: boolean createdAt: type: string description: ISO 8601 timestamp. updatedAt: type: string description: ISO 8601 timestamp. required: - id - revision - type - title - checkGroupId - source - sourceConversationId - sourceAgent - tags - language - retired - draft - preview description: 'A case: one input the candidates are scored on.' Error: type: object required: - error properties: error: type: string description: >- Short error name: `Unauthorized`, `Validation failed`, `Not found`, or a stable snake_case code you can match on, such as `unknown_query_parameter`, `invalid_parameter`, `payload_too_large` or `not_found`. message: type: string description: A plain-English sentence saying what is wrong and how to fix it. code: type: string description: >- Stable, machine-readable cause; match on this rather than on `error` or `message`. On a 401: `missing_credentials`, `malformed_credentials`, `ingest_key_not_allowed` (a write-only observability ingest key was used on an endpoint that needs an API key), `key_expired` or `invalid_credentials`. Elsewhere, for example: `not_found`, `invalid_parameter`, `unknown_query_parameter`, `missing_parameter`, `validation_failed`, `invalid_json`, `payload_too_large`, `conversation_not_found`, `plan_required`. hint: type: string description: >- The next step to try: where to find the right id, which parameters the endpoint takes, how to upgrade the plan, how to authenticate. parameter: type: string description: 'With `invalid_parameter`: the query parameter that was rejected.' unknown: type: array items: type: string description: >- With `unknown_query_parameter`: the parameters that were sent but are not accepted here. valid: type: array items: type: string description: >- With `unknown_query_parameter`: the parameters this endpoint accepts. details: type: object additionalProperties: true BenchmarkAgentRef: type: object properties: id: type: string name: type: string required: - id - name description: An agent the benchmark is assigned to. responses: ValidationFailed: description: >- The request body or query string failed validation. `code` says which kind (`validation_failed`, `invalid_json`, `invalid_parameter`, `unknown_query_parameter`, `missing_parameter`), and an enum filter names the allowed values in `valid` (every repeated value is checked, e.g. `?status=open&status=zz` is a 400). A `validation_failed` body also carries `message` (the first failing fields), `hint` (how to fix it, for example that a finding `description` is a TipTap JSON document and not a string) and `details` (field name to messages). content: application/json: schema: $ref: '#/components/schemas/Error' Unauthorized: description: >- Missing or invalid API key. The body names the cause in `code` (for example `ingest_key_not_allowed` when a write-only ingest key is used here) with a `message` and `hint`. content: application/json: schema: $ref: '#/components/schemas/Error' NotFound: description: >- Resource not found in your organization. Also returned when a path id is not a UUID, and `{ "error": "not_found" }` for a path this API does not have. The body keeps `error: "Not found"` and adds `code: "not_found"` and a `hint`. content: application/json: schema: $ref: '#/components/schemas/Error' securitySchemes: bearerAuth: type: http scheme: bearer description: >- Send `Authorization: Bearer YOUR_API_KEY`. The key resolves to a single organization. ```` This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.