> ## Documentation Index > Fetch the complete documentation index at: https://docs.zelto.ai/llms.txt > Use this file to discover all available pages before exploring further. # Add a Judge > Adds an LLM, code or built-in Judge to the draft. Over MCP, pass `judgeId` to update an existing Judge instead. ## OpenAPI ````yaml /docs-openapi.json post /v1/benchmarks/{id}/judges openapi: 3.1.0 info: title: Zelto REST API version: v1 description: >- Programmatic access to your Zelto organization. Every endpoint is scoped to the organization the API key belongs to. Mint a key at Settings → Integrations → API integration. servers: - url: https://api.zelto.ai description: Production security: - bearerAuth: [] tags: - name: Monitors description: Filtered, asynchronous exports of stored monitor results. - name: Agents description: Voice and text agents normalised through Zelto's ingestion pipeline. - name: Conversations description: >- Phone calls, chats, and voicemails — plus their transcripts and recordings. - name: Customer Metrics description: >- Customer-provided evaluations attached to a conversation (requires metrics enabled). - name: Reviews description: Human review state for conversations. - name: Buckets description: Reviewer routing — group conversations for batched review. - name: Companies description: Clients or brands your agents work for, and their logos. - name: Findings description: >- Structured issues discovered during analysis, with optional conversation attachments and comments. - name: Experiments description: >- Prospective measurement of two externally routed prompt versions across independent monitor outcomes. - name: Organizations description: >- Headless org provisioning. Requires an admin/service API key (not a read-only org-scoped customer key). - name: AI providers description: >- Your organization's own (bring-your-own) AI provider keys. Monitors can run on them instead of Zelto's built-in models. Owner or admin keys only; the secret is write-only. - name: Benchmarks description: >- Repeatable evaluations of agent prompts, models and speech-to-text engines over a fixed set of cases, scored by Judges. paths: /v1/benchmarks/{id}/judges: post: tags: - Benchmarks summary: Add a Judge description: >- Adds an LLM, code or built-in Judge to the draft. Over MCP, pass `judgeId` to update an existing Judge instead. operationId: createBenchmarkJudge parameters: - name: id in: path required: true description: Benchmark id. schema: type: string format: uuid requestBody: required: true content: application/json: schema: type: object properties: label: type: string minLength: 1 maxLength: 120 labelTemplate: type: string maxLength: 200 kind: type: string enum: - judge - code - builtin required: default: true type: boolean weight: default: 1 type: number minimum: 0.1 maximum: 10 checkGroupId: default: null anyOf: - type: string format: uuid - type: 'null' appliesTo: default: {} type: object properties: caseTypes: type: array items: type: string enum: - turn - conversation - audio tags: maxItems: 20 type: array items: type: string maxLength: 40 params: default: [] maxItems: 10 type: array items: type: object properties: key: type: string minLength: 1 maxLength: 40 label: type: string maxLength: 60 type: type: string enum: - number - text - percent default: anyOf: - type: number - type: string maxLength: 500 unit: type: string maxLength: 20 min: type: number max: type: number required: - key - label - type - default config: $ref: '#/components/schemas/BenchmarkJudgeConfig' toolCheck: type: string enum: - tool_call_match - tool_args_schema - tool_args_match required: - label - kind - config responses: '201': description: Successful response. content: application/json: schema: $ref: '#/components/schemas/BenchmarkJudge' '400': $ref: '#/components/responses/ValidationFailed' '401': $ref: '#/components/responses/Unauthorized' '403': description: >- `phi_mode_blocked`: The organization is in PHI mode, which allows only Bedrock-routed models and AWS Transcribe. content: application/json: schema: $ref: '#/components/schemas/BenchmarkError' '404': $ref: '#/components/responses/NotFound' components: schemas: BenchmarkJudgeConfig: oneOf: - type: object properties: kind: type: string const: judge rubric: type: string maxLength: 8000 model: type: object properties: provider: type: string enum: - platform - openai - anthropic - openrouter - vercel_gateway modelId: type: string minLength: 1 maxLength: 200 aiKeyId: type: - string - 'null' required: - provider - modelId scale: oneOf: - type: object properties: type: type: string const: pass_fail required: - type - type: object properties: type: type: string const: score_1_5 threshold: type: integer minimum: 1 maximum: 5 required: - type - threshold required: - kind - rubric - model - scale - type: object properties: kind: type: string const: code code: type: string maxLength: 20000 required: - kind - code - type: object properties: kind: type: string const: builtin rule: type: string enum: - reply_length - contains - regex_match - exact_match - wer_max - key_terms - tool_call_match - tool_args_schema - tool_args_match - transition_match - goal_reached - max_turns - path_includes required: - kind - rule description: >- Kind-specific Judge configuration: LLM rubric + model + scale, code, or a built-in rule. BenchmarkJudge: type: object properties: id: type: string label: type: string kind: type: string enum: - judge - code - builtin required: type: boolean weight: type: number summary: type: string recentPassRate: type: - number - 'null' revision: type: integer costPer1kUsd: type: - number - 'null' params: type: array items: type: object properties: key: type: string minLength: 1 maxLength: 40 label: type: string maxLength: 60 type: type: string enum: - number - text - percent default: anyOf: - type: number - type: string maxLength: 500 unit: type: string maxLength: 20 min: type: number max: type: number required: - key - label - type - default labelTemplate: type: string toolCheck: type: string enum: - tool_call_match - tool_args_schema - tool_args_match config: $ref: '#/components/schemas/BenchmarkJudgeConfig' checkGroupId: type: - string - 'null' appliesTo: type: object properties: caseTypes: type: array items: type: string enum: - turn - conversation - audio tags: maxItems: 20 type: array items: type: string maxLength: 40 required: - id - label - kind - required - weight - summary - recentPassRate - revision description: A Judge (validator) that scores each case. BenchmarkError: type: object required: - error - code - message properties: error: type: string enum: - validation_failed - invalid_parameter - approval_required - plan_limit - phi_mode_blocked - owner_required - not_entitled - forbidden - not_found - payload_too_large - estimate_stale - pointer_moved - no_draft_changes - conflict - idempotency_key_reused - rate_limited - model_unavailable description: Stable error code (same value as `code`). code: type: string enum: - validation_failed - invalid_parameter - approval_required - plan_limit - phi_mode_blocked - owner_required - not_entitled - forbidden - not_found - payload_too_large - estimate_stale - pointer_moved - no_draft_changes - conflict - idempotency_key_reused - rate_limited - model_unavailable description: Stable, machine-readable cause; match on this. message: type: string description: A plain-English sentence saying what is wrong. hint: type: string description: The next step to try. details: type: object additionalProperties: true description: >- Code-specific context, for example `approvalRequestId` with `approval_required` or `cap` with `plan_limit`. Error: type: object required: - error properties: error: type: string description: >- Short error name: `Unauthorized`, `Validation failed`, `Not found`, or a stable snake_case code you can match on, such as `unknown_query_parameter`, `invalid_parameter`, `payload_too_large` or `not_found`. message: type: string description: A plain-English sentence saying what is wrong and how to fix it. code: type: string description: >- Stable, machine-readable cause; match on this rather than on `error` or `message`. On a 401: `missing_credentials`, `malformed_credentials`, `ingest_key_not_allowed` (a write-only observability ingest key was used on an endpoint that needs an API key), `key_expired` or `invalid_credentials`. Elsewhere, for example: `not_found`, `invalid_parameter`, `unknown_query_parameter`, `missing_parameter`, `validation_failed`, `invalid_json`, `payload_too_large`, `conversation_not_found`, `plan_required`. hint: type: string description: >- The next step to try: where to find the right id, which parameters the endpoint takes, how to upgrade the plan, how to authenticate. parameter: type: string description: 'With `invalid_parameter`: the query parameter that was rejected.' unknown: type: array items: type: string description: >- With `unknown_query_parameter`: the parameters that were sent but are not accepted here. valid: type: array items: type: string description: >- With `unknown_query_parameter`: the parameters this endpoint accepts. details: type: object additionalProperties: true responses: ValidationFailed: description: >- The request body or query string failed validation. `code` says which kind (`validation_failed`, `invalid_json`, `invalid_parameter`, `unknown_query_parameter`, `missing_parameter`), and an enum filter names the allowed values in `valid` (every repeated value is checked, e.g. `?status=open&status=zz` is a 400). A `validation_failed` body also carries `message` (the first failing fields), `hint` (how to fix it, for example that a finding `description` is a TipTap JSON document and not a string) and `details` (field name to messages). content: application/json: schema: $ref: '#/components/schemas/Error' Unauthorized: description: >- Missing or invalid API key. The body names the cause in `code` (for example `ingest_key_not_allowed` when a write-only ingest key is used here) with a `message` and `hint`. content: application/json: schema: $ref: '#/components/schemas/Error' NotFound: description: >- Resource not found in your organization. Also returned when a path id is not a UUID, and `{ "error": "not_found" }` for a path this API does not have. The body keeps `error: "Not found"` and adds `code: "not_found"` and a `hint`. content: application/json: schema: $ref: '#/components/schemas/Error' securitySchemes: bearerAuth: type: http scheme: bearer description: >- Send `Authorization: Bearer YOUR_API_KEY`. The key resolves to a single organization. ```` This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.