Get Eval Run Responses

GET /v2/eval-runs/{eval_run_uuid}/responses
Api key Auth bearer

Retrieve all responses for a specific eval run.

Args: eval_run_uuid (str): UUID of the eval run to retrieve responses for. workspace_uuid (str, optional): UUID of the workspace. Defaults to None.

Returns: list[EvalScoredResponse]: List of scored responses for the eval run.

Raises: AymaraAPIError: If the organization is missing or the eval run is not found.

Example: GET /api/eval-runs/{eval_run_uuid}/responses

Path parameters

  • eval_run_uuid string Required

Query parameters

  • workspace_uuid string
  • limit integer

    Minimum value is 1. Default value is 100.

  • offset integer

    Minimum value is 0. Default value is 0.

Responses

  • 200 application/json

    OK

    Hide response attributes Show response attributes object
    • items array[object] Required

      Schema for returning AI response data.

      Hide items attributes Show items attributes object
      • prompt_uuid string Required

        Unique identifier for the prompt.

      • thread_uuid string | null
      • turn_number integer

        Turn number in the conversation (default: 1).

        Default value is 1.

      • continue_thread boolean

        Whether to continue the thread after this response.

        Default value is false.

      • content string | null | object

        Content of the AI response or a file reference.

        Any of:
        string-1 string | null FileReference object | null
      • content_type string

        Content type for AI interactions.

        Values are text or image. Default value is text.

      • exclude_from_scoring boolean

        Whether to exclude this response from scoring.

        Default value is false.

      • ai_refused boolean

        Whether the AI refused to answer the prompt.

        Default value is false.

      • response_uuid string | null
      • explanation string | null
      • confidence number | null
      • is_passed boolean | null
      • next_prompt object | null
        Hide next_prompt attributes Show next_prompt attributes object | null
        • prompt_uuid string Required

          Unique identifier for the prompt.

        • thread_uuid string | null
        • turn_number integer

          Turn number in the conversation (default: 1).

          Default value is 1.

        • content string Required

          Content of the prompt.

        • category string | null
    • count integer Required
  • 400 application/json

    Bad Request

    Hide response attributes Show response attributes object
    • error object Required

      Schema for the contents of an error response.

      This schema defines the structure of the error data inside the error field of an API error response.

      Hide error attributes Show error attributes object
      • code string Required

        Enumeration of all error codes used in the API.

        Values are auth.invalid_key, auth.expired_key, auth.insufficient_permissions, validation.invalid_request, validation.invalid_format, resource.not_found, resource.conflict, quota.limit_exceeded, or server.internal_error.

      • message string Required
      • details object

        Default value is {} (empty).

    • request_id string

      Default value is empty.
  • 401 application/json

    Unauthorized

    Hide response attributes Show response attributes object
    • error object Required

      Schema for the contents of an error response.

      This schema defines the structure of the error data inside the error field of an API error response.

      Hide error attributes Show error attributes object
      • code string Required

        Enumeration of all error codes used in the API.

        Values are auth.invalid_key, auth.expired_key, auth.insufficient_permissions, validation.invalid_request, validation.invalid_format, resource.not_found, resource.conflict, quota.limit_exceeded, or server.internal_error.

      • message string Required
      • details object

        Default value is {} (empty).

    • request_id string

      Default value is empty.
  • 403 application/json

    Forbidden

    Hide response attributes Show response attributes object
    • error object Required

      Schema for the contents of an error response.

      This schema defines the structure of the error data inside the error field of an API error response.

      Hide error attributes Show error attributes object
      • code string Required

        Enumeration of all error codes used in the API.

        Values are auth.invalid_key, auth.expired_key, auth.insufficient_permissions, validation.invalid_request, validation.invalid_format, resource.not_found, resource.conflict, quota.limit_exceeded, or server.internal_error.

      • message string Required
      • details object

        Default value is {} (empty).

    • request_id string

      Default value is empty.
  • 404 application/json

    Not Found

    Hide response attributes Show response attributes object
    • error object Required

      Schema for the contents of an error response.

      This schema defines the structure of the error data inside the error field of an API error response.

      Hide error attributes Show error attributes object
      • code string Required

        Enumeration of all error codes used in the API.

        Values are auth.invalid_key, auth.expired_key, auth.insufficient_permissions, validation.invalid_request, validation.invalid_format, resource.not_found, resource.conflict, quota.limit_exceeded, or server.internal_error.

      • message string Required
      • details object

        Default value is {} (empty).

    • request_id string

      Default value is empty.
  • 409 application/json

    Conflict

    Hide response attributes Show response attributes object
    • error object Required

      Schema for the contents of an error response.

      This schema defines the structure of the error data inside the error field of an API error response.

      Hide error attributes Show error attributes object
      • code string Required

        Enumeration of all error codes used in the API.

        Values are auth.invalid_key, auth.expired_key, auth.insufficient_permissions, validation.invalid_request, validation.invalid_format, resource.not_found, resource.conflict, quota.limit_exceeded, or server.internal_error.

      • message string Required
      • details object

        Default value is {} (empty).

    • request_id string

      Default value is empty.
  • 422 application/json

    Unprocessable Entity

    Hide response attributes Show response attributes object
    • error object Required

      Schema for the contents of an error response.

      This schema defines the structure of the error data inside the error field of an API error response.

      Hide error attributes Show error attributes object
      • code string Required

        Enumeration of all error codes used in the API.

        Values are auth.invalid_key, auth.expired_key, auth.insufficient_permissions, validation.invalid_request, validation.invalid_format, resource.not_found, resource.conflict, quota.limit_exceeded, or server.internal_error.

      • message string Required
      • details object

        Default value is {} (empty).

    • request_id string

      Default value is empty.
  • 429 application/json

    Too Many Requests

    Hide response attributes Show response attributes object
    • error object Required

      Schema for the contents of an error response.

      This schema defines the structure of the error data inside the error field of an API error response.

      Hide error attributes Show error attributes object
      • code string Required

        Enumeration of all error codes used in the API.

        Values are auth.invalid_key, auth.expired_key, auth.insufficient_permissions, validation.invalid_request, validation.invalid_format, resource.not_found, resource.conflict, quota.limit_exceeded, or server.internal_error.

      • message string Required
      • details object

        Default value is {} (empty).

    • request_id string

      Default value is empty.
  • 500 application/json

    Internal Server Error

    Hide response attributes Show response attributes object
    • error object Required

      Schema for the contents of an error response.

      This schema defines the structure of the error data inside the error field of an API error response.

      Hide error attributes Show error attributes object
      • code string Required

        Enumeration of all error codes used in the API.

        Values are auth.invalid_key, auth.expired_key, auth.insufficient_permissions, validation.invalid_request, validation.invalid_format, resource.not_found, resource.conflict, quota.limit_exceeded, or server.internal_error.

      • message string Required
      • details object

        Default value is {} (empty).

    • request_id string

      Default value is empty.
  • 503 application/json

    Service Unavailable

    Hide response attributes Show response attributes object
    • error object Required

      Schema for the contents of an error response.

      This schema defines the structure of the error data inside the error field of an API error response.

      Hide error attributes Show error attributes object
      • code string Required

        Enumeration of all error codes used in the API.

        Values are auth.invalid_key, auth.expired_key, auth.insufficient_permissions, validation.invalid_request, validation.invalid_format, resource.not_found, resource.conflict, quota.limit_exceeded, or server.internal_error.

      • message string Required
      • details object

        Default value is {} (empty).

    • request_id string

      Default value is empty.
GET /v2/eval-runs/{eval_run_uuid}/responses 
import os
from aymara_ai import AymaraAI

client = AymaraAI(
    api_key=os.environ.get("AYMARA_AI_API_KEY"),  # This is the default and can be omitted
)
page = client.evals.runs.list_responses(
    eval_run_uuid="eval_run_uuid",
)
page = page.items[0]
print(page.prompt_uuid)
curl \
 --request GET 'https://api.aymara.ai/v2/eval-runs/{eval_run_uuid}/responses' \
 --header "x-api-key: $API_KEY"
Response examples (200) 
{
  "items": [
    {
      "prompt_uuid": "string",
      "thread_uuid": "string",
      "turn_number": 1,
      "continue_thread": false,
      "content": "string",
      "content_type": "text",
      "exclude_from_scoring": false,
      "ai_refused": false,
      "response_uuid": "string",
      "explanation": "string",
      "confidence": 42.0,
      "is_passed": true,
      "next_prompt": {
        "prompt_uuid": "string",
        "thread_uuid": "string",
        "turn_number": 1,
        "content": "string",
        "category": "string"
      }
    }
  ],
  "count": 42
}
Response examples (400) 
{
  "error": {
    "code": "auth.invalid_key",
    "message": "string",
    "details": {}
  },
  "request_id": ""
}
Response examples (401) 
{
  "error": {
    "code": "auth.invalid_key",
    "message": "string",
    "details": {}
  },
  "request_id": ""
}
Response examples (403) 
{
  "error": {
    "code": "auth.invalid_key",
    "message": "string",
    "details": {}
  },
  "request_id": ""
}
Response examples (404) 
{
  "error": {
    "code": "auth.invalid_key",
    "message": "string",
    "details": {}
  },
  "request_id": ""
}
Response examples (409) 
{
  "error": {
    "code": "auth.invalid_key",
    "message": "string",
    "details": {}
  },
  "request_id": ""
}
Response examples (422) 
{
  "error": {
    "code": "auth.invalid_key",
    "message": "string",
    "details": {}
  },
  "request_id": ""
}
Response examples (429) 
{
  "error": {
    "code": "auth.invalid_key",
    "message": "string",
    "details": {}
  },
  "request_id": ""
}
Response examples (500) 
{
  "error": {
    "code": "auth.invalid_key",
    "message": "string",
    "details": {}
  },
  "request_id": ""
}
Response examples (503) 
{
  "error": {
    "code": "auth.invalid_key",
    "message": "string",
    "details": {}
  },
  "request_id": ""
}