List Eval Runs

GET /v2/eval-runs

List all eval runs, with optional filtering.

Args: eval_uuid (str, optional): UUID of the eval to filter runs by. Defaults to None. workspace_uuid (str, optional): UUID of the workspace. Defaults to None.

Returns: list[EvalRunResult]: List of eval runs matching the filters.

Raises: AymaraAPIError: If the organization is missing.

Example: GET /api/eval-runs?eval_uuid=...&workspace_uuid=...

Query parameters

Responses

  • 200 application/json

    OK

    Hide response attributes Show response attributes object
  • 400 application/json

    Bad Request

    Hide response attributes Show response attributes object
    • error object Required

      Schema for the contents of an error response.

      This schema defines the structure of the error data inside the error field of an API error response.

      Hide error attributes Show error attributes object
      • code string Required

        Enumeration of all error codes used in the API.

        Values are auth.invalid_key, auth.expired_key, auth.insufficient_permissions, validation.invalid_request, validation.invalid_format, resource.not_found, resource.conflict, quota.limit_exceeded, or server.internal_error.

      • message string Required
      • details object

        Default value is {} (empty).

    • Default value is empty.

  • 401 application/json

    Unauthorized

    Hide response attributes Show response attributes object
    • error object Required

      Schema for the contents of an error response.

      This schema defines the structure of the error data inside the error field of an API error response.

      Hide error attributes Show error attributes object
      • code string Required

        Enumeration of all error codes used in the API.

        Values are auth.invalid_key, auth.expired_key, auth.insufficient_permissions, validation.invalid_request, validation.invalid_format, resource.not_found, resource.conflict, quota.limit_exceeded, or server.internal_error.

      • message string Required
      • details object

        Default value is {} (empty).

    • Default value is empty.

  • 403 application/json

    Forbidden

    Hide response attributes Show response attributes object
    • error object Required

      Schema for the contents of an error response.

      This schema defines the structure of the error data inside the error field of an API error response.

      Hide error attributes Show error attributes object
      • code string Required

        Enumeration of all error codes used in the API.

        Values are auth.invalid_key, auth.expired_key, auth.insufficient_permissions, validation.invalid_request, validation.invalid_format, resource.not_found, resource.conflict, quota.limit_exceeded, or server.internal_error.

      • message string Required
      • details object

        Default value is {} (empty).

    • Default value is empty.

  • 404 application/json

    Not Found

    Hide response attributes Show response attributes object
    • error object Required

      Schema for the contents of an error response.

      This schema defines the structure of the error data inside the error field of an API error response.

      Hide error attributes Show error attributes object
      • code string Required

        Enumeration of all error codes used in the API.

        Values are auth.invalid_key, auth.expired_key, auth.insufficient_permissions, validation.invalid_request, validation.invalid_format, resource.not_found, resource.conflict, quota.limit_exceeded, or server.internal_error.

      • message string Required
      • details object

        Default value is {} (empty).

    • Default value is empty.

  • 409 application/json

    Conflict

    Hide response attributes Show response attributes object
    • error object Required

      Schema for the contents of an error response.

      This schema defines the structure of the error data inside the error field of an API error response.

      Hide error attributes Show error attributes object
      • code string Required

        Enumeration of all error codes used in the API.

        Values are auth.invalid_key, auth.expired_key, auth.insufficient_permissions, validation.invalid_request, validation.invalid_format, resource.not_found, resource.conflict, quota.limit_exceeded, or server.internal_error.

      • message string Required
      • details object

        Default value is {} (empty).

    • Default value is empty.

  • 422 application/json

    Unprocessable Entity

    Hide response attributes Show response attributes object
    • error object Required

      Schema for the contents of an error response.

      This schema defines the structure of the error data inside the error field of an API error response.

      Hide error attributes Show error attributes object
      • code string Required

        Enumeration of all error codes used in the API.

        Values are auth.invalid_key, auth.expired_key, auth.insufficient_permissions, validation.invalid_request, validation.invalid_format, resource.not_found, resource.conflict, quota.limit_exceeded, or server.internal_error.

      • message string Required
      • details object

        Default value is {} (empty).

    • Default value is empty.

  • 429 application/json

    Too Many Requests

    Hide response attributes Show response attributes object
    • error object Required

      Schema for the contents of an error response.

      This schema defines the structure of the error data inside the error field of an API error response.

      Hide error attributes Show error attributes object
      • code string Required

        Enumeration of all error codes used in the API.

        Values are auth.invalid_key, auth.expired_key, auth.insufficient_permissions, validation.invalid_request, validation.invalid_format, resource.not_found, resource.conflict, quota.limit_exceeded, or server.internal_error.

      • message string Required
      • details object

        Default value is {} (empty).

    • Default value is empty.

  • 500 application/json

    Internal Server Error

    Hide response attributes Show response attributes object
    • error object Required

      Schema for the contents of an error response.

      This schema defines the structure of the error data inside the error field of an API error response.

      Hide error attributes Show error attributes object
      • code string Required

        Enumeration of all error codes used in the API.

        Values are auth.invalid_key, auth.expired_key, auth.insufficient_permissions, validation.invalid_request, validation.invalid_format, resource.not_found, resource.conflict, quota.limit_exceeded, or server.internal_error.

      • message string Required
      • details object

        Default value is {} (empty).

    • Default value is empty.

  • 503 application/json

    Service Unavailable

    Hide response attributes Show response attributes object
    • error object Required

      Schema for the contents of an error response.

      This schema defines the structure of the error data inside the error field of an API error response.

      Hide error attributes Show error attributes object
      • code string Required

        Enumeration of all error codes used in the API.

        Values are auth.invalid_key, auth.expired_key, auth.insufficient_permissions, validation.invalid_request, validation.invalid_format, resource.not_found, resource.conflict, quota.limit_exceeded, or server.internal_error.

      • message string Required
      • details object

        Default value is {} (empty).

    • Default value is empty.

GET /v2/eval-runs
import os
from aymara_ai import AymaraAI

client = AymaraAI(
    api_key=os.environ.get("AYMARA_AI_API_KEY"),  # This is the default and can be omitted
)
page = client.evals.runs.list()
page = page.items[0]
print(page.eval_run_uuid)
curl \
 --request GET 'https://api.aymara.ai/v2/eval-runs' \
 --header "x-api-key: $API_KEY"
Response examples (200)
{
  "items": [
    {
      "eval_run_uuid": "string",
      "eval_uuid": "string",
      "name": "string",
      "status": "created",
      "created_at": "2025-05-04T09:42:00Z",
      "updated_at": "2025-05-04T09:42:00Z",
      "evaluation": {
        "eval_uuid": "string",
        "name": "string",
        "ai_description": "string",
        "ai_instructions": "string",
        "eval_type": "string",
        "eval_instructions": "string",
        "language": "en",
        "modality": "text",
        "ground_truth": "string",
        "num_prompts": 100,
        "prompt_examples": [
          {
            "content": "string",
            "example_uuid": "string",
            "type": "good",
            "explanation": "string"
          }
        ],
        "is_jailbreak": false,
        "is_sandbox": false,
        "workspace_uuid": "string",
        "status": "created",
        "created_at": "2025-05-04T09:42:00Z",
        "updated_at": "2025-05-04T09:42:00Z"
      },
      "ai_description": "string",
      "workspace_uuid": "string",
      "pass_rate": 42.0,
      "num_prompts": 42,
      "num_responses_scored": 42,
      "responses": [
        {
          "prompt_uuid": "string",
          "thread_uuid": "string",
          "turn_number": 1,
          "continue_thread": false,
          "content": "string",
          "content_type": "text",
          "exclude_from_scoring": false,
          "ai_refused": false,
          "response_uuid": "string",
          "explanation": "string",
          "confidence": 42.0,
          "is_passed": true,
          "next_prompt": {
            "prompt_uuid": "string",
            "thread_uuid": "string",
            "turn_number": 1,
            "content": "string",
            "category": "string"
          }
        }
      ]
    }
  ],
  "count": 42
}
Response examples (400)
{
  "error": {
    "code": "auth.invalid_key",
    "message": "string",
    "details": {}
  },
  "request_id": ""
}
Response examples (401)
{
  "error": {
    "code": "auth.invalid_key",
    "message": "string",
    "details": {}
  },
  "request_id": ""
}
Response examples (403)
{
  "error": {
    "code": "auth.invalid_key",
    "message": "string",
    "details": {}
  },
  "request_id": ""
}
Response examples (404)
{
  "error": {
    "code": "auth.invalid_key",
    "message": "string",
    "details": {}
  },
  "request_id": ""
}
Response examples (409)
{
  "error": {
    "code": "auth.invalid_key",
    "message": "string",
    "details": {}
  },
  "request_id": ""
}
Response examples (422)
{
  "error": {
    "code": "auth.invalid_key",
    "message": "string",
    "details": {}
  },
  "request_id": ""
}
Response examples (429)
{
  "error": {
    "code": "auth.invalid_key",
    "message": "string",
    "details": {}
  },
  "request_id": ""
}
Response examples (500)
{
  "error": {
    "code": "auth.invalid_key",
    "message": "string",
    "details": {}
  },
  "request_id": ""
}
Response examples (503)
{
  "error": {
    "code": "auth.invalid_key",
    "message": "string",
    "details": {}
  },
  "request_id": ""
}