Observability

Backend API Reference

Ingest observability batches and query organization-scoped sessions, traces, policies, summaries, and tokens.

Use the ingest endpoint to write completed traces and the organization endpoints to read stored telemetry. SDKs and product bridges normally handle ingestion for you.

Authentication and organization scope

APIAuthenticationOrganization selection
POST /observability/spansX-API-KeyResolved from the API-key record
All GET /organization/:orgId/observability/... endpointsBearer JWTorgId path parameter plus membership check

The ingest request must not include orgId. The backend rejects an API key that is not bound to an organization.

The API-key guard also accepts an API key in Authorization: Bearer ..., but ArmorIQ SDK shippers use X-API-Key; use that header for direct ingest requests.

If the API-key record has a product, that value overrides payload.product. This is useful for product-specific keys but can explain an unexpected product label in the dashboard.

Endpoints

MethodPathPurpose
POST/observability/spansIngest trace batches
GET/organization/:orgId/observability/sessionsList sessions
GET/organization/:orgId/observability/sessions/:idGet one session with traces
GET/organization/:orgId/observability/tracesList traces
GET/organization/:orgId/observability/traces/:idGet one trace with spans
GET/organization/:orgId/observability/policies/:policyIdAggregate one policy
GET/organization/:orgId/observability/analytics/summaryGet overview metrics
GET/organization/:orgId/observability/tokensGroup token and cost usage

Ingest traces

POST /observability/spans
X-API-Key: YOUR_ARMORIQ_API_KEY
Content-Type: application/json
{
  "product": "armoriq-sdk",
  "sessionId": "22222222-2222-4222-8222-222222222222",
  "batches": [
    {
      "trace": {
        "id": "11111111-1111-4111-8111-111111111111",
        "sessionId": "22222222-2222-4222-8222-222222222222",
        "name": "iap.plan",
        "startTime": "2026-07-17T09:00:00.000Z",
        "endTime": "2026-07-17T09:00:00.240Z",
        "durationMs": 240,
        "status": "ok",
        "userId": null,
        "agentId": "support-agent",
        "attributes": { "input": "Find the account owner" },
        "tags": ["armoriq-sdk", "allowed"]
      },
      "spans": [
        {
          "id": "33333333-3333-4333-8333-333333333333",
          "parentSpanId": null,
          "sessionId": "22222222-2222-4222-8222-222222222222",
          "kind": "generation",
          "name": "llm.generate",
          "startTime": "2026-07-17T09:00:00.050Z",
          "endTime": "2026-07-17T09:00:00.200Z",
          "durationMs": 150,
          "status": "ok",
          "attributes": {
            "kind": "generation",
            "model": "gpt-4o",
            "inputTokens": 420,
            "outputTokens": 96,
            "cacheReadTokens": 0,
            "cacheWriteTokens": 0,
            "costUsd": 0.00201,
            "prompt": null,
            "completion": null,
            "finishReason": "stop"
          }
        }
      ]
    }
  ]
}

Successful response:

{
  "accepted": 1,
  "rejected": 0,
  "duplicate": 0,
  "traceIds": ["11111111-1111-4111-8111-111111111111"]
}

Validation and limits

  • product is required and must be a string. The shared payload schema targets 1–100 characters, but the current HTTP DTO enforces only the string type; keep producers within 1–100 characters for compatibility.
  • A payload contains 1–500 batches.
  • Each batch contains one trace and at most 500 spans.
  • Trace and span names are 1–200 characters.
  • Trace, span, parent-span, session, and user IDs must be UUIDs when present.
  • agentId is a free-form string.
  • Timestamps must be ISO 8601 datetimes.
  • Duration and token values must be non-negative.

The backend validates each batch before writing it. A valid batch can be accepted even if another batch in the same request is rejected.

Duplicate behavior

Trace ID is the ingest idempotency key. If a trace ID already exists, the backend ignores the resend, increments duplicate, and does not count it as rejected. Duplicate trace IDs are not repeated in traceIds, and their spans and session totals are not applied twice.

List sessions

GET /organization/ORG_ID/observability/sessions?days=7&limit=50&status=active
Authorization: Bearer YOUR_JWT

Query parameters:

ParameterTypeDefaultConstraints
daysinteger71–90
limitinteger501–200
cursorstringnoneOpaque cursor from the previous page
statusstringnoneactive, completed, expired, or error
userIdUUIDnoneExact match
agentIdstringnoneExact match
{
  "items": [
    {
      "id": "22222222-2222-4222-8222-222222222222",
      "orgId": "44444444-4444-4444-8444-444444444444",
      "userId": null,
      "agentId": "support-agent",
      "product": "armoriq-sdk",
      "startedAt": "2026-07-17T09:00:00.000Z",
      "lastActivityAt": "2026-07-17T09:00:00.240Z",
      "traceCount": 1,
      "totalInputTokens": 420,
      "totalOutputTokens": 96,
      "totalCacheReadTokens": 0,
      "totalCacheWriteTokens": 0,
      "totalCostUsd": 0.00201,
      "status": "active"
    }
  ],
  "nextCursor": "OPAQUE_CURSOR",
  "hasMore": true
}

nextCursor is omitted on the final page.

Get a session

GET /organization/ORG_ID/observability/sessions/SESSION_ID
Authorization: Bearer YOUR_JWT

The response contains every session field plus a chronological traces array. Each trace includes the base trace record and list metrics:

{
  "id": "22222222-2222-4222-8222-222222222222",
  "orgId": "44444444-4444-4444-8444-444444444444",
  "userId": null,
  "agentId": "support-agent",
  "product": "armoriq-sdk",
  "startedAt": "2026-07-17T09:00:00.000Z",
  "lastActivityAt": "2026-07-17T09:00:00.240Z",
  "traceCount": 1,
  "totalInputTokens": 420,
  "totalOutputTokens": 96,
  "totalCacheReadTokens": 0,
  "totalCacheWriteTokens": 0,
  "totalCostUsd": 0.00201,
  "status": "active",
  "traces": [
    {
      "id": "11111111-1111-4111-8111-111111111111",
      "sessionId": "22222222-2222-4222-8222-222222222222",
      "name": "iap.plan",
      "startTime": "2026-07-17T09:00:00.000Z",
      "endTime": "2026-07-17T09:00:00.240Z",
      "durationMs": 240,
      "status": "ok",
      "userId": null,
      "agentId": "support-agent",
      "attributes": { "input": "Find the account owner" },
      "tags": ["armoriq-sdk", "allowed"],
      "totalInputTokens": 420,
      "totalOutputTokens": 96,
      "totalCostUsd": 0.00201,
      "spanCount": 1,
      "errorCount": 0,
      "inputPreview": "Find the account owner",
      "outputPreview": null
    }
  ]
}

The session-detail trace array is capped at 50,000 rows. If a session exceeds the cap, the response keeps the most recent traces and the backend logs a warning.

If the session ID does not exist in the organization, the current endpoint returns HTTP 200 with JSON null, not 404.

List traces

GET /organization/ORG_ID/observability/traces?days=7&limit=100&status=error&minDurationMs=500
Authorization: Bearer YOUR_JWT
ParameterTypeDefaultConstraints
daysinteger71–90
limitinteger501–200
cursorstringnoneOpaque cursor from the previous page
sessionIdUUIDnoneExact match
namestringnoneExact trace-name match
statusstringnoneok, error, or denied
userIdUUIDnoneExact match
agentIdstringnoneExact match
minDurationMsintegernoneMinimum duration, inclusive
hasErrorbooleannonetrue selects non-ok; false selects ok

hasError overrides status when both are supplied.

The response uses the same cursor envelope as sessions. Trace items contain the base trace fields plus totalInputTokens, totalOutputTokens, totalCostUsd, spanCount, errorCount, inputPreview, and outputPreview.

Get a trace

GET /organization/ORG_ID/observability/traces/TRACE_ID
Authorization: Bearer YOUR_JWT
{
  "id": "11111111-1111-4111-8111-111111111111",
  "sessionId": "22222222-2222-4222-8222-222222222222",
  "name": "iap.plan",
  "startTime": "2026-07-17T09:00:00.000Z",
  "endTime": "2026-07-17T09:00:00.240Z",
  "durationMs": 240,
  "status": "ok",
  "userId": null,
  "agentId": "support-agent",
  "attributes": { "input": "Find the account owner" },
  "tags": ["armoriq-sdk", "allowed"],
  "product": "armoriq-sdk",
  "source": "sdk",
  "spans": [
    {
      "id": "33333333-3333-4333-8333-333333333333",
      "parentSpanId": null,
      "sessionId": "22222222-2222-4222-8222-222222222222",
      "kind": "generation",
      "name": "llm.generate",
      "startTime": "2026-07-17T09:00:00.050Z",
      "endTime": "2026-07-17T09:00:00.200Z",
      "durationMs": 150,
      "status": "ok",
      "attributes": {
        "kind": "generation",
        "model": "gpt-4o",
        "inputTokens": 420,
        "outputTokens": 96,
        "cacheReadTokens": 0,
        "cacheWriteTokens": 0,
        "costUsd": 0.00201,
        "prompt": null,
        "completion": null,
        "finishReason": "stop"
      }
    }
  ]
}

If the trace ID does not exist in the organization, the current endpoint returns HTTP 200 with JSON null, not 404.

Get policy aggregates

GET /organization/ORG_ID/observability/policies/contacts-read?days=7
Authorization: Bearer YOUR_JWT
{
  "policyId": "contacts-read",
  "policyName": "Contacts read policy",
  "calls": 120,
  "allowed": 115,
  "denied": 5,
  "avgDurationMs": 18,
  "p50DurationMs": 14,
  "p95DurationMs": 42,
  "topMatchedRules": [
    { "ruleId": "allow-support", "count": 115 },
    { "ruleId": "deny-untrusted", "count": 5 }
  ]
}

days defaults to 7 and accepts 1–90. calls includes allow, deny, hold, and ask. The allowed subtotal counts only allow, while denied counts only literal deny; hold and ask appear in neither subtotal. Summary topFailingPolicies likewise counts literal deny decisions only.

Get the analytics summary

GET /organization/ORG_ID/observability/analytics/summary?days=7
Authorization: Bearer YOUR_JWT
{
  "period": 7,
  "totalTraces": 980,
  "totalSessions": 82,
  "totalErrors": 12,
  "totalTokens": {
    "input": 250000,
    "output": 64000,
    "cacheRead": 18000,
    "cacheWrite": 4000
  },
  "totalCostUsd": 4.812345,
  "p50DurationMs": 120,
  "p95DurationMs": 920,
  "topFailingPolicies": [
    { "policyId": "contacts-read", "count": 5 }
  ],
  "tracesPerMinute": 1.7
}

tracesPerMinute uses a trailing 60-minute window, clamped to the requested period. The other metrics use the requested days period.

Get token aggregates

GET /organization/ORG_ID/observability/tokens?days=7&groupBy=model
Authorization: Bearer YOUR_JWT

groupBy accepts model, tool, or session and defaults to model.

{
  "groupBy": "model",
  "items": [
    {
      "key": "gpt-4o",
      "inputTokens": 250000,
      "outputTokens": 64000,
      "cacheReadTokens": 18000,
      "cacheWriteTokens": 4000,
      "costUsd": 4.812345
    }
  ]
}

Model and session groups aggregate generation spans. Tool groups read generic span records and use attributes.toolName; tools without generation attributes can therefore have zero token totals.

Pagination

Session and trace lists use cursor pagination ordered by timestamp and ID. Treat nextCursor as opaque and send it unchanged in the next request. A malformed cursor returns 400 Invalid cursor.

Retention and aggregation limits

  • Default retention is 30 days.
  • OBSERVABILITY_RETENTION_DAYS changes retention on the backend.
  • A UTC retention job runs daily at 03:00 and deletes old traces; their spans are cascade-deleted. Old sessions are deleted by lastActivityAt.
  • In-memory analytics queries scan at most 50,000 rows. A result that reaches this cap is approximate and logs a backend warning.

Next: use the dashboard or troubleshoot API and shipping failures.

On this page