Observability

Organization Controls

Set what ArmorIQ collects for your organization, limit it by product, start temporary debug capture, watch quota and pipeline health, and set alerts.

This page is for organization administrators and security teams. You learn what you can control on the observability settings page, what each control changes, who is allowed to change it, and what the Off setting does not do.

Open Settings > Observability in the dashboard. The page is /dashboard/settings/observability, and its title is Observability data collection. The settings apply to every API key and SDK session in the active organization.

Capture mode

The Data collection card sets the capture mode. Save it with Save capture setting.

ModeWhat the page says it does
OffStops new observability collection.
Metadata (recommended)Captures topology, policy decisions, status, latency, token use, cost, and safe error types.
EnhancedCaptures only organization-approved, redacted and bounded input and output summaries.
DebugTemporary detailed capture. It still uses redaction and bounds, and must end within 24 hours.

The page also states: "Secrets and credentials are always removed at every capture mode. This cannot be turned off." For the full list of fields, see What data is captured.

An organization that has never saved a policy uses metadata, with content capture off and external content capture off.

Content capture

When you choose Enhanced or Debug, two check boxes appear:

  1. Enable redacted content capture for this organization. You must turn this on to save Enhanced or Debug. It permits bounded, redacted summaries only. It does not permit secrets, credentials, or authentication headers.
  2. Allow redacted content to be sent to an external observability provider. This lets an SDK that exports to your own destination include redacted content. It needs a separate permission. The page warns that third-party providers have their own access controls, and ArmorIQ membership controls do not apply after export.

Debug capture

Debug is a time-limited lease.

  • You set Debug capture ends to a future time that is no more than 24 hours from now.
  • You cannot make an active debug window longer. While it is active, the other modes are locked.
  • When the time ends, the effective mode returns to metadata.
  • Starting or ending debug needs its own permission. Without it, the Debug choice is locked and shows "Needs the debug permission".
  • ArmorIQ records the start and the end of debug in the change history.

When two people save at once

Every save sends the policy revision that you loaded. If someone else saved first, ArmorIQ rejects your save. The page shows "Someone else changed this policy. The latest version is loaded; review and save again." Review the new values and save again.

Scope overrides

The Effective policy section shows what each scope collects after the organization setting is applied. Its columns are Scope, ID, Set mode, Effective mode, and Note. The first row is the organization.

You can add a scope override for one product. The Scope overrides card lists the products, and you pick a capture mode for the one you choose.

  • A scope can only lower collection. A scope can never collect more than the organization allows. A mode above the organization mode is locked.
  • A scope cannot turn on collection that the organization turned off.
  • A local SDK setting follows the same rule. It can only lower collection.
  • Only product scopes are available today. The API key identifies only the organization and the product, so ArmorIQ cannot yet tell which environment or agent sent a span. New environment and agent scopes are refused. Older environment or agent rows still show, marked Not enforced yet. They do not change what ArmorIQ collects, and you can remove them.
  • Remove a scope to return collection for that scope to the organization mode. ArmorIQ asks you to confirm.

Usage and pipeline health

The Usage and pipeline health card shows how much your organization uses and whether telemetry arrives. You can filter by product.

Monthly quota

Two cards show Spans and Storage as "used of limit" for the current month (a calendar month in UTC). Used counts both committed and reserved capacity.

BadgeMeaning
Approaching limitUse is at 80 percent of the limit, or more
Limit reachedNew telemetry is rejected
Quota is 0, new telemetry is rejectedThe limit is set to zero

When a quota is full, ArmorIQ answers an SDK or Collector with 429 and Retry-After: 60. A retry is safe.

Ingest pipeline

The pipeline status looks at the last 24 hours.

StatusWhen it appears
HealthyTelemetry arrives and nothing is rejected or slow
DelayedThe slowest ingest request took more than 30 seconds
RejectingAny span was rejected, any request failed, or any span failed mapping
UnavailableIngest is switched off by an operator
Status unknownThe page could not load the data
No telemetry in the last 24hNo requests arrived

The counters are: accepted spans, rejected spans, duplicate spans, mapping-failure spans, failed requests, average ingest handling time, and maximum ingest handling time. The page notes that dropped telemetry is the sum of rejected spans and mapping-failure spans.

For help with a bad status, see Troubleshooting.

Change history

Change history lists recent changes to the settings, newest first. Columns are When, Who, Change, and Detail. It records:

  • changes to the data collection setting, and the start and end of debug capture;
  • legal hold turned on or off, and external content export allowed or blocked;
  • scope overrides saved or removed;
  • alert rules created, changed, or turned off;
  • trace export requested, completed, downloaded, expired, or failed;
  • data deletion completed;
  • SDK policy lease issued (hidden by default in the API).

Only people who can view the settings see this section.

Alert rules

Alert rules check your telemetry on a schedule. A rule opens a firing when its condition is met, and resolves it when the condition clears. ArmorIQ checks the rules every 5 minutes. Open Recent firings on a rule to see its history. Each rule has a severity of Info, Warning, or Critical.

ConditionWhat it measures over the windowUnit
Policy denial rateShare of policy decisions that deny or blockPercent
Tool error rateShare of tool, command, and MCP spans that end in errorPercent
Model error rateShare of model spans that end in errorPercent
p95 latencyThe 95th percentile duration of tool, command, MCP, and model spansMilliseconds
Token costTotal model cost over whole UTC hoursUSD
Token countTotal tokens over whole UTC hoursTokens
Retry rateShare of operations that were retriedPercent
Timeout rateShare of operations that timed outPercent
Exporter ingest rejection rateThe higher of the span rejection rate and the failed request ratePercent
Missing telemetryAny product that sent data before now sends fewer requests than the thresholdRequests

The window is 1 to 1,440 minutes. For the two budget rules, the window is 60 to 1,440 minutes. A percent threshold is at most 100, and the missing-telemetry threshold is at least 1. You turn a rule off with Disable. The rule stays in the list, and you can enable it again. You need the alert manage permission to change a rule.

Delete data and export

Delete data permanently removes specific traces and their spans from ArmorIQ. You enter trace IDs (UUID version 4), then confirm by typing DELETE. Deletion is blocked while a legal hold is active. An export that includes a deleted trace is invalidated. Deletion removes data from ArmorIQ only. It does not remove copies that you exported to another system.

Export traces is on the Observability overview page, not on the settings page. It starts an encrypted export of your most recent traces (1 to 100) as a .jsonl file. At most two exports can run for an organization at the same time.

Permissions

Each control has its own permission. The dashboard hides or locks a control when you lack the permission.

What you want to doPermission needed
See metadata, effective policy, usage, and pipeline healthObservability metadata read
See captured content in a traceObservability content read
Change the capture mode and scope overridesObservability settings manage (full)
See change historyObservability settings (view or full)
Start, change, or end debug captureObservability debug manage
Allow content to go to an external providerObservability external export manage
See alert rulesObservability alert read
Create, change, enable, or disable alert rulesObservability alert manage
Export tracesObservability export create, and content read
Delete tracesObservability delete execute
Manage a legal holdObservability legal hold manage

By default, the organization admin role has every one of these permissions. Other default roles have fewer. For example, one default role can start debug capture but cannot change the settings. A person without the content read permission sees a "Permission required" page when they open a trace. They still see metadata in the trace and session lists.

What the Off setting does and does not do

Setting the capture mode to Off controls ArmorIQ collection only. It does not remove data that has already left, and it cannot take back a third-party export.

Off does this:

  • ArmorIQ stores no new telemetry for your organization. If a request arrives, ArmorIQ answers with success and writes nothing.
  • SDKs read the organization policy. While it says Off, they create no ArmorIQ spans, for every export choice. This starts when the SDK next reads its policy. The SDK reads the policy again at least every 5 minutes, and sooner when a policy check from ArmorIQ tells it that the policy changed. For an SDK-owned pipeline, the SDK also checks the policy again just before spans leave the process, so spans that are still in the queue do not ship.
  • Policy enforcement keeps running. A decision to allow, hold, or block does not change.

Off does not do this:

  • It cannot revoke a third-party export. After a span goes to your Collector, your own provider, or another tool, ArmorIQ cannot take it back. That destination has its own access rules. ArmorIQ membership controls do not apply there. This is also true if you later turn off external content capture. The new setting stops future exports only.
  • It does not control tracing that is not from ArmorIQ. Spans that your application or another tool creates, such as OpenInference spans or your own OpenTelemetry code, are not ArmorIQ spans. ArmorIQ does not change, flush, or stop your provider. It only stops its own spans.
  • It does not delete stored data. Use Delete data for that.
  • It does not turn off policy enforcement. The page says this when an operator switch is off: "Turning a telemetry switch off does not turn policy enforcement off. Organization and scope limits still apply."

If you must be sure that no ArmorIQ data reaches a destination, remove the external or provider export in your code or Collector, and keep the capture mode at Off.

Operator switches (self-hosted backends)

Operators of an ArmorIQ backend can switch parts of observability off with environment variables. All of them are on by default. A switch accepts only true or false. Any other value stops the backend at start-up, so a typo cannot change collection. A switch can only lower collection. It never turns on a mode and never skips redaction.

VariableWhen set to false
OBSERVABILITY_OTLP_INGEST_ENABLEDThe OTLP route still checks the request, but it stores nothing and returns success. The effective mode shows as Off.
OBSERVABILITY_ENHANCED_CONTENT_ENABLEDEnhanced capture is lowered to metadata.
OBSERVABILITY_DEBUG_CAPTURE_ENABLEDDebug capture is lowered to metadata.
OBSERVABILITY_LEASE_DELIVERY_ENABLEDThe SDK policy route answers 503.
OBSERVABILITY_DASHBOARD_READS_ENABLEDThe organization observability routes answer 503. This includes the routes that change settings.

Two more variables help with recovery:

  • OTEL_CANONICAL_WRITER_ENABLED=false stops the backend from writing new telemetry. Spans are reported as rejected. Stored data is not deleted or changed, and no other path takes over.
  • OBSERVABILITY_INGEST_TRANSACTION_TIMEOUT_MS sets the ingest write time limit (default 10,000 ms, kept between 1,000 and 30,000).

Next steps

On this page