Deploy an OpenTelemetry Collector

This guide is for operators and platform engineers. You get working Collector configurations that receive traces from ArmorIQ SDKs and forward them with batching, retry, size limits, and TLS. The ArmorIQ API key comes from an environment variable. No file holds a secret.

An OpenTelemetry Collector is a program that you run. It receives trace data, changes it, and sends it on to one or more destinations. A span is one timed step in an agent run, and a trace is all the spans of one run.

When to use a Collector

Use a Collector when you need one of these:

• Send traces to ArmorIQ and to another tool. An SDK has one export choice. For a second destination, the SDK sends to the Collector and the Collector sends to both.
• Use transport features the TypeScript SDK does not have. The TypeScript SDK exporter does not support mTLS, custom CA files, HTTP proxy settings, gRPC, or compression. Set these on the Collector.
• Keep credentials out of application processes. The Collector holds the ArmorIQ API key and the keys for other tools.
• Control memory and queues in one place.

If you only need ArmorIQ dashboards, do not use a Collector. Keep the default armoriq export. See Choose who exports your traces.

How the data flows

1. Set the SDK to exporter=external with the Collector as the endpoint.
2. The Collector receives spans on its OTLP HTTP port.
3. The Collector limits memory, groups spans into batches, and sends each batch to every destination.

What ArmorIQ accepts

Set your Collector inside these limits. ArmorIQ rejects a request that is outside them.

ArmorIQ answers with these codes:

A retry is safe. ArmorIQ stores each span one time, even when a batch arrives twice.

Configuration for ArmorIQ only

Save this as otelcol.yaml. The file holds no secret. The Collector reads ARMORIQ_API_KEY from its environment.

receivers:
  otlp:
    protocols:
      http:
        endpoint: 0.0.0.0:4318
        # Turn on TLS for traffic from your applications.
        tls:
          cert_file: /etc/otelcol/tls/server.crt
          key_file: /etc/otelcol/tls/server.key

processors:
  # Put memory_limiter first. It refuses data before the Collector runs out of memory.
  memory_limiter:
    check_interval: 1s
    limit_mib: 512
    spike_limit_mib: 128
  # Keep every batch inside the ArmorIQ limits.
  batch:
    send_batch_size: 256
    send_batch_max_size: 500
    timeout: 5s

exporters:
  otlphttp/armoriq:
    endpoint: https://api.armoriq.ai # the exporter adds /v1/traces
    headers:
      x-api-key: ${env:ARMORIQ_API_KEY}
    compression: gzip
    timeout: 15s
    tls:
      insecure: false
      min_version: "1.2"
    retry_on_failure:
      enabled: true
      initial_interval: 1s
      max_interval: 30s
      max_elapsed_time: 300s
    sending_queue:
      enabled: true
      num_consumers: 4
      queue_size: 1000

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [otlphttp/armoriq]

What each part does:

• Batching. batch sends a group of spans after 256 spans or 5 seconds, whichever comes first. send_batch_max_size: 500 splits a larger group. This keeps each request under the ArmorIQ limit of 500 spans for each scope. The batch processor default is 8,192 spans, which is too large for ArmorIQ.
• Retry. retry_on_failure retries a failed send with a growing wait, from 1 second up to 30 seconds. It stops after 5 minutes. Then the Collector drops that batch.
• Bounds. memory_limiter caps memory use. sending_queue.queue_size caps how many batches wait for a destination that is slow or down. When the queue is full, the Collector drops new data. It does not use unlimited memory.
• TLS. The https:// endpoint uses TLS, and insecure: false keeps certificate checks on. The receiver tls block protects the link from your applications.
• API key. ${env:ARMORIQ_API_KEY} reads the key from the environment when the Collector starts. If the variable is not set, the key is empty and ArmorIQ answers 401.
• Protocol. The exporter default is protobuf with gzip. Do not set encoding: json. ArmorIQ rejects JSON.

Configuration for ArmorIQ and a second tool

Use one pipeline for each destination. If one destination is slow, the other does not wait.

receivers:
  otlp:
    protocols:
      http:
        endpoint: 0.0.0.0:4318
        tls:
          cert_file: /etc/otelcol/tls/server.crt
          key_file: /etc/otelcol/tls/server.key

processors:
  memory_limiter:
    check_interval: 1s
    limit_mib: 512
    spike_limit_mib: 128
  batch/armoriq:
    send_batch_size: 256
    send_batch_max_size: 500
    timeout: 5s
  batch/vendor:
    send_batch_size: 512
    send_batch_max_size: 1024
    timeout: 5s

exporters:
  otlphttp/armoriq:
    endpoint: https://api.armoriq.ai
    headers:
      x-api-key: ${env:ARMORIQ_API_KEY}
    timeout: 15s
    tls:
      insecure: false
    retry_on_failure:
      enabled: true
      initial_interval: 1s
      max_interval: 30s
      max_elapsed_time: 300s
    sending_queue:
      enabled: true
      queue_size: 1000

  otlphttp/vendor:
    # The full traces URL from the vendor's OTLP documentation.
    traces_endpoint: ${env:VENDOR_OTLP_TRACES_ENDPOINT}
    headers:
      authorization: ${env:VENDOR_OTLP_AUTHORIZATION}
    timeout: 15s
    tls:
      insecure: false
    retry_on_failure:
      enabled: true
      max_elapsed_time: 300s
    sending_queue:
      enabled: true
      queue_size: 1000

service:
  pipelines:
    traces/armoriq:
      receivers: [otlp]
      processors: [memory_limiter, batch/armoriq]
      exporters: [otlphttp/armoriq]
    traces/vendor:
      receivers: [otlp]
      processors: [memory_limiter, batch/vendor]
      exporters: [otlphttp/vendor]

The second tool has its own limits. Check its OTLP documentation for the maximum batch size.

Keep content away from the second tool

The SDK sends prompt and tool content to a Collector only when your organization allows external content. See What data is captured. You can also remove content in the Collector before it reaches a tool. The attributes processor is in the Collector contrib distribution:

processors:
  attributes/no-content:
    actions:
      - key: gen_ai.input.messages
        action: delete
      - key: gen_ai.output.messages
        action: delete
      - key: gen_ai.tool.call.arguments
        action: delete
      - key: gen_ai.tool.call.result
        action: delete

Add attributes/no-content to the traces/vendor pipeline, after memory_limiter and before the batch processor.

Run the Collector

Pass the key from your secret store into the environment. Do not write it in a file or a command history.

docker run --rm \
  -e ARMORIQ_API_KEY \
  -v "$PWD/otelcol.yaml:/etc/otelcol/config.yaml:ro" \
  -v "$PWD/tls:/etc/otelcol/tls:ro" \
  -p 4318:4318 \
  otel/opentelemetry-collector-contrib:<version> \
  --config /etc/otelcol/config.yaml

Replace <version> with a release tag that you have tested. In Kubernetes, put the key in a Secret and map it to the environment variable with secretKeyRef.

Point the SDK at the Collector

Set the SDK to external and give it the Collector address.

export ARMORIQ_OTEL_EXPORTER=external
export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=https://collector.example.com:4318/v1/traces

• The SDK accepts http: only for localhost, 127.0.0.1, and ::1. For any other host, use https:.
• A good pattern is a Collector next to each application, for example as a sidecar. The SDK sends to http://localhost:4318/v1/traces on the same machine. The Collector then sends over TLS to ArmorIQ. This also works for the TypeScript SDK, which has no setting for a custom certificate authority.
• If the Collector needs a header, set OTEL_EXPORTER_OTLP_TRACES_HEADERS. See Choose who exports your traces.

Check that it works

1. Run one agent request.
2. Look at the Collector logs. There must be no export errors for otlphttp/armoriq.
3. In ArmorIQ, open Settings > Observability and read Usage and pipeline health. The count of accepted spans must rise. See Organization controls.

Next steps

• Organization controls
• Troubleshoot missing traces
• Backend API reference