Skip to content
bucker

OTLP ingest

A separate surface from the Sentry-compatible envelope protocol: Bucker also accepts OpenTelemetry traces and logs over OTLP/HTTP, so a stock OTel SDK or collector can point at Bucker with nothing but an endpoint and a header — no Bucker-specific exporter required.

Metrics are accepted too (below); a signal Bucker does not store is not registered, because a route that accepted data and silently dropped it would be worse than the 404 an exporter gets.

Endpoints

POST /v1/traces
POST /v1/logs
POST /v1/metrics
POST /api/<projectId>/otlp/v1/traces
POST /api/<projectId>/otlp/v1/logs
POST /api/<projectId>/otlp/v1/metrics

The bare /v1/* set is what an unmodified OTel exporter reaches when OTEL_EXPORTER_OTLP_ENDPOINT points at Bucker. The project-scoped set is for anyone who prefers an explicit project in the URL; both sets run the same handler.

Authentication and project resolution

OTLP has no DSN-shaped credential and no project segment in its own protocol, so both are recovered from the request, in this order:

  1. The :projectId route parameter (project-scoped pair only).
  2. Authorization: DSN <full dsn> — the simplest option for a stock exporter, since it supplies the public key and the project id together.
  3. The x-bucker-project-id header, paired with a public key from X-Sentry-Auth, Authorization: Sentry sentry_key=<publicKey>, or ?sentry_key=<publicKey>.
  4. A project_id query parameter, paired with a public key the same way.

A key that does not belong to the resolved project is refused, exactly as on the envelope route. The recommended exporter configuration is:

OTEL_EXPORTER_OTLP_ENDPOINT=https://<region>.<ingest-host>
OTEL_EXPORTER_OTLP_HEADERS=Authorization=DSN <full dsn>
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf

See also Log drains for platform drains and Collector, Fluent Bit and Vector examples.

Encodings

Both OTLP/HTTP encodings are accepted, decided from Content-Type:

Content-Type Encoding
application/x-protobuf or application/protobuf protobuf
application/json, or no Content-Type at all JSON

http/protobuf is OTLP's spec default and what an SDK sends unless told otherwise, so both are first-class rather than JSON-only. An absent Content-Type is treated as JSON rather than refused: a bare curl -d @batch.json with no -H is a common first attempt, and guessing protobuf from bytes would be worse. The response is encoded in whatever was received — a protobuf request gets a protobuf response, including on partial success — because several exporters log a deserialization warning on every successful export otherwise.

Protobuf decoding is hand-written rather than pulled from a library: it produces the same OTLP/JSON object shape the JSON path validates, so a protobuf batch and a JSON batch converge on one validated object before anything downstream runs, and quota, scrubbing and outcome accounting cannot fork between the two encodings.

Compression

Content-Encoding: gzip (or x-gzip), deflate or br — the same bounded decompressor the envelope route uses, so a decompression bomb is refused as 413 (too large) rather than expanded.

Limits

Limit Value On exceeding
Request body (received or decompressed) 4 MiB 413, whole request refused
Records per request (spans + log records) 5,000 400, whole request refused
Resource wrapper objects (ResourceSpans/ResourceLogs) 512 400 (schema validation), whole request refused
Scope wrapper objects, per resource (ScopeSpans/ScopeLogs) 512 400 (schema validation), whole request refused
Scope wrapper objects, total across the request 4,096 400 (schema validation), whole request refused

The wrapper ceilings exist because a wrapper is allocated and walked whether or not it holds any records — a batch of a million empty wrappers costs the same CPU as a million empty spans, with no per-record charge to catch it. The per-resource and total scope caps are enforced identically on both encodings, so an exporter cannot be accepted on http/json and refused on http/protobuf for the same batch.

Unknown fields — either encoding — are always skipped rather than refused, because opentelemetry-proto adds fields in minor releases and an exporter one version ahead must not start failing.

Responses

Success is an empty JSON object (or a zero-byte protobuf body) unless some records were rejected on content grounds, in which case the OTLP partial-success shape is used:

{ "partialSuccess": { "rejectedSpans": 3, "errorMessage": "…" } }
{ "partialSuccess": { "rejectedLogRecords": 1, "errorMessage": "…" } }

Durable by default. /v1/traces, /v1/logs and /v1/metrics (and the platform log drains) store the decoded batch as a queue job and answer before any row is written; a worker then maps and writes it. Success is therefore the empty object even when some records will turn out to be unusable: those records are counted as invalid outcome rows (the Drops page), not returned as partialSuccess. A body that cannot be decoded, a batch over the per-request record ceiling and a quota refusal are decided before anything is stored and are still answered in the request. Self-hosters can set INGEST_OTLP_DURABLE=false to write inside the request, as before, and get the partial-success shape back.

An exporter logs a partial success and does not retry, which is correct for records refused on content grounds — retrying would only reproduce the same rejection. Authentication, rate-limit and decompression failures instead raise the ordinary HTTP error statuses (400, 401, 413, 429) described in ingest-protocol.md; those are transport failures, which an exporter should retry, and partial success is not overloaded to mean both.

Rate limits and quota

Ingested through the same auth, rate-limiting and quota machinery as the envelope route: a coarse per-IP ceiling ahead of any database lookup, then a per-key token bucket keyed on the resolved DSN key (not a separate per-IP limit — a NAT egress or a collector fronting a fleet is not an attacker). Over quota, traces and log records degrade to stratified sampling rather than an outright block, exactly as ingest does for Sentry events, and every drop is recorded against a data category (transaction for spans, log for log records) the same way.

Exceptions recorded on spans

An OpenTelemetry SDK's recordException adds an exception event to the span, carrying exception.type, exception.message and exception.stacktrace. Each distinct exception on a stored span becomes an occurrence in the same issue stream the logs path and the envelope path feed, linked to its trace, span, service, release and environment. The message and stack are scrubbed at the stored tier before anything is written, and symbolication is not attempted (OTLP carries no Debug IDs); the event says so in extra.symbolication: "not_attempted".

  • One occurrence per trace, span and exception type. A log record carrying the same exception.type on the same trace and span is not a second occurrence of a span exception that opened it within the last minute, and the other way round. The log record is still stored and links to the issue. Two log records with one triple remain two occurrences.
  • At most 100 grouped exceptions per request. Each is an error outcome at one unit. Past the cap the spans are still stored; the rest are counted as a rate_limited / error outcome with the reason otlp_span_exception_tail_capped, which the drops view words as "stored, not lost" (the log twin is otlp_exception_tail_capped). Nothing needs resending.
  • Error spans with no exception event open an issue only when the project's performance setting error_span_issues is on (PUT .../performance-settings with {"errorSpanIssues":{"enabled":true}}, off by default). They group by service, endpoint and status, so a thousand 500s on one route are one issue.
  • Re-sent batches and redelivered durable jobs do not duplicate occurrences: an occurrence's event id is derived from the project, trace, span and type.

What this document does not cover

Grouping an exception-carrying log record into an issue reuses the same ingestNormalizedEvent + essence pipeline the envelope path uses — see error-essence.md — rather than a second one for OTLP. This document is the wire layer only: encodings, auth, limits, and what a refusal looks like.

Metrics

POST /v1/metrics takes ExportMetricsServiceRequest in the same two encodings, with the same authentication, compression, rate limit and size limits as traces and logs. Gauge, sum, histogram, exponential histogram and summary points are stored one row per data point and are queried in Explore as the Metrics dataset. There is no Bucker metrics SDK and no metrics API to learn: you keep exporting with OpenTelemetry.

OTEL_EXPORTER_OTLP_ENDPOINT=https://<region>.<ingest-host>
OTEL_EXPORTER_OTLP_HEADERS=Authorization=DSN <full dsn>
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
  • Exemplars link to traces. A data point's first exemplar with a trace id becomes the row's trace_id (all of them, up to five, are kept), so a metric spike is one click from the trace that produced it. Enable exemplars in your SDK (OTEL_METRICS_EXEMPLAR_FILTER=trace_based).
  • Cardinality is capped, and the cap is reported. Each metric may have 1,000 distinct series (one per distinct set of attributes) over a rolling day of arrival. A data point belonging to a series past that is dropped and counted on the drops panel under metric_series_cap; series that already exist keep reporting. At most 200 distinct metric names per request (metric_name_cap). The fix for either is to remove the unbounded attribute before export.
  • Priced as logs. A data point weighs what a log record weighs (0.05 units) and shows on the usage page and the drops panel as its own metric category, apart from logs. There is no separate metrics meter: it is one weight in the one budget, and a histogram point costs one point whatever its bucket count. Over quota, points are sampled the way log records are.
  • PII scrubbing runs on attributes, the metric name and the unit before anything is stored, as for spans and logs.
  • Retention follows the plan's window, like the other telemetry.
  • Units come from the name when the sender sent none. A point that carries a unit keeps it, whatever its name says. A point with no unit gets one from the metric name's suffix, by the OpenTelemetry and Prometheus naming conventions: _seconds s, _milliseconds ms, _microseconds us, _nanoseconds ns, _minutes min, _hours h, _days d, _bytes By, _kilobytes kBy, _megabytes MBy, _gigabytes GBy, _bits bit, _percent %, _ratio 1, _celsius Cel, _hertz Hz, _meters m, _volts V, _amperes A, _watts W, _joules J. A Prometheus _total suffix is read past, so process_cpu_seconds_total is s, and a counter whose name is only <things>_total is 1. A name that ends in none of these keeps no unit. Unit is part of a series' identity, so a metric that gains an inferred unit starts a new series the first time it is seen with one.
  • Start times are honoured. startTimeUnixNano on a cumulative point is stored, and a change in it between two points of one series is read as a reset by the cumulative differencing, so a restarted process is not a negative rate. A start time after the point's own timestamp is ignored (the point is kept; it is never read as a reset).
  • Old points are refused, and counted. A point whose UTC day is already past its retention class window would be removed by the next sweep, so it is not stored and not billed. It is counted on the drops panel as invalid with the reason metric_point_expired; the rest of the request is stored. Any timestamp inside the window is accepted, so a backfill within it works.
  • Partial success. A point with no timestamp, no finite value, or an unreadable bucket layout is rejected and reported as rejectedDataPoints; the rest of the request is stored.
  • Alerts do not read metrics yet. A metric alert needs a rate/delta semantic per kind and temporality that Explore's per-point aggregates deliberately do not have.

Collector recipes

Bucker accepts OTLP over HTTP, with http/protobuf or http/json, and nothing else on the wire: no gRPC listener, no Prometheus remote-write endpoint, no StatsD port. Everything else reaches it through an OpenTelemetry Collector (the upstream otelcol or otelcol-contrib distribution) running next to the source. Each recipe below is a complete file in collector-recipes/, and a test checks that each parses and that every pipeline names only components it defines. The test does not run otelcol validate: run that on your own copy before you deploy it.

All three use the same exporter. BUCKER_OTLP_ENDPOINT is https://<region>.<ingest-host> with no path (the exporter appends /v1/metrics and the other signal paths), and BUCKER_DSN is the project's full DSN, sent as Authorization: DSN <dsn>. Keep the DSN in the environment, never in the file you commit.

Source Recipe
An application that exports OTLP over gRPC grpc-sidecar.yaml
A Prometheus /metrics endpoint prometheus-scrape.yaml
StatsD datagrams statsd.yaml
A Prometheus server in agent mode the two files below

gRPC through a sidecar, with batches under 4 MiB

Run the Collector as a sidecar and point the application at 127.0.0.1:4317. The batch processor counts items, not bytes, while Bucker refuses a body over 4 MiB (413) and a request over 5,000 records (400), in both cases refusing the whole request. The recipe sends at most 2,500 items and compresses with gzip, which keeps ordinary telemetry far under the byte limit; lower send_batch_max_size if your records are large (long log bodies, wide span attributes). A refused request is not retried by an exporter, so the ceiling is something to set, not to hit.

A Collector scraping Prometheus

The prometheus receiver scrapes /metrics and the same exporter forwards it. Counters arrive as cumulative monotonic sums, histograms as explicit-bucket histograms, and units are inferred from the names (_seconds, _bytes). Every distinct label set is a series, so scrape what you need.

Prometheus in agent mode, remote-writing to a Collector

Prometheus in agent mode keeps no local storage and remote-writes. Point it at a Collector with the contrib prometheusremotewrite receiver, and the Collector forwards over OTLP/HTTP:

# prometheus.yml
global:
  scrape_interval: 30s
scrape_configs:
  - job_name: app
    static_configs:
      - targets: ['127.0.0.1:9100']
remote_write:
  - url: http://127.0.0.1:19291/api/v1/write
    protobuf_message: io.prometheus.write.v2.Request

Start Prometheus 3 with --agent. The protobuf_message line is not optional: the prometheusremotewrite receiver accepts Remote-Write 2.0 only and answers a 1.0 request (what Prometheus sends by default) with 415. The receiver is still at development stability in the contrib distribution, so pin the Collector version you tested. The Collector side:

receivers:
  prometheusremotewrite:
    endpoint: 127.0.0.1:19291
processors:
  batch:
    timeout: 10s
    send_batch_size: 2000
    send_batch_max_size: 2500
exporters:
  otlphttp/bucker:
    endpoint: ${env:BUCKER_OTLP_ENDPOINT}
    encoding: proto
    compression: gzip
    headers:
      Authorization: DSN ${env:BUCKER_DSN}
service:
  pipelines:
    metrics:
      receivers: [prometheusremotewrite]
      processors: [batch]
      exporters: [otlphttp/bucker]

StatsD through the Collector

Bucker runs no StatsD server. The Collector's statsd receiver listens on UDP 8125, aggregates for aggregation_interval, and exports OTLP metrics: counters become delta sums, gauges gauges, and timers and histograms (with observer_type: histogram) exponential histograms. Datagram tags become attributes, so the cardinality advice above applies to them as well.

Every recipe states send_batch_size beside send_batch_max_size. The Collector refuses a configuration whose maximum is below the batch size, and the batch size defaults to 8192, so a recipe that set only the maximum would not start.