# The ingest wire protocol

This is the specification for the bytes a client sends to Bucker. It exists so that a third
party can implement a client without reading Bucker's source, and so that the compatibility
claim below can be checked rather than believed.

## Sentry SDK compatibility, stated plainly

**The envelope format is deliberately Sentry-envelope compatible, and an unmodified Sentry
SDK pointed at a Bucker DSN reports here — with two qualifications this section used to
leave out.** What is kept, per SDK and version, is the certified matrix in
[Sentry SDK compatibility](sentry-compat.md), produced by a conformance suite that runs the
real SDKs against a local api and worker rather than by reading this page.

The two qualifications:

1. **Give a Sentry SDK the key's `sentryDsn`, not its `dsn`.** Every Sentry SDK refuses a DSN
   whose project segment is not an integer — the JavaScript SDKs disable themselves silently,
   sentry-python raises at `init` — and a Bucker project id is not one. `sentryDsn` names the
   same project by its number; every DSN-authenticated route accepts both forms (see
   [DSNs and regions](dsn-and-regions.md)).
2. **Not every item type carries over.** Errors, traces (`transaction`, and v11's streamed
   `span` batches), logs, per-session and aggregate release health, cron check-ins and user
   feedback are kept, and so are text, log and minidump attachments (screenshots by opt-in).
   Session Replay (rrweb), profiles and other binary attachments are refused by name,
   unbilled, each with its own reason on the drops panel. Crash-free users cannot be derived
   from a server SDK's aggregate sessions, and the dashboard says so.

This is a design decision rather than a coincidence, and it is the same migration path
GlitchTip and Bugsink took. Three things make it hold, and all three are load-bearing:

- The grammar below is Sentry's newline-delimited envelope grammar.
- All three historical credential placements are accepted, so an SDK of any vintage
  authenticates.
- A 429 carries `X-Sentry-Rate-Limits` in Sentry's own advisory format, which real SDKs
  parse to back off per category rather than retry-storming.

If you are evaluating Bucker and already run a Sentry SDK, you do not need to install
anything. Point the SDK at the key's `sentryDsn` and your existing instrumentation reports
here.

## Endpoint

```
POST https://<region>.<ingest-host>/api/<projectId>/envelope/
```

The trailing slash is optional; both forms are registered. `<region>` and `<ingest-host>`
come from the DSN, which has the shape:

```
https://<publicKey>@<region>.<ingest-host>/<projectId>
```

A DSN carrying a password is refused. The public key is public by construction — it ships
in every browser bundle — and it authenticates ingest only. It is never an API credential.

## Authentication

Any one of these. They are checked in this order, and the first present wins:

| Placement | Form |
| --- | --- |
| Header | `X-Sentry-Auth: Sentry sentry_version=7, sentry_key=<publicKey>` |
| Header | `Authorization: DSN <full dsn>` or `Authorization: Sentry sentry_key=<publicKey>` |
| Query | `?sentry_key=<publicKey>` |

The query form exists because a browser cannot always set a custom header on a beacon or a
`sendBeacon` call. The key must belong to the `<projectId>` in the path; a key valid for a
different project is refused rather than silently accepted.

## Grammar

Newline-delimited JSON. One envelope header, then any number of item header and payload
pairs:

```
{"event_id":"...","sent_at":"...","dsn":"..."}    <- envelope header
{"type":"event","length":123}                     <- item header
{...payload...}                                   <- item payload
{"type":"repro_capsule"}                          <- another item header
{...payload...}
```

An item header may declare `length` in bytes. When it does, exactly that many bytes are read
as the payload and the newline that follows is consumed. When it does not, the payload runs
to the next newline. Declaring the length is what allows a payload to contain newlines.

### Item types

Understood today: `event`, `transaction`, `session`, `sessions`, `check_in`, `feedback`,
`user_report`, `attachment`, `client_report`, `replay_event`, `replay_recording`, `profile`,
`profile_chunk`, `log`, `span`, and three
Bucker extensions, `repro_capsule`, `state_patches` and `product_event`. What each one becomes:

| Item | Becomes |
| --- | --- |
| `event` | An error event, grouped into an issue. |
| `session` | Release-health data for the release in `attrs.release`. `attrs.user_agent`, when present, is read for the session's operating system, browser family and device class (release health by segment) and not stored. A session with no release is refused — release health is per release — and recorded as `invalid` with the reason `session_missing_release`. |
| `sessions` | A Sentry server SDK's request-mode release health: `{"aggregates": [{"started", "exited", "errored", "abnormal", "crashed"}], "attrs": {"release", "environment"}}`, one row per bucket. The counts join every crash-free **session** figure for that release. They never join a **user** figure: an aggregate carries no per-session identity, so crash-free users cannot be derived from it, and the API says how many sessions arrived this way (`aggregatedSessions`) so the dashboard shows "not derivable from aggregates" rather than a wrong number. Refused without a release, like `session`. A redelivered item adds nothing. |
| `check_in` | A Sentry cron check-in (`Sentry.withMonitor`, `captureCheckIn`): `check_in_id`, `monitor_slug`, `status` (`in_progress`/`ok`/`error`), `duration` in seconds, `environment`, and an optional `monitor_config` (`schedule` as `{"type":"crontab","value"}` or `{"type":"interval","value","unit"}` with unit `minute`/`hour`/`day`/`week`, `checkin_margin`, `max_runtime`, `timezone`). Recorded through the same service and auto-create policy as the HTTP check-in route: the monitor is created from `monitor_config` only when it does not exist. An unknown slug with no config is refused as `check_in_unknown_monitor`; a config the route would not create from (slug grammar, a monthly or yearly interval, the monitor limit) as `check_in_monitor_refused`. |
| `feedback`, `user_report` | User feedback (`Sentry.captureFeedback`, the feedback widget, and the pre-8.x `user_report`), stored through the same service as the HTTP feedback route — spam scoring and the injection scanner included — and linked to the issue of `associated_event_id` (or, for `user_report`, `event_id`). Stored once per feedback id however often the envelope is redelivered. At most 10 per envelope. Its own category, `feedback`, at weight 0: it uses none of your allowance, and over plan volume an envelope that carries only feedback is kept rather than sampled. |
| `product_event` | One product-analytics event: `{uuid, event, distinct_id, timestamp, properties, …}`. The accept edge understands it and meters it as the `product` category, **0.05 units an event whether the user is anonymous, identified or linked to an account** — identifying a user never changes what an event costs. The worker does not store product events yet, so an accepted one is recorded as `invalid` with the reason `unsupported_item_type` until it does. |
| `span` | Stored spans, in the same table OTLP spans land in. Two shapes share the type. The Bucker SDK's is one span per item: `trace_id` (32 hex), `span_id` (16 hex), `parent_span_id` (16 hex or null), `name`, `kind`, `start_timestamp` and `timestamp` in epoch **seconds**, `status`, `attributes`, and optional `environment` and `release`. Sentry JS v11's streamed spans (`content_type: application/vnd.sentry.items.span.v2+json`, or any payload of the form `{"items": [...]}`) are a batch: each entry has `trace_id`, `span_id`, `parent_span_id`, `name`, `status` (`ok`/`error`), `is_segment`, `start_timestamp` and `end_timestamp`, and typed `attributes` (`{"key": {"value": …, "type": …}}`) from which `sentry.environment` and `sentry.release` are read. At most 5,000 spans per item; a batch is metered as its span count. A redelivered span is not stored twice. |
| `transaction` | A Sentry SDK's trace: the root span from `contexts.trace` (named after `transaction`) and one child span per entry of `spans[]` (named after `description`, falling back to `op`), stored as spans of one trace. `op` lands in the `sentry.op` attribute, `tags` under `sentry.tag.<name>`, and any status but `ok` is an error with the status name kept as its message. Metered as one `transaction` unit per span (root plus children, at most 5,000), as OTLP meters the same rows. |
| `log` | Stored log records, one per entry of `{"items": [...]}` — each with `timestamp` in epoch seconds, `severity`/`severity_number`, `body`, `attributes`, and optional `trace_id`, `span_id`, `environment` and `release`. Sentry's log records (`content_type: application/vnd.sentry.items.log+json`, or records carrying `level` instead of `severity`) are read with `level`, typed attributes, and `sentry.environment`/`sentry.release` from the attributes. At most 5,000 records per item. |
| `repro_capsule` | Merged into the NEAREST PRECEDING `event` item of the same envelope, as that event's `repro_capsule` field — which is where both Bucker SDKs put it. It is bounded and scrubbed with the event. A capsule with no event before it, or one that does not validate, is refused on its own; the event it came with is still stored. |
| `replay_event`, `replay_recording` | One replay segment, the two items together — in the Bucker browser SDK's interaction-stream format. A Sentry SDK's Session Replay (rrweb: a `{"segment_id":N}` header line followed by the rrweb event array, compressed or not) is **refused by name** when it arrives: both items are recorded as `invalid` with the reason `unsupported_item_type:sentry_rrweb_replay`, are not billed, and the rest of the envelope is processed. Bucker's player does not render rrweb DOM snapshots. |
| `profile`, `profile_chunk` | **Refused by name** when it arrives: recorded as `invalid` with the reason `unsupported_item_type:profile` and not billed. Bucker does not offer profiling; set the SDK's profile sample rate to zero. |
| `attachment` | Stored against the event named by the item header's `event_id`, else the envelope's event, else the envelope header's `event_id` — text and logs scrubbed with the project's rules first, minidumps as sent, screenshots only on a project that opted in. Metered per started 256 KiB. **Refused by name** (`invalid`, not billed) when it is another binary (`unsupported_item_type:attachment_binary`), a credential-shaped filename (`…:attachment_secret_name`) or an un-opted screenshot (`…:attachment_screenshot`); past the per-event or org storage limit it is `invalid` as `attachment_per_event_limit` / `attachment_storage_cap`. The event it was attached to is processed normally either way. See [Sentry SDK compatibility](sentry-compat.md#attachments). |
| `client_report` | The SDK's own drop counts, recorded as it arrives. |

`state_patches` and `product_event` are accepted and not yet processed.

**An unknown item type is accepted, not rejected.** This is a guarantee, not an
implementation detail: it is what lets Bucker add item types without breaking Sentry SDK
clients, and what lets a Sentry SDK send item types Bucker does not yet process without
losing the rest of the envelope. Do not write a client that depends on an unknown type
being an error. Such an item — and one of the not-yet-processed types above — is recorded
as `invalid` with the reason `unsupported_item_type`, so it shows on the drops dashboard
instead of vanishing after a `200`.

## Compression

`Content-Encoding: gzip` (or `x-gzip`), `deflate` or `br` on the request body. Decompression is bounded:
a body that expands past the envelope ceiling is refused with a `too_large` outcome rather
than expanded. Sending compressed is recommended and is what most SDKs do by default.

## Limits

| Limit | Value | On exceeding |
| --- | --- | --- |
| Envelope body | 20 MiB | Whole envelope refused |
| Single item | 1 MiB | That item dropped, envelope still processed |
| Items per envelope | 1000 | Whole envelope refused |
| `client_report` entries | 100 | Excess entries ignored |
| `client_report` `quantity` | 100,000 | Clamped |

Individual event fields are bounded too — message and exception values at 8192 bytes, stack
frame fields at 4096, tag values at 1024, with bounded counts on frames, breadcrumbs,
exceptions, tags and context keys. **Fields are truncated rather than rejected**, because
dropping a customer's whole event because one breadcrumb was long is the worse failure. What
was truncated is recorded on the stored event under `extra.__bucker_truncations`, so a
missing tail is visible rather than mysterious.

An oversized single item is dropped and accounted for, not silently discarded — see outcomes
below.

## Responses

| Status | Meaning |
| --- | --- |
| `200` | Accepted. Body is `{"id": "<eventId>"}`. |
| `400` | The envelope did not parse, or exceeded a hard limit. |
| `401` | The key is missing, malformed, or not valid for this project. |
| `413` | The body exceeded the envelope ceiling, before or after decompression. |
| `429` | Rate limited or over quota. See the header below. |

Acceptance means the envelope is durably enqueued, not that it has been processed. Grouping,
scrubbing and symbolication happen afterwards, off the request path.

### Rate limit headers

A 429 carries Sentry's advisory format:

```
X-Sentry-Rate-Limits: <retry_after_seconds>:<categories>:<scope>
```

Categories are the data categories below. A client that parses this backs off for the named
categories only, which is why the header is worth honouring rather than treating a 429 as
uniform.

## Data categories and drop accounting

Every item maps to a category — `error`, `transaction`, `session`, `attachment`, `replay`,
`profile`, `log`, `product`, `metric`, `source_row`, `feedback`, `internal` — and **every drop is
recorded against one**. The recorded outcomes are `rate_limited`, `quota_exceeded`, `too_large`,
`invalid` and `sampled`.

A category is also a **weight in one unit budget**, never a meter of its own: an error is 1 unit,
a log record, a metric data point or a product event 0.05, a synced source row 0.01, and
sessions, feedback and Bucker's own bookkeeping 0 (the full table, with what happens to each
category over budget, is in [billing-and-usage.md](billing-and-usage.md#what-a-unit-is)).
`metric` has no envelope item: OpenTelemetry data points are counted in it, apart from `log`,
on `/v1/metrics`. `source_row` has none either; sources are pulled by the server. A client report
may name `product`, `metric` and `feedback`; an unknown category name is counted as `internal`.

This matters to a client author: silent data loss is the complaint this category of product
earns most often, so a drop here is always attributable. If you send something and it does
not appear, there is a row saying which category it belonged to and why it went. Ask for it
rather than guessing.

## Warnings before a cap, and spikes

Owners and billing members of an organization get an email — and the organization's
org-wide notification channels a message — at **80%, 90% and 100%** of the plan's monthly
event volume and of the AI-lane budget, each once per billing period. Past 100% nothing
extra is billed: ingest degrades to stratified sampling, and the notice says so. An hour
whose volume is far above a project's own 7-day hourly baseline (at least 500 events, three
times the median and well outside its usual spread) is reported once per project per day.
Each person can turn these off under **Settings → Notifications**.

Over quota, the accept edge does not keep a uniform slice. Before any fingerprint exists it
admits the first few envelopes per minute of each **(project, DSN key, item type, release)**
— the release read from the envelope header's `trace.release` — and samples the rest on the
envelope id; the worker then keeps every new fingerprint and thins repeats. A new release
arriving in the middle of a storm on an old one is therefore still seen. Admitted envelopes
are recorded as `sampled`, refused ones as `quota_exceeded` with reason
`over_quota_sampling`, so the drops panel accounts for both.

Two categories are sampled differently. A **product event** is admitted under its **event name
and a bucket of 64 subject hashes** instead of the release, and the slice past that floor is
cut on the **subject** (`distinct_id`), not the envelope id: a person the sampler keeps is kept
on every event, so a kept person's funnel is whole. An envelope that carries several people is
kept or refused as a unit on its first event's person. An envelope that carries **only
feedback** is not sampled at all: an answer someone typed is degraded around, never sampled
away. A project at its hard ceiling still receives a `429` for it, as for everything else: a
refusal the SDK hears and counts in its client report, never a silent `200`.

## Inbound filters

A project can refuse known noise server-side, for every SDK — builds already deployed and
Sentry SDKs included — under **Settings → Projects → Inbound filters**: browser
extensions, localhost, web crawlers, legacy browsers, IP addresses or CIDR ranges, release
globs and error-message globs. Filtered events are never stored and never billed.

- **At the accept edge**, an envelope whose request matches a filtered IP, or whose
  User-Agent is a crawler or a legacy browser, is answered `200` with
  `X-Bucker-Ingest-Disposition: filtered` before its body is read: an SDK must not retry
  a deliberate drop. It is recorded as `invalid` with reason `inbound_filter.<filter>`.
  The filters are resolved with the DSN key and cached with it, so a change takes up to a
  few seconds to reach every api machine.
- **In the worker**, every filter is judged against the event itself — its request
  headers and URL, the user's IP, its frames, release and message — **after source maps
  are applied**, so a message pattern meets the readable message rather than the minified
  one. It is recorded as `invalid` with reason `event_filter.<filter>`, and because the
  edge had already counted the envelope accepted, that volume is taken back out of
  billable usage and quota.

The drops panel names the filter in both cases.

**Every DSN door, not only the envelope.** The same request-level refusal runs on the
OTLP endpoints (`/v1/traces`, `/v1/logs`, `/v1/metrics` and their `/api/:projectId/otlp/`
spellings), on `/api/:projectId/agent/v1/traces` and on `/api/:projectId/feedback`, with
the same `X-Bucker-Ingest-Disposition: filtered` header and the same outcome row. Each
door answers its own success shape, so nothing retries: OTLP gets a full-success body
(`{}`, or the zero-byte protobuf response), agent traces an empty result, feedback the
`201` a kept report gets. An OTLP log record carrying an exception is also judged by the
event-level filters before it is stored or grouped; that door is synchronous, so the drop
is recorded under `inbound_filter.<filter>` and was never billed. Monitor check-ins and
log drains are not filtered: they come from cron hosts and platform egress, not browsers.

## Client reports

An SDK reports its own drops with a `client_report` item, so events discarded before they
reached the network are still counted. Entries are capped at 100 per envelope and each
`quantity` is clamped; a client that exceeds either is not an error, but the excess is not
counted. Client reports are metered like any other category.

## OpenTelemetry

Separate surface, separate document: OTLP traces and logs are accepted at `/v1/traces` and
`/v1/logs`. See [otlp.md](otlp.md) for encodings, headers and the environment variables a
stock OTel exporter needs.

## What this document does not cover

The payload schema of an `event` — the fields inside it — is Sentry's event schema, and this
document does not restate it. What is written here is the envelope layer: framing, auth,
limits, and what happens when something is refused. Where the two disagree in practice, the
envelope layer is the one Bucker guarantees.
