# Sampling rules

Choose what share of a project's spans Bucker stores, on the server, without changing any
SDK. This page covers span sampling rules: how they behave, how a person or an agent
changes them, and what holds whoever asks.

Log pipeline rules share the same table and the same preview engine and are described in
the logs documentation.

## What a rule is

A rule says "of the spans matching these filters, keep this share". The filters are the
Explore filters (`resource.service.name`, `span.name`, any attribute), ANDed, and the rate
runs from `0` (drop every match) to `1` (keep every match). The first rule that matches a
span decides; a span no rule matches is kept.

A rule is in one of three states:

| State | What it does |
| --- | --- |
| **shadow** | Evaluated beside the live rules and counts what it _would_ have kept and dropped (`shadowKept`, `shadowDropped`). It **drops nothing**. |
| **active** | Applies: matching traces are kept at the rate. |
| **disabled** | Not evaluated. |

A shadow rule is stored switched off for an older image (`enabled: false`), so a server that
predates the status column can never apply it by mistake.

## What is never sampled

The sampler keeps these whole, whatever a rule says, and a rule that **matches only** these is
refused as a no-op rather than approved and then ignored:

- error spans, spans carrying an `exception.*` attribute, and every other span of their traces;
- traces whose root returned a 5xx;
- traces linked to an open issue's occurrence;
- APM-19's sampling-budget floor (reserved: nothing registers it yet).

The promise is that the cross-batch gap is **reduced, not closed**. The floor decides per
request from what the request and a bounded lookback of recent error traces say. A trace
whose error arrives in a later request than its other spans can have been decided before
the error was known. The preview reports that as `crossBatchExposure`.

## Previewing a rule

`POST /orgs/{org}/projects/{project}/sampling-rules/preview` (and the MCP tool
`sampling_rules_preview`) replays a rule through the sampler over the last 24 hours of the
project's stored spans (at most 168). It answers:

- the rows the rule matches and the rows it would drop and keep (counted, not extrapolated);
- the share of the project's volume, the meter units and dollars saved, and the days of the
  monthly allowance gained;
- the alert rules, saved views and dashboards whose match sets change, named, with the
  number of evaluations that would have changed;
- the never-drop floor: a rule that would drop an error span is refused with a count and no
  digest, and says how to narrow it (`span.status neq error`).

An accepted preview returns a **digest**, valid for 30 minutes and bound to the project. A
proposal cites it.

Previewing needs `event:read` on the project, because it reads stored spans.

## Changing rules

### In settings

A person with project write changes rules in Settings, Projects, Sampling rules, or over
`POST|PATCH|DELETE .../sampling-rules`. A rule made there is `active` (as it always was) unless
the request says `status: "shadow"`, may carry `expiresInDays`, and every change to a governed
field bumps the rule's `version` and stores the revision it replaced.

Deleting a rule is a human act in settings. No tool can ask for it.

### As an agent

Two governed tools, both held until the SG-059 review (they need `apm:write`):

- `sampling_rules_create` proposes a new rule. It lands in **shadow**, so the act is derived
  ELEVATED: it changes no data.
- `sampling_rules_update` changes a rule's rate, filters, position or renewal, or moves a
  **shadow** rule to **active**. The engine derives the act from the diff, never from an
  argument. A change to a shadow rule is ELEVATED. A change to a live rule, and the move to
  active, are SENSITIVE and ask a person. A rate that _rises_ keeps more data and costs more, so
  it asks too.

Each write must cite the `previewDigest` of a preview of the rule _as it would be_, and repeat
the preview's arguments (the reason included): they are the plan. The figures the plan carries
are loaded from the stored preview, never from the caller.

A previous revision is restored by `operation_revert` and by the sweeper at expiry, from the
revision the rule itself stored. It is not a tool.

## The caps

Computed on the server and refused **before any approval opens**, each naming the cap:

| Cap | Limit |
| --- | --- |
| Services changed by one proposal | at most 3 (a filter set that names none changes every service) |
| Share of the project's span volume changed by one proposal | at most 25 % |
| A service's keep rate within 24 hours | falls at most 4x |
| Approved changes to a service's live sampling in a day | at most 2 |
| Shadow rules in a project | at most 5 |
| Shadow before a move to active | at least 72 hours of tallies |
| Tallies against the preview the move cites | within 25 %, or preview the rule again |

They are constants, not organisation settings: a cap an agent's own organisation could widen
would be a cap in name only.

## Expiry

Every rule an agent makes carries `expiresAt`: 7 days, renewable (`renew: true`) up to 30 in
all. Expiry is judged **on read**, so a stopped worker never leaves a temporary rule dropping
data. The operations sweeper's end-of-TTL `restore` takes the rule out of force, or back to the
revision it replaced, and tells the person who approved it (`window_ended`). A person adopts a
rule in settings (clear its expiry) or lets it end.

## Verification

After a live change, once its 60-minute window has closed, the verification compares what the
rule dropped of the spans it matched **since the apply** (its own live tallies, measured from the
base the apply recorded) with the dropped share the cited preview stated. Within 10 % of the
preview, or within the sampling noise of the spans seen, passes; 10 to 25 % is recorded as drift
and asks nobody; above 25 % opens the ask "keep or revert?" and sends the `receipt_failed`
notice. A loss of the protected classes (error spans and error traces, the hour after the apply
against the hour before) does the same whatever the effect did, when the fall is larger than both
5 % and the hour-to-hour noise of the counts (at least 20 before, and more than three standard
deviations). Fewer than 100 spans matched since the apply is `not_enough_data`; a rule another
change owns since (a renewal, a person's edit in settings) is `superseded` and nobody is asked.
The check never reverts by itself.

## Tables and fields

`sampling_rules` carries `status`, `kind`, `expiresAt`, `proposedByPrincipalId`,
`approvalRequestId`, `previousRevision`, `previewDigest`, `shadowKept`, `shadowDropped`,
`operationId` and `version`. `previousRevision` holds the definition the last governed change
replaced (null for a create) and the start of the rule's shadow clock; it is written by every
door and never recomputed, because it is the inverse. `sampling_rule_previews` holds the
digests.
