Skip to content
bucker

Where is my data? (data_gap_explain)

An empty answer is ambiguous: "no errors" and "no data" look alike. Bucker answers the question in two ways. A read that comes back empty states why in evidence.emptyReason (no_data_in_window, filtered_out, not_ingested, retention_expired, scope_hidden, not_enough_data). When that is not enough, data_gap_explain walks the pipeline for one project, one signal and one window, and names the first stage that explains the absence.

Asking

data_gap_explain (MCP, read, project:read; platform toolset; 700 tokens; light):

Argument Meaning
org, project The project.
signal error, transaction (spans), log, metric or product.
windowHours / from / to The window. Default the last 24 hours; at most 400 days.
source A connected source the signal rides (for example vercel): its connection is checked.
environment, release, service Filters, where the signal has the column. A filter a signal cannot honour is a 400, never ignored.

The answer

status is data_present (nothing is missing), gap (a stage explains it) or undetermined (reason: unknown). reason comes from a closed list:

Reason Stage Meaning
no_events sdk or the end of the walk Nothing of this signal was ever sent, refused or queued.
filtered_out rows Rows exist in the window, none under the filters.
sampled_out sampling A rule that keeps none of the signal, or a ledger that recorded only sampling. The stored rate is in the evidence.
dropped_at_ingest edge, analytics_gate Refused at the door, by reason, with the drops panel's sentence.
retention_expired retention The window is past the hot window and the cold tier (the plan ceilings are in the evidence).
in_cold_archive retention The window is past the hot window and the cold archive holds it.
source_not_connected connection No active connection for the source, or one whose coverage says the signal is not served.
sdk_silent sdk An SDK sent before and nothing arrived in the window.
clock_skew_default_partition partition Rows with timestamps far outside the window sit in the default partition.
consent_withheld consent Events refused for want of consent.
unknown The walk cannot say. missing names what would let it decide.

The walk order is analytics_gate, rows, retention, connection, sdk, consent, sampling, edge, partition, worker_queue. stages lists every stage that ran with its verdict (pass, fail, unknown, not_applicable) and a sentence.

nextCalls names what would close the gap: the tool, its token price, its scan class and the decision the caller would get (allow for a read, ask for a governed change, human_only for an act only a person performs, such as widening a sample rate or connecting a source).

The rules

  • A stage that is not built reads unknown, never a guess. Product events have no row store yet; a connection with no coverage entry for the signal is unknown; a stage that cannot be examined inside the time budget is unknown. no_events is claimed only when the rows stage looked, the door accepted nothing of the signal, and no stage was left unknown.
  • A window that cannot be looked into halts the walk. Past the hot window, the data may be where the ingest-side stages cannot see; the walk answers unknown rather than claim nothing was sent.
  • Accepted is not dropped. A window with items accepted and none stored is unknown (or the queue's figures, when envelopes are still queued), never dropped_at_ingest. A repeat or an item stored with less work is not a loss.
  • Without event:read, the stages that read telemetry rows (or counts of them) answer unknown and name the scope. The accept ledger's counts and the project's configuration, which ingest_health_get already shows to project:read, still explain what they can. No "hidden by scope" count exists.
  • Consent counts respect the cohort floor. consent_withheld carries a count only at or above the project's floor (default 5) and only for a caller who may read rows; otherwise count is null and countWithheld says why.
  • Product analytics stay behind the PA-01 gate. A shut gate stops the walk first and no end-user figure or consent count is read.
  • Untrusted strings are fenced: ledger reasons, SDK names, the source name and a coverage reason arrive sanitized and are listed in trust.untrustedFields with their F-16 class. Every sentence is Bucker's.

Proving each reason

api/test/ingest/fault-injection/ injects each failure into a fresh project (a drop at the edge, a rule that keeps nothing, a window past retention, a cold archive, a client clock two years ahead landing in the default partition, consent withheld, a source with no connection, an SDK that went quiet) and asserts the reason and the stage. A new reason adds an injector in harness.ts and a row in reasons.test.ts.