Data quality over your exports
A warehouse sink ships your telemetry to a place you own. A stream that quietly stops, thins out, changes shape or writes a row twice is worse than one that fails loudly, so Bucker watches its own export streams with five standing checks. They cover Bucker's own export streams only: error essences, issue rollups, release-health rollups and ingest outcomes. Monitors on your own database tables are a separate feature (they need a connected database, which this release does not have), and the tools below say so rather than guess.
The five checks
Each check belongs to one sink and one stream. Lower thresholds are always stricter.
| Check | What it judges | Threshold (default) | States |
|---|---|---|---|
freshness |
How old the oldest row still waiting behind the export cursor is, against the export interval (five minutes). | Intervals of backlog before it is stale (2). Failing at three times that (6). | healthy, warning (stale), failing, not_run before the first export attempt. |
volume |
Rows exported in the last complete hour against the learned median and spread for that hour of the week. | Robust z (6). Failing at twice that. | not_run until four weeks of history exist: never a verdict before. |
schema_version |
Every record matches the stream's declared, versioned record schema. | none | healthy or failing. |
uniqueness |
(recordId, watermark) appears once per stream, every batch starts strictly after the cursor it was read from, and the cursor never moves back. |
none | healthy or failing. |
completeness |
Rows exported against rows eligible between the two cursors of a batch (one count query per batch). | Share of missing rows (0.01). | healthy or failing. |
A check that could not run is erroring (and says basis: evaluation_failed); a check never reads
healthy because it failed to look.
The schema tag
Every object an export writes carries bucker-schema: <STREAM>@<n> on every JSON line, for example
ISSUE_ROLLUPS@1. The version is raised whenever the shape of a record changes, in the same change
that changes it. A record with a field the declared schema does not name fails schema_version: that
is how a silent change of contract is caught before a loader downstream breaks on it.
Turning checks on
Checks are off until two things are true:
- The sink names the project that receives its findings (
alertProjectId, set withPATCH /orgs/{org}/warehouse/sinks/{sinkId}by an organization admin;nullturns every check on that sink off). A sink belongs to the organization and an issue to a project, so the findings need a project to open in. The project must be your own. - A check is turned on with a governed change (below).
A sink with no alert project cannot have a check changed, and is refused with that remedy. Sinks without checks cost the exporter nothing.
What a failing check does
It opens an ordinary issue in the sink's alert project, with exception.type DataQualityFailed,
fingerprinted on the sink, the stream and the check, so the one alert engine routes it like any
other. The issue opens when a check enters warning or failing, or climbs from warning to
failing, never on every run: a stalled export is an issue within two export intervals, and the same
issue gets another occurrence when it becomes failing. A stale check opens at level warning; a
failing one opens at the check's severity (error by default). The text of the issue is Bucker's own
words over a closed set: the sink's name and your data are never put in it.
Runs are stored for 30 days: when a state changes, and at most once an hour otherwise.
Reading the state
- Dashboard: Warehouse, under the sink list.
- REST:
GET /orgs/{org}/data-quality(optionally?sinkId=). - MCP:
data_quality_status_get(project:read, thedatatoolset). By default only the checks that are on are listed; passallto list the others.includeoftables,check_sqlormonitor_backtestanswersnot_availableand needsdb:readon top.
A sink's alert project, and the issue a check opened there, are shown only to a caller who can read that project. The sink's name is text a member typed: the API and the tool label it untrusted, and the tool fences it before an agent reads it.
Changing a check
POST /orgs/{org}/data-quality/checks and the MCP tool data_quality_checks_update change one
check: turn it on or off (enabled), change its threshold, or change the level of its issue
(severity). It is a governed write (data_quality_check.update, risk class SENSITIVE, asked
for by default):
- The act is read from the difference, never from an argument. Turning a check on, a lower threshold or a higher severity is a tightening. Turning a check off is silencing; a higher threshold or a lower severity is a loosening.
- Silencing and loosening changes carry a 30-day TTL: when it ends, the stored inverse puts the previous setting back (through the sweeper), unless somebody has changed the check since.
- Every plan shows a 30-day replay: what the stored runs of the last thirty days would have raised under the new setting, against what they did raise. The window ends at the start of the current UTC day, so the figure an approver reads is the figure the retry re-derives.
- A first call answers
202input_requiredwith the approval request; a person with the project'sconfigure_dataapprover rights decides it, and the retry (approvalRequestId) applies once, against the revision that was asked on. A check someone edited in between is refused asTARGET_CHANGEDand the yes is not spent. - An organization's own approval-policy row can let a change apply without a person. It applies to the action, so scope it with a condition if you want a tightening to apply while a loosening still asks. Review mode never lifts these changes: a check on an export stream states no environment, and an unstated environment is production.
- Every apply writes a signed receipt, and
operation_revertcan undo it through the stored inverse.
The export gate (data_gate.enable) and monitors on your own tables (data_monitor.disable) are not
reachable here: they are other features with their own approval, and the request is refused by name.
What this does not do
- It does not block an export. A failing check reports; the one thing the exporter itself enforces, with checks on or off, is that the cursor only moves forward: a batch that ends at or before the cursor it started from is refused, and the checkpoint stays where it was.
- It does not use a model. Nothing in this feature phrases anything with one.
- It does not read your warehouse. Every judgement is over what Bucker exported and what Bucker holds.