Web analytics: rollups, freshness, channels and the registry
Web analytics in Bucker is counted at write and recomputed on a ladder. Nothing here reads a raw
event: an accepted $pageview is counted into an hourly rollup and dropped, and everything a
session says (sessions, bounces, duration, uniques, entry and exit, channel, UTM fields) is
recomputed from the per-session rows. Both lines need product analytics enabled for the project
(PA-01) and the web_rollups pillar on; with either off nothing is stored.
Rollups
| Table | Grain | Kept |
|---|---|---|
web_rollup_hours |
one row per project, site, UTC hour, dimension kind and value | 13 months (396 days) |
web_rollup_days |
the same shape per UTC day | at most 25 months (760 days), the CNIL and AEPD cap on audience-measurement data |
Each row holds pageviews, sessions, bounces, summed duration, bot pageviews, goal conversions, the
consent-basis mix, a visitor sketch, and the classifier and channel versions it was computed under.
Every hourly and daily bucket states computedAt and lateEvents (events that arrived more than an
hour after the bucket closed and were still counted in it).
Dimensions: total, and, per hour, channel, device_class, traffic_type, country (top 50),
consent_basis, and the high-cardinality path, entry_path, exit_path, referrer_domain and UTM
fields (top 100 values an hour, counts only, the rest folded into $other with the total kept
exact; top 500 a day). Crawlers are counted as bot pageviews on the total and traffic_type rows
and never in pageviews.
Uniques are approximate, and say so
Visitors are a HyperLogLog sketch of keyed visitor pseudonyms, precision 12 on a totals row (standard error about 1.6%) and precision 10 on a low-cardinality dimension (about 3.3%). A day sketch is built from the day's own sessions, so a returning visitor counts once. A range of 24 hours or less answers from hot rows and is exact; the evidence block says which tier answered.
Freshness: the recompute ladder
| Rung | Cadence |
|---|---|
| the current hour | every 5 minutes |
| the previous 6 hours | hourly |
| the last 7 days | nightly (the retention sweep runs with it) |
A recompute replaces the session-owned columns of a bucket and keeps the event-owned ones the fold
wrote. A project is visited only when its gate is open; one project failing is reported and the
others still run. The scheduler leaves a web_rollups row in scheduler_runs, so a ladder that
stopped is a lastSuccessAt that stopped moving.
computedAt on a row is the time of its last write, by either writer: a pageview folded a minute
ago makes it recent even when the session columns were last recomputed at the previous rung.
Erasure
A sketch cannot subtract a visitor, so erasing a person is a rebuild. The erasure seam
(eraseFromWebRollups) notes every hour and day the person's sessions started in, lets the subject
erasure delete the sessions, and then recomputes exactly those buckets, whatever their age and
whatever the project's analytics gate says now. Afterwards no row carries the person's sessions,
bounces, duration, consent basis or any visitor register they set. Pageview counts the fold wrote
from events stay: they are counts that name nobody.
Channels
The channel of a session is stored per rule-set version, so a rule-set release is a recompute of
sessions, never a re-read of events. A release that would move more than 2% of a project's last 30
days runs in shadow, with a before-and-after notice for that project, until it is accepted
(measureChannelShift is the measure: each of the project's human sessions of the last 30 days
classified under both versions). A project held on no rule set keeps its own version.
The attribution registry
GET /web/attribution-registry serves the registry the SDK and the fold parse a landing with
(UTM fields and aliases, click ids, in-app browser rules): public, cacheable for an hour, ETag is
the sha256 of the document as served. It carries no tenant data and no input. When no registry is installed the route answers
503 attribution_registry_unavailable rather than an empty document.
The evidence block
Every web read carries a web-evidence/1 block: tier, approximate uniques and error, computedAt,
lateEvents, bucket, requested and applied timezone (non-integer offsets are refused), classifier
and channel versions, consent counts, sources, and a digest. Free text inside it is listed as
untrusted.
Export
WEB_SESSION_ROLLUPS (bucker-schema WEB_SESSION_ROLLUPS@1) exports the hourly rows to a
warehouse, paged by (computedAt, recordId); a recomputed bucket is re-exported.