Skip to content
bucker

The insight query

An insight is a question about how people use your product: how many, how they convert, who comes back, where they go. It is written as one node, a small JSON document from a closed set, and answered from the project's rollups, never from raw events. The answer comes back with an envelope that says which data cut produced it, so the same number can be re-derived tomorrow.

Everything here is behind the project's product-analytics gate (PA-01): a project that has not enabled analytics, with its lawful basis recorded and the amended DPA accepted, refuses every door below before the node is even read.

The doors

Three REST doors, all reads of the project (analytics:read, relation read_analytics). They are POST because the node is a body; none of them changes anything you can see.

Route What it does
POST /orgs/{org}/projects/{project}/insights/query Run a node. Returns { envelope, data }.
POST /orgs/{org}/projects/{project}/insights/preview The plan the node would make, without running it: what it resolves to, the specs and tables it compiles to, why the rollups cannot answer it if they cannot, lint. Reads no rollup, stores nothing.
POST /orgs/{org}/projects/{project}/insights/upgrade Migrate a node written in an older version to the current one. Rewrites JSON only.

The body is { "node": { ... } }. The project is the path and nothing else: a node cannot name a project or an organization.

The node grammar (version 1)

Every node is a strict object. A field this page does not name is refused with its path, not ignored. Every node carries v: 1, a kind, a range, and the shared fields below.

Shared field Meaning
range { "kind": "relative", "last": 30, "unit": "day" | "week" | "month" } (ends today, UTC) or { "kind": "absolute", "from": "YYYY-MM-DD", "to": "YYYY-MM-DD" }. At most 400 days.
filters Up to 10, from the closed grammar: event, action, property (eq, neq, in, not_in, exists, not_exists), cohort, account. There is no expression language and no regular expression.
exclude { "internal": boolean, "actors": [subject keys] }.
identity as_of_event (default) or current.
Kind Fields Limits
trends series[] (event, math: total unique_subjects sessions accounts, filters), breakdowns[], interval (hour day week month), compare 20 series; 2 breakdowns of at most 25 values, the rest folded into other
funnel steps[] (event or action), window (value, unit: minute hour day), order (ordered unordered strict) 2 to 20 steps; a window of 1 minute to 90 days
retention event, periods, period (day week), lookback_days 12 periods
paths start, steps, max_edges, exclude_events[] 5 steps, 50 edges
stickiness event DAU, WAU, MAU and the ratios per day
lifecycle event, period, lookback_days new, returning, resurrecting, dormant
web metric, interval, breakdown compiled by the web-analytics extension

A version-0 node (type, a bare days, events, window_days) is migrated by upgrade. A version this build does not know is refused, never guessed at.

Events must be core events

A rollup holds an event only if it is $any (every event) or one of the project's declared core events (up to 20). Asking for another event is a 422 with reason: event_not_core and the list of core events, not an empty answer. A project still in its first-week observation has $any only. A read never freezes the observation default; only the taxonomy update and the first fold do.

The horizon

"New" depends on how far back you trust the data. Retention and lifecycle take lookback_days (default 90, at most 365): a subject active on the cohort day and on any day from range.from - lookback_days to the day before is not new. The envelope echoes the horizon.

The envelope

{
  "v": 1,
  "engine": "insight-compiler/1",
  "kind": "trends",
  "nodeHash": "…",
  "result": "ok",
  "compiled": { "specs": [ … ], "reads": [{ "table": "pa_core_event_daily", … }] },
  "resolved": { "range": { "from": "2026-11-01", "to": "2026-11-30", "days": 30, "relative": true }, "interval": "day", "coreEventsVersion": 2, "identity": "as_of_event", "horizon": null },
  "isApproximate": true,
  "approximation": ["subject_sketch_estimate"],
  "scanned": { "rows": 90, "resultBytes": 2113 },
  "tier": "postgres_rollup",
  "killed": false,
  "trustCheckDigest": null,
  "reconciliation": { "drops": 4, "sampled": 0, "quarantined": 0, "botsExcluded": null, "internalExcluded": null, "personless": null, "unavailable": ["botsExcluded", "internalExcluded", "personless"], "window": { … } },
  "rollup": { "id": "pa_core_event_daily", "version": 1, "state": "live", "watermark": "…", "reconciledWithin": 0.005, "lastDriftPct": 0.1 },
  "cohortFloor": { "applied": false, "k": 5, "floorVersion": 1, "suppressed": 0 },
  "warnings": [],
  "extensions": {},
  "cache": { "key": "…", "hit": false, "ageSeconds": 0, "ttlSeconds": 21600 }
}
  • compiled is the node as F-05 query specs over rollup:<table> datasets, and the tables read. Every table is a sufficient-statistics rollup (pa_core_event_daily, pa_subject_activity, pa_subject_event_first, pa_funnel_daily, pa_path_edges_daily); a plan that names any other table is refused. The specs parse with the printer's grammar.
  • resolved is what the node meant: a relative range as the absolute days it covered.
  • isApproximate with its reasons. subject_sketch_estimate (a daily distinct count is a sketch, about 1% off), sessions_summed_across_days and its siblings (a week's sessions are the sum of days, an over-count), funnel_first_of_day_rule (a subject counts by its first sighting of a step each day, never above the strict funnel), edges_folded_into_other. An exact answer says isApproximate: false.
  • reconciliation is what never reached the rollups: drops, sampled and quarantined counts from the outcome ledger and the filters. Facts the ledger does not keep today are null and listed in unavailable; a null is not a zero.
  • rollup is the audited rollup's watermark and how closely the last audit matched the ledger (half a percent). A range that reaches past the watermark carries a warning.
  • cache: see below.
  • result is ok, no_data (no rollup row fell in the range) or tier_unavailable.

What the rollups cannot answer

Where the grammar allows more than the rollups hold, the answer is result: tier_unavailable with a reason, status 200, no data and no approximation. The columnar tier answers these later.

Reason Node
interval_hour_unavailable an hourly trend (the rollups are daily)
multiple_breakdowns_unavailable, multiple_dimensions_unavailable two breakdowns, or a filter and a breakdown on different properties (a rollup row carries one key)
breakdown_not_promoted a breakdown on a property the project has not verified and promoted
property_filter_unavailable a property filter that is not eq or in on a promoted property, or on any node but trends
event_filter_unavailable an event filter (the series or step already names the event)
funnel_order_unavailable an unordered or strict funnel (first-of-day rows keep no order inside a day)
funnel_steps_unavailable, funnel_window_unavailable, step_filters_unavailable more than 10 steps, a window past 30 days, a filter on a step
actor_exclusion_unavailable, identity_current_unavailable exclude.actors, identity: current
subject_filter_unavailable a cohort or account restriction on a trends or paths node
web_compiler_unavailable the web node with no web-analytics extension registered

A cohort, account or action filter with no extension that resolves it is a 422 (extension_unavailable), not an empty answer.

The results cache

A computed answer is kept in insight_results, keyed by everything it depends on and nothing it does not: the node's hash, the resolved absolute range, the rollup watermark, the core-event version, the floor version and k, the viewer's floor class and the engine version. A changed cut is a new key, so nothing is ever invalidated; an entry just ages out, by interval: hour 1 hour, day 6 hours, week and month 24 hours. The gate is checked before the cache is consulted, so a project whose gate has since shut is never served a cached answer. A stored answer carries no timestamp, which is what lets a nightly re-run at the echoed range and watermark reproduce it exactly.

The floor, and what it is not

Every answer states a cohortFloor block: applied, k, floorVersion, how many cells were withheld (suppressed) and the exemption when there is one. k is the project's minCohortSize (default 5, 3 to 50). Once installCohortFloor() has run at api boot, any cell (a trends data point, a breakdown value, a funnel step, a retention cell, a lifecycle bucket) with fewer than k distinct subjects is folded into other or shown as "<k", and every ratio, total or histogram that would hand the small count back is withheld with it: a trends total beside any withheld point, a funnel entry-day column with one withheld day, an other that is itself under k. A paths node is refused to a floored caller (floor_refused, rule paths_count_transitions): an edge counts transitions, and one person looping between two pages makes an edge of any size, so nothing in the rollup can show an edge holds k people. A filter that pins one subject (a property that names a person, or a promoted value held by fewer than k subjects) is refused with floor_refused. The floor applies to every caller except a signed-in human holding the human-only pii:read and admitted by the organization's person-level gate; an agent, an OAuth client, a delegated call or an organization token is floored even holding the scope, and the review-mode step cannot lift it. An exempt read is ledgered (one data.read row, no value on it) and cached under its own viewer class, never served to a floored caller.

k moves through the analytics_floor family: analytics_floor.narrow (raise, governed, ALLOW) and analytics_floor.widen (lower, human-only org:admin on the person's own session), at GET/PUT .../analytics/floor and POST .../analytics/floor/preview.

What the floor is not: differential privacy. The difference of two SHOWN counts (a funnel step's entered minus converted) or of two answers can still isolate a small group; query budgets (PA-11) and the read ledger bound that, the floor does not. And person-level reads (a subject's properties, an event timeline, an actor list) are not insight nodes and never will be. They are dashboard reads for a human holding pii:read, each writing an audit row.

Extensions

Cohorts, accounts, actions, the web node, revenue and metric definitions plug in through registerNodeExtension, from their own files. An extension can add a filter resolver, a breakdown key, an x_<name> aggregation, a compiler for a node kind, an envelope block and the rollup tables it reads. It cannot drop the tenant predicate (the tenant is the read's scope, applied by the readers, not a clause), name a tenant key in a clause, mutate the context it is handed, widen a subject set (an include set can only narrow, and two intersect), or read a table outside the rollups. A failing or oversized envelope block is dropped with a warning; it never fails the read.