Logs: keys on every record, and how agents and people read them
A log record carries the keys it is joined and filtered on, and every door that reads logs
speaks the same search grammar. This page is the reference for what a stored record holds, how
where and fields work, what a search says about itself, and the three MCP tools that read
logs. The search box and its grammar are described under
Searching logs; how records arrive is in OTLP ingest and
Log drains.
What a record carries
| Field | What it is |
|---|---|
service |
The emitting service: OTLP service.name, or the drain's service (one per Vercel project, Worker, Heroku app or Supabase service). Null when the sender named none, and on every record written before the column existed. |
resource |
At most 16 allow-listed resource facts, strings only, each cut to 200 characters: service.version, host.name, k8s.pod.name, k8s.namespace.name, cloud.region, faas.name, vercel.deployment.id, vercel.request.id. A Vercel drain carries the last two on each line, so a record's own value wins over the batch's. Anything else a sender puts in its resource is not stored on the row. |
entityId |
The infrastructure entity the record's resource resolved to, when entity resolution has run. |
subjectKey |
Reserved for the keyed pseudonym of the person a record is about. Empty until F-04 ships: nothing writes it today, so it is not offered as a join key (session_id is in the same position). Erasure already matches it, so rows written after F-04 are reached without a change. |
traceId, spanId, issueId, release, environment, sampleRate |
As before. |
service and resource go through the same stored-tier scrub as the body and the attributes
before they are written: a credential in a host name is redacted, and what is not sensitive is
kept. A drain still drops the client address, the referer and the query string; nothing here
changes that.
In the Explore logs dataset the keys are service, entity_id and the path resource.<key>:
service:checkout, resource.host.name:ip-10-0-0-7. They work in the log search, in Explore
aggregates, in saved queries and in the MCP tools alike.
Searching: where, fields and the evidence
GET /orgs/:org/projects/:project/logs takes the grammar in query (the name the log view has
always used) and, under the name the MCP tool uses, in where; if both are sent they are ANDed.
It also takes fields, up to ten attribute keys (or resource.<key>) separated by commas, and
returns each row's present keys as projected, so a reader gets http.route without the whole
attribute bag. More than ten fields is a 400.
Every page now says what it rests on, as evidence:
| Field | Meaning |
|---|---|
compiled |
The parse of where: filters, text and, for an OR or a negated group, expression. Null when no where was sent. |
window |
from, to and defaulted. A search the caller bounded by neither time, ids, a trace, a span nor an issue is bounded to the last 24 hours by the search itself (defaulted: true); the REST list keeps its older behaviour of reading newest-first from the whole table when it is sent no window. |
scanned, truncated, scannedThrough |
With a scan cap (the MCP tool sets one): how many matching rows were counted, up to the newest 100,000, whether more matched, and how far back the scan reached (the window's start when it was covered whole). A page never reaches past a cut scan. Null and false when the search was not capped, which is the dashboard's case. |
tier, index |
The tier that served it (hot) and the index behind it. A search with a body term says trigram (the body index exists and every partition has it) or none; any other search names the access path its filters let the planner use: primary_key, project_trace_time, issue_time, project_service_time or project_time. The planner chooses; this is the shape's. |
rowsExamined |
The rows the read looked at. Without the index a body search reads the newest 100,000 rows of its window and says how many it read; with it, the matching rows counted (scanned). Null when the search was not capped. |
nextArrivalCursor |
Where "what arrived since" resumes; see Reading what arrived. |
floorNote |
Records at error severity or worse, records with an exception, records linked to an issue and records on an error trace are never sampled away. Anything else may have been, and says so in its sampleRate. |
sampleRates |
The distinct sample rates below 1 on this page. A count over a sampled record is an estimate of 1 / rate per row. |
GET /orgs/:org/projects/:project/logs/:recordId returns one record with its joins (below). A
record of another project, or of another organization, is a 404 identical to an unknown id.
Searching the body
Three body terms, all case-insensitive, all combinable with the grammar and with each other:
- contains:
q(and a free-text term inqueryorwhere),body ILIKE '%term%'; - prefix:
bodyPrefix,body ILIKE 'term%'; - regex:
bodyRegex, a POSIX regular expression (body ~* pattern; note\yis the word boundary and\bis a backspace). At most 100 characters, no backreferences (\1..\9), and cancelled after 10 seconds with a 422 that says so. A pattern longer than that, with a backreference or that does not compile is a 400.*:termover every attribute value is not offered: a blanket index over attribute values is a write cost this deployment does not take on.
LIKE characters in a term (%, _) are literals.
With the index. log_records_body_trgm_idx is a pg_trgm GIN index on body (fastupdate
on) on every log_records partition, behind the log_index pillar. It is built by code and by an
operator script, never by a migration: a migration would build it on every partition inside one
transaction on the busiest insert table, whether or not the operator chose its write cost.
- New partitions are indexed as they are created once the pillar is on (
pillar.log_index.enabled, switched by the operator after the insert-cost harness, H-30). - Existing partitions are indexed by
pnpm -C code --filter @bucker/api backfill-log-trgm(dry run by default,-- --applyto build,-- --days Nto take the newest N days first): one partition at a time,CREATE INDEX CONCURRENTLY, then attached to the parent index, resumable, and refused while the pillar is off. The DEFAULT partition is indexed in a full run. - The response says
index: trigramonly when the parent index exists and every partition, the DEFAULT included, carries a valid trigram index of its own. A half-finished backfill readsnone. (This is read from the partitions, not from the parent's own valid flag: Postgres sets that flag only on an attach, so a parent left invalid by a day that retention has since dropped would otherwise readnonefor good. A backfill stopped between building a day's index and attaching it attaches it on the next run rather than skipping the day.) - Turning the pillar off stops this code indexing new partitions; it does not drop the index, and
once the parent index exists Postgres indexes every new partition itself. Removing it is an
operator's
DROP INDEX log_records_body_trgm_idx.
Without it (index: none). A body search does not run into the statement timeout. It reads the
newest 100,000 rows of its window that pass the other filters, newest first, applies the body term
to them, and says so: rowsExamined, truncated: true and scannedThrough, the timestamp of the
oldest row it read. When the page is not full and the window is not covered, hasMore is true and
nextCursor continues from the last row read, so the next call reads the next slice. A search
with no window is bounded to 24 hours, never more than the deployment's hot window.
Reading what arrived since: a cursor, not a stream
A person's live tail is a server-sent stream. An agent gets no stream: search_logs takes
afterArrivalCursor, the nextArrivalCursor of an earlier answer, and returns only the records
received after it, oldest first, so "what came in since my last read" is one budgeted call.
- Every
search_logsanswer carriessearch.nextArrivalCursor. Start from the one on any ordinary read, or from a change event's time, and pass it back. An ordinary read that found nothing hands out its own instant, so a quiet project can be watched from "now". - An arrival read looks at the last 5 minutes of record time:
receivedAtis not indexed, and this window keeps the read a range scan on the timestamp index. A record stamped by a clock more than five minutes behind its arrival is in a search, not in an arrival read. - Nothing new returns an empty page and the same cursor. When a page was cut for the token budget the cursor resumes after the last record kept, so a record dropped for size is read next time.
- At most 6 arrival reads a minute per principal per project; the seventh is a 429 carrying the seconds to wait. An ordinary read is not counted. The limit is per api machine.
The dashboard tail's bounds
GET …/logs/stream is bounded so one tenant cannot starve ingest of the primary's 8 connections:
| Bound | Value |
|---|---|
| Open tails | 3 per project and 10 per organization, per api machine; the next is a 429 |
| Poll interval | 2 seconds, 10 seconds after 30 seconds with nothing new, back to 2 on a record |
| One tick's query | a 2 second statement timeout; a tick that cannot finish is retried from the same cursor |
A bad bodyRegex |
a 400 before the stream opens (too long, a backreference, or a pattern Postgres cannot compile), taking no slot |
| High volume | above 500 matching records a second a tick keeps a trace-hash sample (a trace is kept or dropped whole) and sends { kind: 'sampled', rate, note: 'sampled at 0.333' } |
| A capped tick | sends { kind: 'capped', shown, skipped }: how many matching records it did not deliver |
The MCP tools
| Tool | What it does | Needs |
|---|---|---|
search_logs |
Rows. where, fields, a cursor and the evidence above; every hit carries its id, service, sampleRate and the short id of its issue. An empty answer says why (not_ingested, no_data_in_window, filtered_out). |
project:read |
logs_query |
Numbers: counts and measures per bucket and group, through the query printer. | event:read |
log_record_get |
One record with its joins in 1,200 tokens. | event:read |
Everything these tools return that came from outside Bucker (bodies, attribute keys and values,
service names, group labels, titles) passes the agent-tier scrub and the injection scanner and is
named in trust.untrustedFields. Attributes, projected fields included, are scrubbed as a bag,
so the rules that match on a key apply: user.id, user.email and user.ip_address come back
hashed, and a credential-named key comes back filtered.
logs_query
"ERROR lines by service every 5 minutes for 6 hours" is one call:
{ "org": "acme", "project": "checkout", "where": "level:>=error",
"groupBy": ["service"], "intervalMinutes": 5, "windowHours": 6 }
whereis the log search grammar, AND only: anORor a negated group is refused with the parse echoed, rather than narrowed to a different question. A simple negation (!env:staging) is fine.measures(up to 4):count(the default),count_unique,sum,avg,min,max,p50,p75,p95,p99. All butcounttake afield.groupBytakes up to three keys. Each key is its own breakdown over the same filters and window: three marginal series, not the cross product. The printer groups by one key per run.- A series is a dense array per measure:
v.<measure>[i]is the bucket atfrom + i × step. - Each breakdown says
countBasis:exact, orsampledwhen any scanned row was stored sampled, in which case every count is a lower bound and thesamplingblock gives the number of sampled rows, the lowest rate and the corrected estimate. - The scan is the newest 100,000 matching rows inside the window (at most 30 days) and says so
when it was cut. It is charged to the QUERY_SCAN budget before it runs and logged on the
mcpsurface. - With more than one key, a later page carries only the breakdowns that still have groups to
read; each names its
groupBy. - When the answer would exceed its budget the smallest groups are dropped,
omittedsays how many, and no cursor is offered; ask again with a smallerlimit.
log_record_get
The record, plus:
| Join | Source | When it is empty |
|---|---|---|
issue |
the issue the record became: short id, status, level, title | no_key, or not_found |
trace |
a summary of the record's trace: span count, duration, services, error spans, the root. At most 200 spans are read; truncated says when more exist. |
no_key, or not_kept (logs and traces are sampled independently, so a log can name a trace that was not stored) |
release |
the release and its deploy change event: when, the summary, minutes between | no_key; the deploy is null when none was recorded |
flags |
the variants the request's root span carried (flagsBasis: trace_root), else the variants the issue's events carried over a span of time that covers the record (issue_events) |
no_key, or not_found |
pattern |
a placeholder until log patterns ship | always not_available |
neighbours |
the five nearest records in time, within 15 minutes, on the trace, else on the service, else in the project | not_found |
The bundle is shrunk in a fixed order to fit: attribute values, then neighbour bodies, then
neighbours from the farthest, then the attribute bag. The head of the body, the severity, the
service and every join's key are never cut, and omitted counts what was.
What this page does not cover
- Exception records inside a logs export are not split out. A log record that carries
exception.*becomes an issue at ingest, and the export of a project's logs still lists it as one row among the others. Separating the exception records in an export is open. subjectKeyandsession_idare not populated; see above.- Patterns (
patternonlog_record_get) arrive with the pattern miner. - Insert cost of the index is measured by the capacity harness before an operator switches the pillar on; this page states the mechanism, not the measurement.