Skip to content
bucker

Logs: keys on every record, and how agents and people read them

A log record carries the keys it is joined and filtered on, and every door that reads logs speaks the same search grammar. This page is the reference for what a stored record holds, how where and fields work, what a search says about itself, and the three MCP tools that read logs. The search box and its grammar are described under Searching logs; how records arrive is in OTLP ingest and Log drains.

What a record carries

Field What it is
service The emitting service: OTLP service.name, or the drain's service (one per Vercel project, Worker, Heroku app or Supabase service). Null when the sender named none, and on every record written before the column existed.
resource At most 16 allow-listed resource facts, strings only, each cut to 200 characters: service.version, host.name, k8s.pod.name, k8s.namespace.name, cloud.region, faas.name, vercel.deployment.id, vercel.request.id. A Vercel drain carries the last two on each line, so a record's own value wins over the batch's. Anything else a sender puts in its resource is not stored on the row.
entityId The infrastructure entity the record's resource resolved to, when entity resolution has run.
subjectKey Reserved for the keyed pseudonym of the person a record is about. Empty until F-04 ships: nothing writes it today, so it is not offered as a join key (session_id is in the same position). Erasure already matches it, so rows written after F-04 are reached without a change.
traceId, spanId, issueId, release, environment, sampleRate As before.

service and resource go through the same stored-tier scrub as the body and the attributes before they are written: a credential in a host name is redacted, and what is not sensitive is kept. A drain still drops the client address, the referer and the query string; nothing here changes that.

In the Explore logs dataset the keys are service, entity_id and the path resource.<key>: service:checkout, resource.host.name:ip-10-0-0-7. They work in the log search, in Explore aggregates, in saved queries and in the MCP tools alike.

Searching: where, fields and the evidence

GET /orgs/:org/projects/:project/logs takes the grammar in query (the name the log view has always used) and, under the name the MCP tool uses, in where; if both are sent they are ANDed. It also takes fields, up to ten attribute keys (or resource.<key>) separated by commas, and returns each row's present keys as projected, so a reader gets http.route without the whole attribute bag. More than ten fields is a 400.

Every page now says what it rests on, as evidence:

Field Meaning
compiled The parse of where: filters, text and, for an OR or a negated group, expression. Null when no where was sent.
window from, to and defaulted. A search the caller bounded by neither time, ids, a trace, a span nor an issue is bounded to the last 24 hours by the search itself (defaulted: true); the REST list keeps its older behaviour of reading newest-first from the whole table when it is sent no window.
scanned, truncated, scannedThrough With a scan cap (the MCP tool sets one): how many matching rows were counted, up to the newest 100,000, whether more matched, and how far back the scan reached (the window's start when it was covered whole). A page never reaches past a cut scan. Null and false when the search was not capped, which is the dashboard's case.
tier, index The tier that served it (hot) and the index behind it. A search with a body term says trigram (the body index exists and every partition has it) or none; any other search names the access path its filters let the planner use: primary_key, project_trace_time, issue_time, project_service_time or project_time. The planner chooses; this is the shape's.
rowsExamined The rows the read looked at. Without the index a body search reads the newest 100,000 rows of its window and says how many it read; with it, the matching rows counted (scanned). Null when the search was not capped.
nextArrivalCursor Where "what arrived since" resumes; see Reading what arrived.
floorNote Records at error severity or worse, records with an exception, records linked to an issue and records on an error trace are never sampled away. Anything else may have been, and says so in its sampleRate.
sampleRates The distinct sample rates below 1 on this page. A count over a sampled record is an estimate of 1 / rate per row.

GET /orgs/:org/projects/:project/logs/:recordId returns one record with its joins (below). A record of another project, or of another organization, is a 404 identical to an unknown id.

Searching the body

Three body terms, all case-insensitive, all combinable with the grammar and with each other:

  • contains: q (and a free-text term in query or where), body ILIKE '%term%';
  • prefix: bodyPrefix, body ILIKE 'term%';
  • regex: bodyRegex, a POSIX regular expression (body ~* pattern; note \y is the word boundary and \b is a backspace). At most 100 characters, no backreferences (\1..\9), and cancelled after 10 seconds with a 422 that says so. A pattern longer than that, with a backreference or that does not compile is a 400. *:term over every attribute value is not offered: a blanket index over attribute values is a write cost this deployment does not take on.

LIKE characters in a term (%, _) are literals.

With the index. log_records_body_trgm_idx is a pg_trgm GIN index on body (fastupdate on) on every log_records partition, behind the log_index pillar. It is built by code and by an operator script, never by a migration: a migration would build it on every partition inside one transaction on the busiest insert table, whether or not the operator chose its write cost.

  • New partitions are indexed as they are created once the pillar is on (pillar.log_index.enabled, switched by the operator after the insert-cost harness, H-30).
  • Existing partitions are indexed by pnpm -C code --filter @bucker/api backfill-log-trgm (dry run by default, -- --apply to build, -- --days N to take the newest N days first): one partition at a time, CREATE INDEX CONCURRENTLY, then attached to the parent index, resumable, and refused while the pillar is off. The DEFAULT partition is indexed in a full run.
  • The response says index: trigram only when the parent index exists and every partition, the DEFAULT included, carries a valid trigram index of its own. A half-finished backfill reads none. (This is read from the partitions, not from the parent's own valid flag: Postgres sets that flag only on an attach, so a parent left invalid by a day that retention has since dropped would otherwise read none for good. A backfill stopped between building a day's index and attaching it attaches it on the next run rather than skipping the day.)
  • Turning the pillar off stops this code indexing new partitions; it does not drop the index, and once the parent index exists Postgres indexes every new partition itself. Removing it is an operator's DROP INDEX log_records_body_trgm_idx.

Without it (index: none). A body search does not run into the statement timeout. It reads the newest 100,000 rows of its window that pass the other filters, newest first, applies the body term to them, and says so: rowsExamined, truncated: true and scannedThrough, the timestamp of the oldest row it read. When the page is not full and the window is not covered, hasMore is true and nextCursor continues from the last row read, so the next call reads the next slice. A search with no window is bounded to 24 hours, never more than the deployment's hot window.

Reading what arrived since: a cursor, not a stream

A person's live tail is a server-sent stream. An agent gets no stream: search_logs takes afterArrivalCursor, the nextArrivalCursor of an earlier answer, and returns only the records received after it, oldest first, so "what came in since my last read" is one budgeted call.

  • Every search_logs answer carries search.nextArrivalCursor. Start from the one on any ordinary read, or from a change event's time, and pass it back. An ordinary read that found nothing hands out its own instant, so a quiet project can be watched from "now".
  • An arrival read looks at the last 5 minutes of record time: receivedAt is not indexed, and this window keeps the read a range scan on the timestamp index. A record stamped by a clock more than five minutes behind its arrival is in a search, not in an arrival read.
  • Nothing new returns an empty page and the same cursor. When a page was cut for the token budget the cursor resumes after the last record kept, so a record dropped for size is read next time.
  • At most 6 arrival reads a minute per principal per project; the seventh is a 429 carrying the seconds to wait. An ordinary read is not counted. The limit is per api machine.

The dashboard tail's bounds

GET …/logs/stream is bounded so one tenant cannot starve ingest of the primary's 8 connections:

Bound Value
Open tails 3 per project and 10 per organization, per api machine; the next is a 429
Poll interval 2 seconds, 10 seconds after 30 seconds with nothing new, back to 2 on a record
One tick's query a 2 second statement timeout; a tick that cannot finish is retried from the same cursor
A bad bodyRegex a 400 before the stream opens (too long, a backreference, or a pattern Postgres cannot compile), taking no slot
High volume above 500 matching records a second a tick keeps a trace-hash sample (a trace is kept or dropped whole) and sends { kind: 'sampled', rate, note: 'sampled at 0.333' }
A capped tick sends { kind: 'capped', shown, skipped }: how many matching records it did not deliver

The MCP tools

Tool What it does Needs
search_logs Rows. where, fields, a cursor and the evidence above; every hit carries its id, service, sampleRate and the short id of its issue. An empty answer says why (not_ingested, no_data_in_window, filtered_out). project:read
logs_query Numbers: counts and measures per bucket and group, through the query printer. event:read
log_record_get One record with its joins in 1,200 tokens. event:read

Everything these tools return that came from outside Bucker (bodies, attribute keys and values, service names, group labels, titles) passes the agent-tier scrub and the injection scanner and is named in trust.untrustedFields. Attributes, projected fields included, are scrubbed as a bag, so the rules that match on a key apply: user.id, user.email and user.ip_address come back hashed, and a credential-named key comes back filtered.

logs_query

"ERROR lines by service every 5 minutes for 6 hours" is one call:

{ "org": "acme", "project": "checkout", "where": "level:>=error",
  "groupBy": ["service"], "intervalMinutes": 5, "windowHours": 6 }
  • where is the log search grammar, AND only: an OR or a negated group is refused with the parse echoed, rather than narrowed to a different question. A simple negation (!env:staging) is fine.
  • measures (up to 4): count (the default), count_unique, sum, avg, min, max, p50, p75, p95, p99. All but count take a field.
  • groupBy takes up to three keys. Each key is its own breakdown over the same filters and window: three marginal series, not the cross product. The printer groups by one key per run.
  • A series is a dense array per measure: v.<measure>[i] is the bucket at from + i × step.
  • Each breakdown says countBasis: exact, or sampled when any scanned row was stored sampled, in which case every count is a lower bound and the sampling block gives the number of sampled rows, the lowest rate and the corrected estimate.
  • The scan is the newest 100,000 matching rows inside the window (at most 30 days) and says so when it was cut. It is charged to the QUERY_SCAN budget before it runs and logged on the mcp surface.
  • With more than one key, a later page carries only the breakdowns that still have groups to read; each names its groupBy.
  • When the answer would exceed its budget the smallest groups are dropped, omitted says how many, and no cursor is offered; ask again with a smaller limit.

log_record_get

The record, plus:

Join Source When it is empty
issue the issue the record became: short id, status, level, title no_key, or not_found
trace a summary of the record's trace: span count, duration, services, error spans, the root. At most 200 spans are read; truncated says when more exist. no_key, or not_kept (logs and traces are sampled independently, so a log can name a trace that was not stored)
release the release and its deploy change event: when, the summary, minutes between no_key; the deploy is null when none was recorded
flags the variants the request's root span carried (flagsBasis: trace_root), else the variants the issue's events carried over a span of time that covers the record (issue_events) no_key, or not_found
pattern a placeholder until log patterns ship always not_available
neighbours the five nearest records in time, within 15 minutes, on the trace, else on the service, else in the project not_found

The bundle is shrunk in a fixed order to fit: attribute values, then neighbour bodies, then neighbours from the farthest, then the attribute bag. The head of the body, the severity, the service and every join's key are never cut, and omitted counts what was.

What this page does not cover

  • Exception records inside a logs export are not split out. A log record that carries exception.* becomes an issue at ingest, and the export of a project's logs still lists it as one row among the others. Separating the exception records in an export is open.
  • subjectKey and session_id are not populated; see above.
  • Patterns (pattern on log_record_get) arrive with the pattern miner.
  • Insert cost of the index is measured by the capacity harness before an operator switches the pillar on; this page states the mechanism, not the measurement.