Agentic traces

A trace is one agent invocation, stored as a tree of spans whose root — span 0 — is the trace itself. Traces are anchored to a run, or to a project alone for inference traces (see writing traces). Reading them takes three steps:

1. ListSpans finds traces, or spans, by name, step, outcome and attributes.
2. ListTraceSpans reads one trace’s whole tree.
3. GetSpanPayload reads the bodies — prompts, completions, tool calls, thoughts — a window at a time.

Finding traces

{
  "workspace": "my-workspace",
  "project": "agents",
  "runs": ["grpo-7b-v2"],
  "filter": "attr(step) >= 1000 AND attr(status) == \"ERROR\"",
  "orderBy": "attr(duration_ms) DESC",
  "attributeNames": ["reward"],
  "limit": "50"
}

• runs names up to 10 runs; leave it empty to search the project’s own (inference) traces.
• By default only root spans — traces — are returned. Set includeChildren to search every span; the filter must then name the operations, with attr(op_name) == "…" or attr(op_name) IN […] as a top-level AND term.
• attributeNames lists logged attributes to return with each span.
• Paging uses limit (at most 1,000), offset and totalCount. Across several runs, limit + offset may not exceed 1,000, and orderBy must name a span field rather than a logged attribute.

Filters and orderings use the resource filter syntax over the span’s logged attributes and these span fields:

Field	Type
attr(op_name)	string — the trace’s name on a root, the operation name elsewhere
attr(step)	integer — the training step that produced the trace
attr(status)	string — RUNNING, OK, ERROR, TRUNCATED, CANCELLED
attr(kind)	string — THOUGHT, ITERATION, LLM_CALL, TOOL_CALL, SUB_AGENT, ARTIFACT, FUNCTION
attr(start_time), attr(end_time)	timestamp
attr(duration_ms)	integer
attr(token_count)	integer
attr(span_count), attr(iteration_count)	integer (roots only)

Traces are stored by operation name and step, so conditions on attr(op_name) and attr(step) are the cheap way to narrow a search; prefer a step range to a start_time window, and filtering to deep paging. The default order is newest step first.

ListTracingOpNames lists the operation names in a run (or the project’s own traces), with their kinds, and ListTracingAttributeDefinitions the attributes logged on spans — useful to build these filters.

Reading a trace

Each Span carries a traceId — an opaque string to pass back — plus its spanId, parentSpanId, ordinal, kind, opName, times, status and tokenCount. A root span also carries the trace’s spanCount and iterationCount.

ListTraceSpans returns every span of one trace, root first and in creation order. Give it the traceId, and the run the trace belongs to (omit it for a project trace). Build the tree by attaching each span to its parentSpanId, ordering siblings by ordinal.

A span whose status is RUNNING has not been closed — still in progress, or its producer stopped before closing it.

Reading payloads

GetSpanPayload reads the bodies of 1 to 100 spans of a trace:

{
  "workspace": "my-workspace",
  "project": "agents",
  "run": "grpo-7b-v2",
  "traceId": "AAAAAAAAAAAAAAAAAAAAAg==",
  "spanIds": ["0", "3", "4"],
  "payloadKinds": ["TRACE_PAYLOAD_KIND_LLM_PROMPT", "TRACE_PAYLOAD_KIND_LLM_COMPLETION"],
  "maxDataLength": 262144
}

• Span 0 asks for the trace’s own payloads: the task input, the final output and the trace’s attributes — and the bodies of the root’s own kind, so a trace rooted at an LLM_CALL answers with its prompt and completion there.
• payloadKinds narrows the kinds; leave it empty for everything the spans logged. Besides the kind-specific bodies, any node may carry the generic INPUT and OUTPUT — what a function was called with and returned.
• Each returned part carries its spanId, payloadKind and body. Text bodies are returned as text; attributes as a JSON object mapping names to values.

Payloads can be large, so they are read as windows. maxDataLength sets the byte budget of one response (1 MiB by default, 16 MiB at most). When more remains, the response carries resumeToken; repeat the identical request with it to continue. Each part reports dataOffset and dataLength — where it starts and the payload’s full size — so you can show progress or fetch only the beginning of a long completion. truncated means the payload was cut at ingestion because it exceeded the size cap; that is different from a response simply ending its window.