Skip to content

Spans and kinds

A span is one node of a trace. It has a kind, a name, an input and an output, attributes, a status and a token count, and it nests under the span whose with block encloses it.

with tracing.tool_call("search", input={"q": "spiders legs"}) as tool: # open enqueued now, input with it
results = search(...)
tool.output(results) # sent with the close
tool.set_attributes({"hits": len(results)})
tool.token_count = 0

Every node records its input and output through input() and output(). Which wire body they become follows the kind, so the portal renders a prompt as a prompt and a tool’s JSON as JSON:

Kind Constructor input() becomes output() becomes Represents
LLM_CALL llm_call(name, prompt=…) llm_prompt llm_completion A call to a model
TOOL_CALL tool_call(name, input=…) tool_input tool_output A tool invocation
THOUGHT thought(name) input thought The agent’s reasoning; record it with .text(…)
SUB_AGENT sub_agent(name, input=…) input output A nested agent, with children of its own
ITERATION iteration(name="iteration", input=…) input output One pass of the perceive-act loop; counted on the trace
ARTIFACT artifact(name) input output Something the agent produced; reference it in attributes
FUNCTION function(name, input=…) input output A plain function call; what @tracing.op uses
SESSION_SUMMARY written by session(...) at its exit input output The childless root that carries a session’s reward; not opened by hand

Kind-named aliases exist where a name reads better: llm.prompt(…) and llm.completion(…), thought.text(…). They are input() and output() under other names.

Roots. Any kind may root a trace. On an agent-shaped root (SUB_AGENT, ITERATION, ARTIFACT) the input and output are the invocation’s own task input and final output (trace.task_input(…), trace.final_output(…)); on an LLM_CALL root they are the prompt and completion; on a FUNCTION root the generic pair. See Roots.

One body per kind. Storage keeps one body of each kind per span, so recording the same one twice is an error (MetranaTracingError) rather than a silent overwrite. Text (str or UTF-8 bytes) passes through as is; anything else — a messages list, a dict — is sent as JSON. A value with no JSON form — a dict keyed by tuples or NumPy integers, an object that contains itself — is sent as its repr instead, and the span gets a serialization_error attribute saying why, so a value’s shape never fails the call.

When they are sent. An input recorded before the span starts — the constructor’s prompt= / input=, or input() called before start() — rides the open, so a running span already shows it. Everything else rides the close. Bodies are capped at 16 MiB and cut, not rejected, past that: a trace must never fail a training job.

tracing.agent_trace(name, task_input=None, *, step=None, attributes=None) # SUB_AGENT root
tracing.model_trace(name, prompt=None, *, step=None, attributes=None) # LLM_CALL root
tracing.trace(name, *, kind=SpanKind.SUB_AGENT, step=None, input=None, attributes=None)

agent_trace is a rollout or any agent invocation. model_trace is a trace that is one model call — the inference case. trace roots at any kind. On a tracer anchored to a run the trace needs its policy step, given explicitly or by an enclosing tracing.step(n) block; on a tracer anchored to a project alone it must have none.

Children come from the module functions, which hang off the span whose block encloses the call:

with tracing.agent_trace("rollout", step=step) as trace:
with tracing.sub_agent("planner") as planner:
with tracing.llm_call("policy", prompt=messages): # parent: planner
...
with tracing.llm_call("policy", prompt=messages): # parent: the trace
...

The enclosing span is tracked per thread and per asyncio task, so async code nests correctly on its own. A worker thread does not inherit it: pass the parent explicitly, either as parent= or by calling the constructor on the parent span, which every span also offers (planner.llm_call(...), trace.child(kind, name)).

Span ids and sibling ordinals are assigned for you, in creation order; a span’s span_id, parent_span_id and ordinal are what the query SDK returns.

span.set_attributes({"model": "claude-y", "temperature": 0.2, "cached": False, "tags": ["a", "b"]})
span.set_attribute("retries", 1)

Values are bool, int, float, str or a list of str. Attributes are sent with the close, so they may be added for as long as the span is open. They are what filters and the portal’s trace list select on — a reward on the rollout root is the typical one. A name must not be one of a span’s own fields (status, step, kind, op_name, token_count, trace_id, span_id, …); the close rejects those with MetranaValidationError. Each value may be up to 64 KiB; a span keeps its first 256 attributes and drops the rest.

Ambient attributes. What a whole batch shares — the sample index, the rollout number, whether this is a validation pass — goes on every node opened in a tracing.attributes(...) block, the @op nodes included:

with tracing.step(step), tracing.attributes({"sample_index": i, "validate": False}):
rollout(task) # every span of it carries both

Blocks nest, inner values over outer ones, and a node’s own attributes= / set_attributes win over both. Like the enclosing span, the block is tracked per thread and per asyncio task. tracing.current_attributes() reads the merged set.

error_count on the root. The SDK counts the spans of a trace that ended ERROR and records the count on the root as the error_count attribute (0 included), so AttrFilter("error_count") > 0 finds every rollout in which something failed, whether or not the rollout itself recovered. The root’s own status is never changed by its children. An error_count you set yourself is left alone.

Status, exceptions and the manual lifecycle

Section titled “Status, exceptions and the manual lifecycle”

Leaving a block closes the span with SpanStatus.OK. An exception escaping it closes the span with SpanStatus.ERROR, records "ExceptionType: message" under the exception attribute, and propagates. To choose the status yourself, or to open and close a span from different places, use start() and end():

span = tracing.tool_call("deploy", input=plan).start()
...
span.end(tracing.SpanStatus.CANCELLED) # OK, ERROR, TRUNCATED or CANCELLED

span.token_count = n records what a span consumed, on its close. The trace’s own token_count is the sum over its spans unless you set it; its span_count and iteration_count are counted from the tree. All three are stored on the root, and the trace list sorts and filters on them — iteration_count is how many loop turns a rollout took. The trace’s error_count is counted the same way and lands as an attribute (see above).

Where there is nothing to record on, the constructors hand out a NoopSpan instead of raising: tracing switched off with METRANA_TRACING_DISABLED=1, nothing initialized and no run to derive a tracer from, or a root the tracer’s sample_rate left out. It has the whole surface — with, input() / output() and the kind-named bodies, attributes, token_count, children — and does nothing, and every child opened inside its block is inert too, so instrumented code runs unchanged. See Lifecycle & reliability.