Lifecycle & reliability

The tracer

One tracer per process, like the logger, and paired with it:

metrana.init(...)
tracing.init()           # anchored to that run, with its credentials and delivery settings
...
tracing.close()
metrana.close()

tracing.init() with no arguments takes its target and settings from the run metrana.init opened. With a target of its own — tracing.init(workspace_name, project_name, run_name=None) — it anchors there instead, which is how inference traces are set up in a process with no run. Compression is the one setting a run-derived tracer does not inherit: traces default to zstd, since text shrinks where floats do not.

If tracing.init() is left out, the first trace opened after metrana.init builds the same run-derived tracer; the explicit pair simply reads better. metrana.tracing.Tracer(...) is the same thing as an object, for code that needs several or wants to own the lifetime; the module functions take tracer= to use one.

Closing

metrana.close() does not close the tracer, and tracing.close() does not close the run: each side owns its own stream and is closed by its own call, in the order shown above. Forgetting tracing.close() earns a warning at interpreter exit, as forgetting metrana.close() does, and for the same reason — flushing against a stalled backend at exit is a hang nobody wants, so the SDK warns rather than blocks. Spans still open when the tracer closes are not closed for you; they stay RUNNING in the portal, and the close warns with their count.

After tracing.close() tracing stays off: no tracer is built from the active run any more, traces and decorated calls run inert (with a warning once), and get_tracer() raises. Call tracing.init() to trace again.

metrana.tracing.flush() is the durability barrier: it returns once everything recorded so far is acknowledged by the service, or written to the disk spool. Same contract and defaults as the logger’s flush.

Delivery guarantees

Tracing uses the logger’s engine and its stream to the ingestion service, so everything in Guarantees & retries holds: at-least-once delivery with retries and backoff, the disk spool as failover, bounded waits, and the fail-loud default when every channel has given up. Two things are specific to traces:

• Order. The backend resolves two writes of one span by arrival order, so a close arriving before its open would reset the span. The SDK enqueues the open when a block is entered and the close when it is left, on one tracer; the stream keeps that order end to end, and a re-sent message after a reconnect replays in the same order. No call sequence can produce a close ahead of its open.
• Bodies are cut, not rejected. A prompt or output over 16 MiB is truncated at a character boundary and the payload flagged truncated; a trace must never fail a training job. Attributes past the 256th on a span are cut the same way. Only a single attribute value over 64 KiB is rejected, at the close, with MetranaValidationError.

The spool for traces lives under a subtree of the spool directory root (#traces/...), apart from the run’s own, so a replay of the run’s spool never sees trace records. The default location and the disk= argument are the logger’s. While the tracer runs, it drains its spool back to the service once the service is reachable again; what is still on it when the tracer closes stays on disk, and the close warns with its path. Deliver it with replay_traces, from the same base disk path and target:

tracing.replay_traces(workspace_name="my-team", project_name="agents", run_name="grpo-7b-v2")  # a run's traces
tracing.replay_traces(workspace_name="my-team", project_name="agents")                       # the project's own

A run started offline (init_mode="offline") takes its traces offline too: the tracer writes them to the trace spool only and never contacts the service, which does not know the run yet. Replay the run first — metrana.replay_spool(...) creates it on the service — and its traces after, with tracing.replay_traces(...).

Switching off, and sampling

Instrumented code has to run where no traces are wanted, so tracing degrades to no-ops rather than raising:

• METRANA_TRACING_DISABLED=1 (also true, yes, on) switches tracing off for the process: tracing.init() builds nothing, every trace, span and decorated call is an inert NoopSpan, flush() and close() return at once. Nothing connects, nothing spools. tracing.is_disabled() reports it; tracing.get_tracer() is the one call that still raises, since there is no tracer to hand out.
• Nothing initialized — no tracing.init(), no metrana.init to derive from — gives the same inert nodes, with one warning per process saying so. The typical case is a decorated library function called in a process that never traces.
• sample_rate (tracing.init(sample_rate=0.01), Tracer(..., sample_rate=), or METRANA_TRACING_SAMPLE_RATE) records that share of traces. The decision is made once per root: a trace is kept whole or dropped whole, so a kept trace is never missing spans and a dropped one costs nothing below it — its children, blocks and @op calls alike, are inert. The default is 1.0, every trace.

A child opened with no enclosing span and no parent= is a structural mistake and still raises MetranaTracingError, except under the environment switch, where instrumented code must not be reachable by any tracing error.

Queue and backpressure

Every span open and close waits in the tracer’s queue until the sender takes it, and the queue is bounded by size: queue_max_bytes, 256 MiB by default, with each update counted at its size, bodies included. Under Block, the default, a full queue makes the next with block’s entry wait; the other strategies behave as for the logger. A rollout with hundreds of spans and long prompts is a few megabytes, and the service pipelines a stream’s messages so such a burst drains at network speed rather than one message per batch interval.

Forking

A forked process inherits a copy of the tracer whose threads live in the parent. The first trace in the child detects the new pid, releases the inherited copy without a flush, and rebuilds the tracer from the same configuration — the same behavior the logger has after fork().