Message types
Messages and enums reachable from the RPC request and response messages. Request and response messages themselves are documented next to their RPC. Field names are shown as declared in the .proto files and in the camelCase form the REST API uses in JSON.
metrana_protobuf.ingestion.v1.svc
Section titled “metrana_protobuf.ingestion.v1.svc”ResumePolicy
Section titled “ResumePolicy”How the client wants an already-existing run handled at init. Letting the server adjudicate (instead of the client deciding after the handshake) means a refused attach never creates a logger row — otherwise the orphaned initializing row would keep the victim run from resolving CLOSED and eventually flip it to CRASHED via the crash sweep.
| Value | Number | Description |
|---|---|---|
RESUME_POLICY_UNSPECIFIED |
0 | Treated as RESUME_POLICY_NEVER — the wire zero-value gets the safe behavior, so a client that forgets to set the field cannot silently append to a foreign run. |
RESUME_POLICY_NEVER |
1 | Attach to an existing run only when it provably belongs to the same job launch: the run’s stored orchestration identifier equals the request’s (or this logger created the run). Otherwise the server sends InitializedRun{created: false, ...} and ends the stream WITHOUT creating a logger row; the client surfaces the conflict error. |
RESUME_POLICY_ALLOW |
2 | Always attach. |
metrana_protobuf.common.v1
Section titled “metrana_protobuf.common.v1”Attribute
Section titled “Attribute”Attribute represents snapshot value of an Attribute
| Field | JSON | Type | Description |
|---|---|---|---|
path |
path |
string |
Attribute path. |
bool_value |
boolValue |
booloneof value |
Boolean value |
string_value |
stringValue |
stringoneof value |
String value |
float_value |
floatValue |
doubleoneof value |
Float value |
int_value |
intValue |
int64oneof value |
Integer value |
string_set_value |
stringSetValue |
StringSet oneof value |
Snapshot of string set |
timestamp_value |
timestampValue |
google.protobuf.Timestamponeof value |
Timestamp value |
SeriesValueKind
Section titled “SeriesValueKind”Value type of a series’ points.
| Value | Number | Description |
|---|---|---|
SERIES_VALUE_KIND_UNDEFINED |
0 | |
SERIES_VALUE_KIND_FLOAT |
1 | Scalar float per step. |
SERIES_VALUE_KIND_HISTOGRAM |
2 | Histogram per step: bin edges + integer counts. Counts are additive, so these series support merging and count-based reductions. |
SERIES_VALUE_KIND_DENSITY_HISTOGRAM |
3 | Histogram per step normalized to a probability density (numpy density=True). Sample counts are divided out, so merging and count/sum reductions are undefined; shape reductions (mean/std/pXX) remain available. |
SERIES_VALUE_KIND_SCATTER |
4 | Scatter plot per step: parallel x/y coordinate arrays. A step holds the full point set — re-logging a step overwrites it (never merges), and no reductions are defined. |
SERIES_VALUE_KIND_SCATTER_3D |
5 | 3D scatter plot per step: as SCATTER, with a third parallel z coordinate array. Dimensionality is part of the series kind — one series is either 2D or 3D for its whole lifetime. |
StringSet
Section titled “StringSet”StringSet represents a set of unique string values.
| Field | JSON | Type | Description |
|---|---|---|---|
values |
values |
repeated string |
List of unique values in a set. |
metrana_protobuf.ingestion.v1.types
Section titled “metrana_protobuf.ingestion.v1.types”AggregationFn
Section titled “AggregationFn”AggregationFn is the function applied when aggregating series.
| Value | Number | Description |
|---|---|---|
AGGREGATION_FN_UNSPECIFIED |
0 | |
AGGREGATION_FN_MEAN |
1 | |
AGGREGATION_FN_MAX |
2 | |
AGGREGATION_FN_SUM |
3 | |
AGGREGATION_FN_MIN |
4 | |
AGGREGATION_FN_STD_DEV |
5 | |
AGGREGATION_FN_COUNT |
6 |
AggregationRule
Section titled “AggregationRule”AggregationRule declares how one or more series should be aggregated into a derived series for a run. Rules are registered at run creation time and applied by the ingestion worker.
Grouping semantics: candidate series are grouped by (metric_name, source_scale, labels \ aggregate_over_labels). Only series sharing the same metric_name and source_scale may be combined.
| Field | JSON | Type | Description |
|---|---|---|---|
metric_name |
metricName |
optional string |
Metric to apply the rule to. If absent, the rule applies to every metric whose series matches source_scale and aggregate_over_labels. |
source_scale |
sourceScale |
string |
Scale of the source series (e.g. “EPISODE”, “ENVIRONMENT_STEP”). |
output_scale |
outputScale |
string |
Scale of the derived output series (e.g. “EPISODE”). |
fns |
fns |
repeated AggregationFn | Aggregation functions to apply. Each function produces a separate output series. Must contain at least one value; AGGREGATION_FN_UNSPECIFIED is not valid here. |
aggregate_over_labels |
aggregateOverLabels |
repeated string |
Labels to aggregate over. Series that share the same values for all labels NOT in this set are merged together. Labels in this set are stripped from the output series. Example: aggregate_over_labels=[“environment_id”] collapses all sub-series that differ only in environment_id into one output series; the output series carries no environment_id label. |
output_metric_name |
outputMetricName |
optional string |
Name for the output series. Only valid when metric_name is set. If absent and metric_name is set, defaults to metric_name. |
output_name_suffix |
outputNameSuffix |
optional string |
Suffix appended to each source metric_name to form the output metric name. Used only when metric_name is absent. Ignored when both metric_name and output_metric_name are set. |
AttributesSet
Section titled “AttributesSet”| Field | JSON | Type | Description |
|---|---|---|---|
attributes |
attributes |
repeated Attribute |
AttributeUpdate
Section titled “AttributeUpdate”AttributeUpdate represents insertion or update of an Attribute
| Field | JSON | Type | Description |
|---|---|---|---|
path |
path |
string |
Attribute path, must be unique within Run context. |
bool_value |
boolValue |
booloneof value |
Boolean value |
string_value |
stringValue |
stringoneof value |
String value |
float_value |
floatValue |
doubleoneof value |
Float value |
int_value |
intValue |
int64oneof value |
Integer value |
string_set_value |
stringSetValue |
StringSet oneof value |
Snapshot of string set |
string_set_delta_value |
stringSetDeltaValue |
StringSetDelta oneof value |
Delta to apply on existing string set. If set does not exist, added values will be treated as a snapshot. |
timestamp_value |
timestampValue |
google.protobuf.Timestamponeof value |
Timestamp value |
CloseRun
Section titled “CloseRun”| Field | JSON | Type | Description |
|---|---|---|---|
close_message |
closeMessage |
optional string |
CompoundFloatPointSet
Section titled “CompoundFloatPointSet”Set of points for float series. DEPRECATED - this was replacing array of CompoundPoint, but later improved to RlStepBatch that is even smaller for RL series, other kinds were not using it.
| Field | JSON | Type | Description |
|---|---|---|---|
major_steps |
majorSteps |
repeated uint64 |
|
minor_steps |
minorSteps |
repeated uint64 |
|
timestamp_millis |
timestampMillis |
repeated int64 |
|
values |
values |
repeated double |
CompoundPoint
Section titled “CompoundPoint”CompoundPoint consists of 2 step parts - major and minor. Context of usage determines its logic. DEPRECATED - this is the oldest version of points, supported by JSON loggers
- metrana < 0.5.0
| Field | JSON | Type | Description |
|---|---|---|---|
major_step |
majorStep |
uint64 |
First part of X-axis (major). Allows values for up to 2**63-1. |
minor_step |
minorStep |
uint64 |
Second part of X-axis (minor). Allows values for up to 2**63-1. |
timestamp |
timestamp |
optional google.protobuf.Timestamp |
Optional timestamp assigned to a point. |
float_value |
floatValue |
doubleoneof value |
Floating point value |
EnvAttrUpdates
Section titled “EnvAttrUpdates”EnvAttrUpdates carries a batch of attribute updates for a single RL environment, optionally scoped to an episode. Mirrors how RLSeriesUpdate associates data with an environment_id. Attributes logged at an episode stay in effect for all later episodes until the same path is re-logged for the environment (see the rl_env_attribute ClickHouse table); they are not materialized per episode.
| Field | JSON | Type | Description |
|---|---|---|---|
environment_id |
environmentId |
string |
Name of the environment, required. |
episode |
episode |
optional uint64 |
Optional episode at which these attributes were logged. When absent, the attributes are treated as applying from the environment’s earliest (no-episode) point. |
attributes |
attributes |
repeated AttributeUpdate | Attribute updates to apply to this environment. |
FloatPointSet
Section titled “FloatPointSet”Set of points for float series. DEPRECATED - this was replacing array of Point, but later improved to PointBatch that is even smaller.
| Field | JSON | Type | Description |
|---|---|---|---|
steps |
steps |
repeated uint64 |
|
timestamp_millis |
timestampMillis |
repeated int64 |
|
values |
values |
repeated double |
ForkPoint
Section titled “ForkPoint”ForkPoint represents a point in the parent Run where other run can be forked.
| Field | JSON | Type | Description |
|---|---|---|---|
parent_run_name |
parentRunName |
string |
Parent run name. It must exist within same workspace and project. |
fork_step |
forkStep |
uint64 |
Step where fork is executed. Fork step is inclusive - it is forwarded to the forked run. Be careful about series - some use simple 8 byte steps, some compound 16 bytes - major and minor. As of now, compound steps are used for RL metrics only, where major step is called rl_step. Fork step is mapped to major points there. |
HistogramPoint
Section titled “HistogramPoint”HistogramPoint is a histogram logged at a single step. Histograms are heavy per point and never logged in dense vectors, so there is no columnar batch form. Same-step re-log overwrites the previous histogram (never merges). HistogramPoint is numpy compatible.
| Field | JSON | Type | Description |
|---|---|---|---|
step |
step |
uint64 |
X-axis step position of the point being added. |
timestamp_millis |
timestampMillis |
optional int64 |
Optional timestamp assigned to the point. |
bin_edges |
binEdges |
repeated double |
Strictly increasing, finite bin edges; length = bins + 1. At most 513 edges (512 bins). |
counts |
counts |
repeated uint64 |
Per-bin element counts. |
densities |
densities |
repeated double |
Per-bin probability densities. |
stats |
stats |
HistogramPoint.HistogramStats | Exact moments of the raw (pre-binning) values, when the client had them. Alongside counts they are raw moments (sum over all values). Alongside densities the sample count is unrecoverable, so they must be NORMALIZED moments: sum = mean, sum_of_squares = mean of squares (equivalently, the raw moments of a count=1 sample); min/max stay the sample bounds. When omitted, the ingestion service stores a bin-approximation instead (min/max = occupied-bin outer edges, midpoint-weighted sum, Sheppard-corrected sum of squares), so stored histograms of both kinds always carry stats. An all-zero histogram (valid: “no events”) stores the exact empty sums and NaN for the undefined min/max. Total count is not carried: it is the sum of counts for counting histograms, and the 0-or-1 empty/non-empty sentinel for density ones. |
HistogramPoint.HistogramStats
Section titled “HistogramPoint.HistogramStats”Exact moments of the raw values behind a histogram.
| Field | JSON | Type | Description |
|---|---|---|---|
min |
min |
double |
|
max |
max |
double |
|
sum |
sum |
double |
|
sum_of_squares |
sumOfSquares |
double |
HistogramSeriesUpdate
Section titled “HistogramSeriesUpdate”HistogramSeriesUpdate represents an update (or creation) of a histogram series. Histogram series are a separate update type from scalar series: they carry no ordering mode — points may arrive at any step in any order, and re-logging a step overwrites its histogram (never merges). Whether the series is a counting or density histogram is inferred from the first point’s HistogramValue and must not change during metric lifetime.
| Field | JSON | Type | Description |
|---|---|---|---|
metric_name |
metricName |
string |
Name of the metric. |
scale |
scale |
string |
Scale this series belongs to. Part of the series identity (metric_name, scale, labels). Must not change during metric lifetime. |
labels |
labels |
map<string, string> |
List of key-value pairs that fully identify a series object. Label key and values must be alpha numeric only plus following characters: ‘.’ | ‘_’ | ‘-’ | ‘/’ | ‘:’ |
points |
points |
repeated HistogramPoint | Histogram points to append. Must not be empty. |
OrderingMode
Section titled “OrderingMode”OrderingMode describes how data points in a series are ordered relative to previously submitted points.
| Value | Number | Description |
|---|---|---|
ORDERING_MODE_UNDEFINED |
0 | |
ORDERING_MODE_RELATIVE_ORDERED |
1 | Points submitted in relative order (each step >= previous), resolved against session-relative state. REMOVED — the ingestion service rejects this value. Steps are now always absolute; use ORDERING_MODE_ABSOLUTE_ORDERED. The number stays reserved so historical payloads and stored rows still decode, and the v0 ingestion path still interprets data written before the removal. |
ORDERING_MODE_ABSOLUTE_UNORDERED |
2 | Points may be submitted out-of-order with absolute step values. Auto-aggregations are skipped. |
ORDERING_MODE_ABSOLUTE_ORDERED |
3 | Points are submitted in order (each step > previous) using absolute step values. Because steps are globally monotonic rather than session-relative, the backend can deduplicate on step value alone (step <= last accepted step is dropped), without consulting session_id or any session-relative last-step bookkeeping. Auto-aggregations are applied normally. This is the preferred mode and lets the ingestion pipeline eventually stop tracking its own per-session step increments. |
Point represents an amendment for single step. DEPRECATED - this is the oldest version of points, supported by JSON loggers
- metrana < 0.5.0
| Field | JSON | Type | Description |
|---|---|---|---|
step |
step |
uint64 |
X-axis step position of the point being added. |
timestamp |
timestamp |
optional google.protobuf.Timestamp |
Optional timestamp assigned to a point. |
float_value |
floatValue |
doubleoneof value |
Floating point value |
PointBatch
Section titled “PointBatch”PointBatch describes set of points for generic series.
| Field | JSON | Type | Description |
|---|---|---|---|
values |
values |
repeated double |
Point values, columnar. Exactly one of values / values_f32 carries the batch (both empty only for an empty batch). |
values_f32 |
valuesF32 |
repeated float |
Point values narrowed to float32 by the client (per-series dtype). Consumers widen back to float64 — the widening is exact, and the zeroed mantissa tail compresses away in columnar storage. |
first_step |
firstStep |
optional uint64 |
Contiguous steps starting here (one per value). Ignored when steps is non-empty. |
steps |
steps |
repeated uint64 |
Explicit per-point steps (fallback when not contiguous). |
timestamp_millis |
timestampMillis |
optional int64 |
Uniform timestamp applied to every point. Ignored when timestamps_millis is non-empty. |
timestamps_millis |
timestampsMillis |
repeated int64 |
Explicit per-point timestamps (fallback when not uniform). |
RLSeriesUpdate
Section titled “RLSeriesUpdate”RLSeriesUpdate is similar to SeriesUpdate, but dedicated for RL series (assigned to environment ID). RL series are identified by metric name, environment, episode, and extra optional labels.
| Field | JSON | Type | Description |
|---|---|---|---|
metric_name |
metricName |
string |
Name of the metric. |
environment_id |
environmentId |
string |
Name of the environment, required. |
episode |
episode |
optional uint64 |
Optional episode for given series, if relevant. DEPRECATED: Use rl_step_batches |
scale |
scale |
string |
Scale this series belongs to. Part of the series identity (metric_name, scale, labels). Together with metric_name, identifies which series processor descriptor governs aggregations and transformations for this series. Must not change during metric lifetime. |
labels |
labels |
map<string, string> |
List of key-value pairs that fully identify a series object. Label key and values must be alpha numeric only plus following characters: ‘.’ | ‘_’ | ‘-’ | ‘/’ | ‘:’ |
points |
points |
repeated CompoundPoint | Number of data points to append. Must be greater than 0. Compound point have 2 step levels. Major step is equivalent to RL step. It is also a forking step point. Its value must be always provided as absolute value. It must be always increasing for individual series. If it drops, points are deduplicated. Minor step is within specific RL step. It must respect ordering field. If ordering is relative, minor step must be always increasing, and must reset across logger process restarts. Ingestion on session_id change will rewind automatically minor step to match next available, so no data points are lost. If ordering is “absolute”, then minor step may be any value within allowed integer range (2**63-1), and data may arrive in any order. DEPRECATED: Use rl_step_batches |
ordering |
ordering |
OrderingMode | Ordering mode for data points in this series. Note it affects only “minor step” value in CompoundPoint object. Major step must be absolute and direct value. It is a RL step value and maps to ML step (for forking). |
float_points |
floatPoints |
CompoundFloatPointSet | Float points for this series. If populated, other kinds must not be used. DEPRECATED: Use rl_step_batches |
rl_step_batches |
rlStepBatches |
repeated RlStepBatch | One batch per (rl_step, episode) contributing to this (env, series). |
RlStepBatch
Section titled “RlStepBatch”RlStepBatch describes set of points for specific rl step/episode. Used for RL series updates.
| Field | JSON | Type | Description |
|---|---|---|---|
rl_step |
rlStep |
uint64 |
Single major step shared by every point in this batch. |
episode |
episode |
optional uint64 |
Optional episode shared by every point in this batch. |
values |
values |
repeated double |
Point values, columnar. Exactly one of values / values_f32 carries the batch (both empty only for an empty batch). |
values_f32 |
valuesF32 |
repeated float |
Point values narrowed to float32 by the client (per-series dtype). See PointBatch.values_f32. |
first_minor_step |
firstMinorStep |
optional uint64 |
Contiguous minor steps starting here (one per value). Ignored when minor_steps is non-empty. |
minor_steps |
minorSteps |
repeated uint64 |
Explicit per-point minor steps (fallback when not contiguous). |
timestamp_millis |
timestampMillis |
optional int64 |
Uniform timestamp applied to every point. Ignored when timestamps_millis is non-empty. |
timestamps_millis |
timestampsMillis |
repeated int64 |
Explicit per-point timestamps (fallback when not uniform). |
treat_minors_as_flat_steps |
treatMinorsAsFlatSteps |
bool |
If true, the per-point minor field carries the raw flat (env) step: the worker stores it verbatim as flat_step and assigns a dense compound minor (reset to 0 each rl_step). If false, the minor field is the compound minor and flat_step is derived by accumulation. |
env_steps_from_one |
envStepsFromOne |
bool |
If true, the accumulated flat (env) step axis starts at 1 instead of 0: the first point of a fresh (or restarted) episode gets flat_step 1. Consulted only when the batch initializes or restarts an episode — a continuing episode keeps accumulating from its current flat step — so the flag must stay constant across a series’ batches. Episodic batches only (the non-episodic axis always starts at 0), and only meaningful when treat_minors_as_flat_steps is false (explicit flat steps are the client’s own numbering); setting it without an episode or with flat steps is rejected. |
Run represents a user facing definition of Run.
| Field | JSON | Type | Description |
|---|---|---|---|
workspace |
workspace |
string |
Workspace where run belongs to |
project |
project |
string |
Project where run belongs to |
run_name |
runName |
string |
Unique run name within a workspace/project. |
experiment |
experiment |
optional string |
Optional experiment name run belongs to. If experiment is set and does not exist during run creation, it is created along. It cannot be changed. |
creation_time |
creationTime |
optional google.protobuf.Timestamp |
Time of Run creation. If not provided system will use current time. |
fork_point |
forkPoint |
optional ForkPoint | Optional fork point from other run. |
directory |
directory |
optional string |
Optional full directory path where run should be created initially. Change of directory is possible via management service. Directory must have elements separated by “/” sign. |
exp_directory |
expDirectory |
optional string |
Optional full directory for experiment, if it is created for the first time. If experiment already existed, this field has no effect. If experiment is created along Run, then experiment directory will be: • If exp_directory is set, experiment will be put there. • If exp_directory is not set BUT directory is set, experiment will be put there. • If neither exp_directory nor directory is set, then experiment will be without specific directory (visible in project experiment list). Change of directory for an experiment is possible via management service. Directory must have elements separated by “/” sign. |
orchestration_identifier |
orchestrationIdentifier |
optional string |
Opaque token identifying the orchestration (job launch) that created this run. All processes legitimately sharing a run within one job carry the same token; a new job launch carries a different one. Compared client-side on a conflict to tell a sibling process that won the create race from a stale run left by a prior job. Not interpreted by the server. |
organization |
organization |
optional string |
Slug of the organization owning the workspace. Workspace names are unique per organization, so this is required once another organization has a workspace of the same name; until then it may be omitted. |
ScatterPoint
Section titled “ScatterPoint”ScatterPoint is a scatter plot logged at a single step: the FULL point set of that step as parallel coordinate arrays. Same-step re-log overwrites the previous set (never appends/merges). Scatter points are heavy and never logged in dense step vectors, so there is no columnar batch form.
| Field | JSON | Type | Description |
|---|---|---|---|
step |
step |
uint64 |
X-axis step position of the point being added. |
timestamp_millis |
timestampMillis |
optional int64 |
Optional timestamp assigned to the point. |
xs |
xs |
repeated double |
Parallel coordinate arrays: always equal length, 0 to 32768 entries. An empty set is a valid step — “no points here” — and overwrites a previously logged set like any other re-log. Since empty arrays cannot tell 2D from 3D, dimensionality rides the update’s is_3d flag. The upper bound keeps a full-size point under the 1 MiB transport message budget (~768 KiB worst case with the third axis). Coordinates must be finite — a NaN/Inf coordinate is unplottable and rejected at ingestion. |
ys |
ys |
repeated double |
|
zs |
zs |
repeated double |
Optional third axis: empty for a 2D scatter, else parallel to xs/ys. |
ScatterSeriesUpdate
Section titled “ScatterSeriesUpdate”ScatterSeriesUpdate represents an update (or creation) of a scatter series. Like histogram series, scatter series are a separate update type from scalar series: they carry no ordering mode — points may arrive at any step in any order, and re-logging a step overwrites its point set (never merges).
| Field | JSON | Type | Description |
|---|---|---|---|
metric_name |
metricName |
string |
Name of the metric. |
scale |
scale |
string |
Scale this series belongs to. Part of the series identity (metric_name, scale, labels). Must not change during metric lifetime. |
labels |
labels |
map<string, string> |
List of key-value pairs that fully identify a series object. Label key and values must be alpha numeric only plus following characters: ‘.’ | ‘_’ | ‘-’ | ‘/’ | ‘:’ |
points |
points |
repeated ScatterPoint | Scatter points to log. Must not be empty. |
metadata_labels |
metadataLabels |
map<string, string> |
Non-identity display metadata; see StreamingScatterSeriesUpdate.metadata_labels for the semantics. Scatter axis titles go under the well-known “xt”/“yt”/“zt” keys (absent titles render as plain axes). |
is_3d |
is3d |
bool |
Series dimensionality: a scatter series is 2D or 3D for its whole lifetime. The flag is authoritative — every non-empty point must agree with it (zs present iff 3D); an empty point set carries no arrays, so the flag alone states its dimensionality. |
SeriesDeclaration
Section titled “SeriesDeclaration”| Field | JSON | Type | Description |
|---|---|---|---|
metric_name |
metricName |
string |
Name of the metric. |
scale |
scale |
string |
Scale this series belongs to. Part of the series identity (metric_name, scale, labels). Together with metric_name, identifies which series processor descriptor governs aggregations and transformations for this series. Must not change during metric lifetime. |
labels |
labels |
map<string, string> |
List of key-value pairs that fully identify a series object. Label key and values must be alpha numeric only plus following characters: ‘.’ | ‘_’ | ‘-’ | ‘/’ | ‘:’ |
ordering |
ordering |
OrderingMode | Ordering mode for data points in this series. |
declaration_id |
declarationId |
uint64 |
Mandatory ID, valid only in context of single streaming session. Disconnection requires resending. |
SeriesUpdate
Section titled “SeriesUpdate”SeriesUpdate represents update (or creation) of a new series. Type of value will be inferred from point value types. Note that unique value of a series is metric name, scale and labels.
| Field | JSON | Type | Description |
|---|---|---|---|
metric_name |
metricName |
string |
Name of the metric. |
scale |
scale |
string |
Scale this series belongs to. Part of the series identity (metric_name, scale, labels). Together with metric_name, identifies which series processor descriptor governs aggregations and transformations for this series. Must not change during metric lifetime. |
labels |
labels |
map<string, string> |
List of key-value pairs that fully identify a series object. Label key and values must be alpha numeric only plus following characters: ‘.’ | ‘_’ | ‘-’ | ‘/’ | ‘:’ |
points |
points |
repeated Point | Number of data points to append. Must be greater than 0. DEPRECATED: Use point_batches |
ordering |
ordering |
OrderingMode | Ordering mode for data points in this series. |
float_points |
floatPoints |
FloatPointSet | Float points for this series. If populated, other kinds must not be used. DEPRECATED: use point_batches |
point_batches |
pointBatches |
repeated PointBatch | List of point batches |
SpanClose
Section titled “SpanClose”Completion of a span, carrying whatever bodies it produced.
It embeds the whole SpanOpen rather than just the span id: the stored row is a ReplacingMergeTree row replaced wholesale by the higher-precedence write, so a close that carried only the outcome would blank the span’s kind, name, parent and ordinal. Restating the open message the SDK already holds is both complete and impossible to drift field-by-field, and it keeps the close matching the open because the SDK reuses the same values.
| Field | JSON | Type | Description |
|---|---|---|---|
span |
span |
SpanOpen | The open, restated as sent: name must not change within a span. |
end_time |
endTime |
google.protobuf.Timestamp |
|
status |
status |
SpanStatus | Must be terminal — not RUNNING. |
payloads |
payloads |
repeated SpanPayload | The span’s bodies. The restated span must carry none: they were delivered by the open. |
token_count |
tokenCount |
uint64 |
Tokens this span consumed, for per-span search and sorting. The trace’s total rides TraceClose; this is not summed into it. |
SpanKind
Section titled “SpanKind”What a span node represents. ITERATION is one pass of the agent’s perceive-act loop — not a training step, and not a conversational turn (which is a whole trace, since a trace is one invocation).
| Value | Number | Description |
|---|---|---|
SPAN_KIND_UNSPECIFIED |
0 | |
SPAN_KIND_THOUGHT |
1 | |
SPAN_KIND_ITERATION |
2 | |
SPAN_KIND_LLM_CALL |
3 | |
SPAN_KIND_TOOL_CALL |
4 | |
SPAN_KIND_SUB_AGENT |
5 | Sub-agents are spans with children, so they need no separate model. |
SPAN_KIND_ARTIFACT |
6 | |
SPAN_KIND_FUNCTION |
7 | A plain function call: the kind-neutral node a decorator wraps any function in. Its bodies are the generic input / output. |
SPAN_KIND_SESSION_SUMMARY |
8 | The record a session leaves when it ends: a childless root carrying the session’s own attributes (session/reward, session/turn_count) under the session/id its turns carry. A session may leave several; the newest is its current state. |
SpanOpen
Section titled “SpanOpen”Start of a span. Never the root: that is TraceOpen.
| Field | JSON | Type | Description |
|---|---|---|---|
span_id |
spanId |
uint64 |
A per-trace counter the SDK increments from 1. Not a storage id: it only discriminates within a trace. |
parent_span_id |
parentSpanId |
uint64 |
Another span’s counter value; 0 means the span hangs off the trace root. |
ordinal |
ordinal |
uint32 |
Sibling order. |
kind |
kind |
SpanKind | |
name |
name |
string |
|
start_time |
startTime |
google.protobuf.Timestamp |
|
payloads |
payloads |
repeated SpanPayload |
SpanPayload
Section titled “SpanPayload”A body attached to a span, or one chunk of it. The client chunks a payload too large for one request; the ingestion service chunks further whenever a message would exceed the Kafka limit, restating the carrying update with each chunk. Every chunk is self-addressing through data_offset, so chunks of one payload may arrive in any order and any number of messages.
| Field | JSON | Type | Description |
|---|---|---|---|
data_offset |
dataOffset |
uint32 |
Where this chunk starts: a byte offset for text bodies, an attribute index for attributes. |
data_length |
dataLength |
uint32 |
The whole payload’s size in the same unit as data_offset, repeated on every chunk. The client states it; the service never fills it in or changes it. Required, and a chunk reaching past it is rejected. |
thought |
thought |
bytesoneof payload |
|
tool_input |
toolInput |
bytesoneof payload |
|
tool_output |
toolOutput |
bytesoneof payload |
|
llm_prompt |
llmPrompt |
bytesoneof payload |
|
llm_completion |
llmCompletion |
bytesoneof payload |
|
task_input |
taskInput |
bytesoneof payload |
|
final_output |
finalOutput |
bytesoneof payload |
|
attributes |
attributes |
AttributesSet oneof payload |
|
input |
input |
bytesoneof payload |
Generic input and output, valid on every node whatever its kind: what a function was called with and what it returned, beside (not instead of) the kind-specific bodies above. |
output |
output |
bytesoneof payload |
|
truncated |
truncated |
bool |
Server-set on EVERY chunk of a payload whose data_length exceeds the cap, so a preview reading only the first chunk can say the body was cut without summing the rest. Clients leave it unset. |
SpanStatus
Section titled “SpanStatus”Terminal state of a trace or span. Only the close message carries one, and it must not be RUNNING — that is what the open write stores.
| Value | Number | Description |
|---|---|---|
SPAN_STATUS_UNSPECIFIED |
0 | |
SPAN_STATUS_RUNNING |
1 | |
SPAN_STATUS_OK |
2 | |
SPAN_STATUS_ERROR |
3 | |
SPAN_STATUS_TRUNCATED |
4 | The work completed but its recorded payload was capped. |
SPAN_STATUS_CANCELLED |
5 |
StreamingHistogramSeriesUpdate
Section titled “StreamingHistogramSeriesUpdate”StreamingHistogramSeriesUpdate is the histogram series update for the streaming (gRPC) path, addressing the series via its declaration. The declaration’s ordering mode is ignored for histogram series.
| Field | JSON | Type | Description |
|---|---|---|---|
series_id |
seriesId |
uint64 |
Declared series ID. |
points |
points |
repeated HistogramPoint | Histogram points contributing to this series. |
StreamingRLSeriesUpdate
Section titled “StreamingRLSeriesUpdate”StreamingRLSeriesUpdate is the compacted RL series update for the streaming (gRPC) path. Points are grouped into per-(rl_step, episode) batches so each batch’s major step and episode are single values rather than per-point arrays, and the environment name is sent once per (env, series). ingestion_svc expands each batch into a CompoundFloatPointSet for the worker.
| Field | JSON | Type | Description |
|---|---|---|---|
series_id |
seriesId |
uint64 |
Declared series ID. |
environment_id |
environmentId |
string |
Name of the environment, required. |
rl_step_batches |
rlStepBatches |
repeated RlStepBatch | One batch per (rl_step, episode) contributing to this (env, series). |
StreamingScatterSeriesUpdate
Section titled “StreamingScatterSeriesUpdate”StreamingScatterSeriesUpdate is the scatter series update for the streaming (gRPC) path, addressing the series via its declaration. The declaration’s ordering mode is ignored for scatter series.
| Field | JSON | Type | Description |
|---|---|---|---|
series_id |
seriesId |
uint64 |
Declared series ID. |
points |
points |
repeated ScatterPoint | Scatter points contributing to this series. |
metadata_labels |
metadataLabels |
map<string, string> |
Non-identity display metadata (e.g. scatter axis titles under the well-known “xt”/“yt”/“zt” keys). Never part of the series identity: stored on the series header, where the latest non-empty set logged for a series wins. An empty map leaves stored metadata untouched, so clients send the map only when it changes. Keys and values follow the identity-label charset for now (an escaping scheme may later open values up to free-form text). |
is_3d |
is3d |
bool |
Series dimensionality; see ScatterSeriesUpdate.is_3d. The declaration carries no value kind, so the flag rides every update. |
StreamingSeriesUpdate
Section titled “StreamingSeriesUpdate”StreamingSeriesUpdate is the compacted series update for the streaming (gRPC) path. Points are grouped into batches so a batch with contiguous steps sends a single first_step rather than a per-point steps array, and a batch with a uniform timestamp sends a single timestamp_millis rather than a per-point array. ingestion_svc concatenates every batch back into one FloatPointSet for the worker (a plain series has a single flat step axis, so there is no per-batch key — multiple batches just accumulate, e.g. when the client merges several log calls or a non-contiguous segment arrives).
| Field | JSON | Type | Description |
|---|---|---|---|
series_id |
seriesId |
uint64 |
Declared series ID. |
point_batches |
pointBatches |
repeated PointBatch | Batches contributing to this series, concatenated in order. |
StringSetDelta
Section titled “StringSetDelta”StringSetDelta represents update of some existing StringSet
| Field | JSON | Type | Description |
|---|---|---|---|
added_values |
addedValues |
repeated string |
List of unique added values |
removed_values |
removedValues |
repeated string |
List of unique removed values. |
TraceClose
Section titled “TraceClose”Completion of a trace: the close of its root span. The counters are display values for the trace list, written once here rather than incremented per span.
| Field | JSON | Type | Description |
|---|---|---|---|
end_time |
endTime |
google.protobuf.Timestamp |
|
status |
status |
SpanStatus | Must be terminal — not RUNNING. |
span_count |
spanCount |
uint32 |
|
iteration_count |
iterationCount |
uint32 |
|
token_count |
tokenCount |
uint64 |
|
payloads |
payloads |
repeated SpanPayload | The agent’s terminal result. final_output and attributes belong here whatever the root is; a root also takes the output body of its own kind — an LLM_CALL root its llm_completion, a TOOL_CALL root its tool_output. |
open |
open |
TraceOpen | The open this closes, restated for the same reason SpanClose.span is: the root row is replaced wholesale, so a close carrying only the outcome would blank the trace’s name. |
TraceIdentity
Section titled “TraceIdentity”Everything that identifies one trace before the worker derives its trace_id. Carried by EVERY trace message, not just the open one: the worker re-derives the id for each message, and the producer routes on the identity because trace_id does not exist yet at produce time.
| Field | JSON | Type | Description |
|---|---|---|---|
run_id |
runId |
bytes |
RunId, u128 big endian. Empty means an unanchored (inference) trace, owned by the project alone. |
step |
step |
optional uint64 |
The generating policy version. Present exactly when run_id is. |
client_trace_key |
clientTraceKey |
bytes |
The SDK’s handle for this trace: 16 random bytes. MUST be random, never an incrementing counter — the SDK cannot guarantee a counter does not restart inside a run, and a collision aliases two traces onto one storage identity. |
trace_start_time |
traceStartTime |
google.protobuf.Timestamp |
Start of the trace, carried on every message and never read from the worker’s clock: it feeds the derived trace_id, so a clock read on the close would derive a different trace. |
TraceOpen
Section titled “TraceOpen”Start of a trace: the open of its root span. The trace has no row of its own — it is span 0, the parent every top-level span hangs off — so this is a SpanOpen for span 0 with the root’s own fields: no parent, no ordinal, and a start time carried by the identity.
| Field | JSON | Type | Description |
|---|---|---|---|
name |
name |
string |
What the invocation was: the agent or task name, listed and searched on as the root’s op_name. |
payloads |
payloads |
repeated SpanPayload | The task the agent was given, known at open, and the trace’s attributes. task_input and attributes belong here whatever the root is; a root also takes the input body of its own kind — an LLM_CALL root its llm_prompt, a TOOL_CALL root its tool_input. |
kind |
kind |
SpanKind | What the root is: an agent rollout roots at a SUB_AGENT, an inference trace around one model call at an LLM_CALL. |
TraceUpdate
Section titled “TraceUpdate”One trace-structure update from a client. Order within a request is preserved, so a span’s open precedes its close.
| Field | JSON | Type | Description |
|---|---|---|---|
identity |
identity |
TraceIdentity | Identity of the trace this update belongs to. |
trace_open |
traceOpen |
TraceOpen oneof update |
|
span_open |
spanOpen |
SpanOpen oneof update |
|
span_close |
spanClose |
SpanClose oneof update |
|
trace_close |
traceClose |
TraceClose oneof update |