RL series

RL series live inside a run’s environments, so reading them needs both the run and which environments. Two methods, and the choice is whether you want the environments separately or reduced together.

Per-environment, overlaid

from metrana.query import SeriesQuery

reward = SeriesQuery("reward").with_scale("ENVIRONMENT_STEP")
frame = client.fetch_rl_env_float_series_buckets("rl-001", ["envA", "envB"], reward, buckets_count=500)

log_rl_environment_step writes at the ENVIRONMENT_STEP scale and log_rl_episode at EPISODE, while a selector naming no scale means ML_STEP — which no RL series ever has, so it would match nothing. The client cannot know which of the two a metric used, so rather than hand back an empty frame it refuses the call:

client.fetch_rl_env_float_series_buckets("rl-001", "envA", "reward")
# QueryValueError: series 'reward': RL series carry the ENVIRONMENT_STEP or EPISODE scale,
#                  never the default ML_STEP; set one with .with_scale("ENVIRONMENT_STEP")
#                  or .with_scale("EPISODE")

This covers every RL selector — series, summary columns, SeriesFilter, SeriesOrderBy, and group_by conditions. A MetricFilter is the one exemption: it takes the scale from the definitions it resolves against, so it cannot be scale-less.

The environment-level counterpart of fetch_float_series_buckets: index (environment, env_step | rl_step), columns a (series, quantity) MultiIndex over value/min/max/count. Every environment is bucketed on one shared axis, so they overlay directly instead of needing alignment. Non-finite values behave exactly as in the run-level fetch — finite-only aggregates, with frame.attrs["non_finite_buckets"] keyed by (environment, series).

The timeline anchor picks the x axis as well as the reading point:

from metrana.query import Episode, RLStep

reward = SeriesQuery("reward").with_scale("ENVIRONMENT_STEP")
client.fetch_rl_env_float_series_buckets("rl-001", "envA", reward, at=Episode(12))  # x = env step
client.fetch_rl_env_float_series_buckets("rl-001", "envA", reward, at=RLStep(400))  # x = RL step

Both anchors look inside one slice of the timeline. Under an Episode anchor that slice is one episode, and the x axis is the environment step within it. Under an RLStep anchor it is one RL step, and the x axis is the position within that step — the rollout index (rollout_idx), as RL practitioners often call it. An RL step may span more than one episode; you get every episode’s part that falls inside the step, with the episode and env_step extras telling you which part is which. include_timestamps=True adds exactly those positions the chosen axis cannot express — rl_step under an episode anchor, episode and env_step under an RL-step one.

Environments are selected like runs are: one id, a list, a Condition, or an EnvironmentsSelection carrying filter and ordering. At most 1000 per fetch, and going over is an error rather than a silent cut:

from metrana.query import AttrFilter, EnvironmentsSelection, SeriesOrderBy

client.fetch_rl_env_float_series_buckets(
    "rl-001",
    EnvironmentsSelection(
        filter=AttrFilter("cfg/policy") == "ppo",
        order_by=SeriesOrderBy("reward", "avg").with_scale("ENVIRONMENT_STEP").desc(),
    ),
    reward,
    max_environments=20,
)

Reduced across environments

With thousands of environments, overlaying every one is neither readable nor cheap. aggregate_rl_env_series reduces them into a handful of lines, server-side:

from metrana.query import EpisodeRange, RLAggregation, SeriesSummary

frame = client.aggregate_rl_env_series(
    "rl-001",
    SeriesSummary("reward", "avg").with_scale("ENVIRONMENT_STEP"),
    EpisodeRange(0, 100),
    aggregations=[RLAggregation.AVG, RLAggregation.STD],
)

The x axis comes from the range you pass, and there are two aggregation levels at every x:

EpisodeRange — each episode is one x position. Within each environment, the episode’s points collapse to a single value first (that is what the summary function on the SeriesSummary is for — it is required).
RLStepRange — each RL step is one x position. Within each environment, the rollout points inside that step collapse first, the same way.

Then, at each x, those per-environment values are reduced across environments by each requested aggregation (AVG, MIN, MAX, SUM, STD, MEDIAN) — one column each.

The series is a SeriesSummary, and its scale is part of the series identity — unrelated to the x axis. Its labels may be partial and use any operator: where they match several of an environment’s series, the summary function reduces across them too, so an unpinned selector is really three levels (across series, then across environments, at each x). See across-label aggregation; the same FIRST/LAST restriction applies here. Filters and groupings that name a series follow the same rules.

Grouping into several lines

client.aggregate_rl_env_series(
    "rl-001", SeriesSummary("reward", "avg").with_scale("ENVIRONMENT_STEP"), EpisodeRange(0, 100),
    aggregations=["avg"],
    group_by=["cfg/policy"],          # one line per policy value
)

Each grouping becomes an index level, so the result is (cfg/policy, episode) and unstacks straight into a multi-line chart. A grouping may also be a Condition (a predicate evaluated at each x), and ENV_ID gives one line per environment:

from metrana.query import ENV_ID

client.aggregate_rl_env_series(
    "rl-001", SeriesSummary("reward", "avg").with_scale("ENVIRONMENT_STEP"), EpisodeRange(0, 100),
    aggregations=["avg"],
    group_by=[ENV_ID],                # one line per environment
    filter=AttrFilter("cfg/policy") == "ppo",
)

The number of output series after grouping is capped (256 by default) — per-environment grouping is for a filtered handful, not for every environment of a large run.

Compressing the x axis

A long range gives one output point per x. Two ways to shorten it:

client.aggregate_rl_env_series(
    "rl-001", SeriesSummary("reward", "avg").with_scale("ENVIRONMENT_STEP"), EpisodeRange(0, 10_000),
    aggregations=["avg"],
    buckets_count=500,               # at most 500 points over the range
)

buckets_count caps the number of output points. Bucketing happens before the cross-environment reduction and is exact, not sampled: each environment’s bucket value is its summary function pooled over all of its data in the bucket, and those pooled values are what the aggregations reduce.

bucket_width instead fixes the width of a bucket in x units:

client.aggregate_rl_env_series(
    "rl-001", SeriesSummary("reward", "avg").with_scale("ENVIRONMENT_STEP"), EpisodeRange(0, 4_000),
    aggregations=["avg"],
    bucket_width=20.0,               # buckets of 20 episodes, whatever the run's length
)

This is what putting several runs on one chart needs. A bucket count over runs of different lengths gives each run a different width, so their lines are not comparable; a fixed width buckets them alike. Pass the same bucket_width and the same range start to every run in the panel — the grid is anchored at the range’s first x.

The two are mutually exclusive. A width of 1 or below means at most one x per bucket, which is the uncompressed result, so it is returned as such rather than rejected — what a run shorter than the chart is wide produces.

The contributing count

frame["environments"]     # how many environments went into each x

The environment set is re-evaluated at every x — a filter on a re-logged attribute, or on a series value, moves environments in and out along the axis. So the count varies, and it is returned alongside every line rather than left implicit. Positions with no contributing environments are absent: the axis is not dense.