RL quickstart guide

Metrana has first-class support for reinforcement learning: per-environment and per-episode metrics, environment attributes, RL-aware querying, and dedicated portal views. This page is a shortcut through those features — each link takes you to the detailed docs, in the order you’d use them: set up, log, query, view.

Before you start

Common setup, whether or not your run is RL:

• Sign in to the portal — get an account, sign in, and join a workspace.
• Get an API key — the SDK authenticates with your individual API key.
• Install the Python SDK — supported platforms and the optional extras.
• Portal overview — how workspaces, projects, and runs are organized, and a tour of the interface.

Log, query, and view RL data

Start with the vocabulary, then follow the data through the system:

• Reinforcement learning concepts — environments, episodes, environment steps, and the two scales; how logging maps to the portal.
• Quickstart: RL training run — a minimal, complete RL logging loop.

Logging

• RL metrics — the full model: per-episode and per-environment-step metrics, vectorized shapes, episode restarts, and resuming RL series.
• Per-environment attributes — attach config to individual environments, env-wide or ranged to specific episodes.

Querying

• Query SDK installation — the query extra and connecting a QueryClient.
• RL environments table — list a run’s environments, their config attributes, and the two timeline axes.
• RL series — pull per-environment curves, choose the scale, and reduce or group across environments.

Viewing in the portal

• Environments overview — the environments table of an RL run, scale toggle, and RL step slider.
• Episodes & steps — navigate RL data by RL step or by episode.
• Environment overview — inspect a single environment’s attributes at a chosen episode.
• Environment charts — visualize the metrics of a single environment.
• Compare environments — overlay metrics from multiple environments on shared charts.
• Environments side-by-side — a grid of environment × metric charts.

Beyond the basics

Not RL-specific, but especially relevant to RL workloads:

• Logging at scale — bulk-logging habits that keep overhead negligible at high env-step rates.
• Distributed logging: RL ownership — many rollout workers logging into one run, and who owns which (environment, episode).
• Guarantees & retries — backpressure, retries, and choosing a resilience profile.
• Disk spool — the recommended durability layer for long training runs.
• Lifecycle: flush, close, shutdown — durability barriers and clean shutdown.
• Tuning & observability — throughput knobs and the SDK’s self-metrics.