Legion.RateLimiter.Policy (Legion v0.6.0)

View Source

Defines the limits enforced by a Legion.RateLimiter.

Every policy has a rolling :window_ms. Within that window, it can limit new agents matching an identity, how many of them run a turn at once, and the token usage and code evaluations recorded for those agents:

%Legion.RateLimiter.Policy{
  window_ms: :timer.minutes(1),
  max_agents: 10,
  max_running_agents: 2,
  max_tokens: 100_000
}

Limits

  • :window_ms (required) - positive rolling-window duration in milliseconds.
  • :max_agents - maximum number of matching agent IDs started during the window, including sub-agents. 0 allows no agents; nil disables this limit.
  • :max_running_agents - maximum number of matching agents mid-turn at the same time, including sub-agents. It ignores :window_ms: a turn holds its slot for as long as it runs. A turn's usage is recorded when it ends, or at each step checkpoint under a :step store, so this is the limit that bounds how far one window can overshoot :max_tokens. 0 allows no turns; nil disables this limit.
  • :max_tokens - maximum recorded token total for matching agents during the window. Checked when a turn starts, never during one, so one turn can carry the recorded total past the maximum before the next one is denied. 0 accepts no tokens and so denies every turn; nil disables this limit.
  • :max_evals - maximum recorded code evaluations for matching agents during the window. Checked when a turn starts, like :max_tokens. 0 denies every turn; nil disables this limit.

A policy with every optional limit set to nil is unrestricted. See Legion.RateLimiter for the adapter interface and Legion.RateLimiter.Postgres for the bundled implementation.

Summary

Functions

Returns :ok when policy is a usable policy.

Types

t()

@type t() :: %Legion.RateLimiter.Policy{
  max_agents: non_neg_integer() | nil,
  max_evals: non_neg_integer() | nil,
  max_running_agents: non_neg_integer() | nil,
  max_tokens: non_neg_integer() | nil,
  window_ms: pos_integer()
}

Functions

validate!(policy)

Returns :ok when policy is a usable policy.

Raises ArgumentError naming the offending field otherwise. Legion calls this when an agent starts, so a misconfigured policy fails at the agent that configured it rather than at the first limit evaluation.