Legion. RateLimiter. Policy
(Legion v0.6.0)
View Source
Defines the limits enforced by a Legion.RateLimiter.
Every policy has a rolling :window_ms. Within that window, it can
limit new agents matching an identity, how many of them run a turn at
once, and the token usage and code evaluations recorded for those agents:
%Legion.RateLimiter.Policy{
window_ms: :timer.minutes(1),
max_agents: 10,
max_running_agents: 2,
max_tokens: 100_000
}Limits
:window_ms(required) - positive rolling-window duration in milliseconds.:max_agents- maximum number of matching agent IDs started during the window, including sub-agents.0allows no agents;nildisables this limit.:max_running_agents- maximum number of matching agents mid-turn at the same time, including sub-agents. It ignores:window_ms: a turn holds its slot for as long as it runs. A turn's usage is recorded when it ends, or at each step checkpoint under a:stepstore, so this is the limit that bounds how far one window can overshoot:max_tokens.0allows no turns;nildisables this limit.:max_tokens- maximum recorded token total for matching agents during the window. Checked when a turn starts, never during one, so one turn can carry the recorded total past the maximum before the next one is denied.0accepts no tokens and so denies every turn;nildisables this limit.:max_evals- maximum recorded code evaluations for matching agents during the window. Checked when a turn starts, like:max_tokens.0denies every turn;nildisables this limit.
A policy with every optional limit set to nil is unrestricted. See
Legion.RateLimiter for the adapter interface and
Legion.RateLimiter.Postgres for the bundled implementation.
Summary
Functions
Returns :ok when policy is a usable policy.
Types
@type t() :: %Legion.RateLimiter.Policy{ max_agents: non_neg_integer() | nil, max_evals: non_neg_integer() | nil, max_running_agents: non_neg_integer() | nil, max_tokens: non_neg_integer() | nil, window_ms: pos_integer() }
Functions
Returns :ok when policy is a usable policy.
Raises ArgumentError naming the offending field otherwise. Legion calls
this when an agent starts, so a misconfigured policy fails at the agent that
configured it rather than at the first limit evaluation.