Behaviour for enforcing rate limits across Legion agents.

A rate limiter decides whether an agent may proceed under a list of Legion.RateLimiter.Rules. Each rule pairs an identity - the group of agents sharing a limit - with a Legion.RateLimiter.Policy. Legion calls enforce!/2 for you: configure a limiter and rules, and every turn is checked against them.

Legion.start_link(ChatAgent,
  rate_limit: [
    limiter: MyApp.RateLimiter,
    rules: [
      %Legion.RateLimiter.Rule{
        identity: %{"ip" => "203.0.113.42", "tenant" => "acme"},
        policy: %Legion.RateLimiter.Policy{
          window_ms: :timer.minutes(30),
          max_agents: 40,
          max_tokens: 200_000
        }
      },
      %Legion.RateLimiter.Rule{
        identity: %{"email" => "someone@example.com"},
        policy: %Legion.RateLimiter.Policy{window_ms: :timer.hours(24), max_agents: 5}
      }
    ]
  ]
)

A turn runs only if every rule allows it; the first rule that denies it cancels the turn. Rules are evaluated in the order given, so put the rule whose denial you want reported first.

For Postgres users there is a ready-made adapter - see Legion.RateLimiter.Postgres. It extends Legion.Store.Postgres, keeping rate-limit metadata in the store's table and counting the usage persisted there:

defmodule MyApp.RateLimiter do
  use Legion.RateLimiter.Postgres, repo: MyApp.Repo
end

Configuration

The limiter and a default policy can be set globally, leaving only the identities to the call site. The policy is a struct, so set it in config/runtime.exs: config/config.exs is read before Legion is loaded.

config :legion, :rate_limit,
  limiter: MyApp.RateLimiter,
  default_policy: %Legion.RateLimiter.Policy{window_ms: :timer.minutes(1), max_agents: 10}

Legion.start_link(ChatAgent,
  rate_limit: [rules: [%Legion.RateLimiter.Rule{identity: %{"ip" => "203.0.113.42"}}]]
)

A rule given without a :policy takes the default one. Rules belong to the call site: the application config accepts :limiter and :default_policy. Rate limiting applies only when a limiter and at least one rule resolve. Rules without a limiter raise. A limiter without rules runs the agent without rate limiting and logs a warning - pass rules: [] to opt out on purpose and silence it. Unknown keys in either place are logged and ignored. Sub-agents inherit whatever their parent resolved - a parent started without rate limiting runs its whole subtree without it, and rules given below it raise rather than re-enable it. Each sub-agent is a separate agent ID in every group, so it counts towards :max_agents, towards :max_running_agents while its turn runs, and its usage towards :max_tokens, from its start, as any agent does. A parent waiting on a sub-agent is mid-turn too, so the two hold two :max_running_agents slots: with a limit of 1, no sub-agent ever runs.

Rules must agree on their identities: two rules may share a field only with the same value, since the adapter records one group membership per agent. Two rules with the same identity and different policies are fine - for example a per-minute and a per-day window on one IP. Identity keys must be strings; an empty identity %{} matches every agent and acts as a global cap.

When Legion enforces

Enforcement happens when an agent starts, and then once per turn, before the incoming message is appended and before any LLM request. A denial therefore leaves the conversation untouched and costs nothing. A denied start leaves nothing at all: Legion.start_link/2 returns {:error, {:rate_limited, violations}}, with no process started and nothing saved. The start is checked against :max_agents only; the other limits are left to the turns, so a supervisor can always restart a crashed agent.

Denied turns return {:cancel, {:rate_limited, violations}} - the same shape as {:cancel, :reached_max_iterations} - so a denied sub-agent reports back to its caller as a value rather than a crash. Legion also emits [:legion, :rate_limit, :exceeded] with the identity and policy of the rule that denied it; see Legion.Telemetry.

Resumed and recovered runs are not checked again. They are finishing work that was already allowed, so checking it again would discard accepted work instead of shedding new load.

Rules are not persisted either, so a resumed agent's later turns are checked only when :rate_limit is passed to Legion.resume/2 again.

A Legion.eval/3 call is checked the same way, before the code runs, with the rules the agent was started with. A denied call runs nothing and records nothing, and returns the same {:cancel, {:rate_limited, violations}}; Legion.MCP.Server turns it into a tool error naming the limits that were reached.

Calling it yourself

resolve!/1 and enforce!/2 are public, so an application can rate-limit its own work - a webhook, a queue worker - under the same configuration as agents. Resolve the configured limiter and rules, which fills in the default policy, then enforce them under a stable id:

%{limiter: limiter, rules: rules} =
  Legion.RateLimiter.resolve!(
    rules: [%Legion.RateLimiter.Rule{identity: %{"ip" => "203.0.113.42"}}]
  )

if limiter, do: :ok = limiter.enforce!(agent_id, rules)

A nil limiter means rate limiting is off. An adapter can also be called directly with complete rules, skipping resolve!/1.

Implementing a rate limiter

Define a module that implements enforce!/2 when rate-limit state lives outside Postgres or needs application-specific coordination:

defmodule MyApp.RateLimiter do
  @behaviour Legion.RateLimiter

  @impl Legion.RateLimiter
  def enforce!(agent_id, rules) do
    # Check and record the rate-limit state for every rule, atomically.
    :ok
  end
end

An adapter receives all of an agent's rules in one call so it can allow or deny the call as a unit. See Legion.RateLimiter.Rule, Legion.RateLimiter.Policy, and Legion.RateLimiter.Postgres.