Legion.RateLimiter.Postgres (Legion v0.6.0)

View Source

A Postgres-backed Legion.RateLimiter extending Legion.Store.Postgres.

This adapter depends on an existing Legion.Store.Postgres configuration. It stores rate-limit metadata in that store's table and reads persisted usage from it. Configure the store first, then define a rate limiter using the same Ecto repo and table:

defmodule MyApp.AgentStore do
  use Legion.Store.Postgres, repo: MyApp.Repo
end

defmodule MyApp.RateLimiter do
  use Legion.RateLimiter.Postgres, repo: MyApp.Repo
end

The Store migration creates the rate-limit metadata column and its GIN index. No separate rate-limiter migration is required:

defmodule MyApp.Repo.Migrations.AddLegionAgents do
  use Ecto.Migration

  def up, do: Legion.Store.Postgres.Migration.up()
  def down, do: Legion.Store.Postgres.Migration.down()
end

Call the generated limiter before running work, passing every rule that applies to the agent:

policy = %Legion.RateLimiter.Policy{
  window_ms: :timer.minutes(1),
  max_agents: 10,
  max_tokens: 100_000
}

:ok =
  MyApp.RateLimiter.enforce!(agent_id, [
    %Legion.RateLimiter.Rule{identity: %{"ip" => "203.0.113.42"}, policy: policy}
  ])

All rules are evaluated inside one transaction: the adapter locks every distinct identity (in a global order, so agents naming the same identities in different orders cannot deadlock), records the agent, then checks the rules in the order given. The first violated rule raises and rolls the whole call back, leaving metadata only for allowed calls.

Limits are evaluated when an agent or a turn starts; a turn that is already running is never interrupted, so one turn can carry the recorded total past :max_tokens before the next call is denied. A :step store records usage at each step checkpoint, so calls made while that turn runs see its spend.

Identity matching

An agent's rule identities are merged into one JSON metadata document and matched with Postgres JSON containment. Each rule is evaluated against the agents whose metadata contains its identity, so an agent recorded under %{"ip" => "203.0.113.42"} and %{"email" => "someone@example.com"} counts towards both groups, and a broader identity such as %{"ip" => "203.0.113.42"} also matches an agent recorded with a "tenant" field. Rules passed directly to enforce!/2 must agree on shared fields; Legion.start_link/2 validates this, and the adapter does not. Calling enforce!/2 again for the same agent replaces its stored metadata while retaining its original start time, and marks it running when a rule limits running agents.

Limit evaluation

The adapter includes the agent being checked when it evaluates :max_agents, so it raises only when that call would make the matching count exceed the configured maximum. Agents are counted by started_at.

:max_running_agents counts the matching agents mid-turn: rows whose stored status is running and whose agent process is alive, however long ago the turn started. A turn interrupted by a crash never writes idle back, so a dead process does not hold a slot; an agent restarted under the same id before its interrupted run is recovered holds one until its next turn ends. When a rule sets this limit, the adapter marks the caller running in the same transaction, before it counts, so concurrent starts see each other; the caller's own row is included, as with :max_agents.

:max_tokens is evaluated from recorded "total_tokens" usage whose "at" timestamp falls inside the policy's window; once that total reaches the configured maximum, later calls raise.

:max_evals is evaluated the same way from recorded "evals" usage.

:max_tokens, :max_evals and :max_running_agents read what the agent's store writes, so they need that store to be a Legion.Store.Postgres on the limiter's repo and table; an agent started with them and any other store, or none, raises ArgumentError. Token and eval limits also require Legion usage tracking, which is enabled by default: with config :legion, :track_usage, false they raise too. :max_agents needs neither.

Options

  • :repo (required) - the Ecto repo used by the configured store.
  • :table - the shared Store table name, defaulting to "legion_agents". It must also be passed to the Store migration.