Built-in Middleware

Choose policies for the failure and resource limits your application actually needs. Add them through the middleware DSL , then check their ordering and total retry budget. All short module names below are under BeamWeaver.Agent.Middleware.

Run this page’s examples in order in one IEx session; later blocks reuse the modules and values defined earlier.

Start with a bounded agent

After Installation , this makes a real OpenAI call with bounded retries and a maximum of four model turns:

defmodule MyApp.BoundedAgent do
  use BeamWeaver.Agent

  model "openai:gpt-5.4-mini", max_tokens: 256, timeout: 60_000
  system_prompt "Answer briefly."
  run_timeout 120_000

  middleware do
    use BeamWeaver.Agent.Middleware.ModelRetry, max_retries: 2, initial_delay: 500, max_delay: 5_000, jitter: true
    use BeamWeaver.Agent.Middleware.ModelCallLimit, run_limit: 4
  end
end

{:ok, result} = MyApp.BoundedAgent.invoke(%{messages: ["Explain parcel tracking in one sentence."]})
true = byte_size(result.messages |> List.last() |> BeamWeaver.Core.Message.text()) > 0

This successful request exercises the configured agent; it does not simulate an outage or exhaust the limits. Testing shows how to verify those branches deterministically without forcing provider failures.

Retries and fallbacks

ModelRetry

Retries transient provider failures. Defaults are two retries after the first attempt, zero initial delay, backoff 2.0, a 5,000 ms maximum delay and no jitter. Set a nonzero delay and jitter for a shared production service.

retry_on: accepts :transient, :error, error types or a predicate. :error still respects explicit retryable: false. Authentication, invalid requests and exhausted billing should be corrected rather than repeated.

on_failure: applies after covered retries are exhausted: :error returns the error, :continue ends with an assistant message, and a function builds that message. Errors outside the retry policy propagate unchanged. Failed streaming attempts remain observable as debug events; see Streaming .

ModelFallback

fallbacks: lists model identifiers or constructed model values in order. retry_on: controls which failures trigger fallback; the default covers all. A fallback uses its own credentials/connection settings and only compatible model options from the primary request.

Place fallback before retry to retry each candidate before moving on. Confirm that each fallback supports your tools, media and answer schema. Fallback changes the provider executing the task; it is not proof that the primary provider recovered.

Configure a real primary and fallback in order. This successful call checks the configuration; the failure branch is exercised with a scripted provider in the application testing guide:

defmodule MyApp.FallbackAgent do
  use BeamWeaver.Agent
  model "openai:gpt-5.4-mini", max_tokens: 128, timeout: 60_000

  middleware [
    {BeamWeaver.Agent.Middleware.ModelFallback, fallbacks: ["google:gemini-3.5-flash"]},
    {BeamWeaver.Agent.Middleware.ModelRetry, max_retries: 1, initial_delay: 500}
  ]
end

{:ok, result} = MyApp.FallbackAgent.invoke(%{messages: ["Say hello briefly."]})
true = byte_size(BeamWeaver.Core.Message.text(List.last(result.messages))) > 0

ToolRetry

Takes the retry settings above plus tools: to restrict it to named actions. Its default retry_on: :error covers tool errors except those explicitly marked nonretryable. The final error normally returns to the model as a tool message. Retry a mutating action only if it is idempotent or has a stable application-level operation key.

This complete example makes the local lookup fail once, then succeed. A trusted per-run counter makes the transient failure reproducible while the model call is real:

defmodule MyApp.UnstableLookup do
  use BeamWeaver.Tool
  name "lookup_order"
  description "Read the delivery day for order A-1001."
  injected :context, :context

  schema do
    field :order_id, :string
  end

  @impl true
  def invoke(_tool, %{"order_id" => "A-1001", context: %{counter: counter}}, _opts) do
    attempt = Agent.get_and_update(counter, &{&1 + 1, &1 + 1})

    if attempt == 1,
      do: {:error, BeamWeaver.Core.Error.new(:timeout, "Temporary lookup failure")},
      else: {:ok, "A-1001 arrives Thursday."}
  end
end

defmodule MyApp.RetryingLookup do
  use BeamWeaver.Agent
  model "openai:gpt-5.4-mini", max_tokens: 256, timeout: 60_000
  system_prompt "Use lookup_order once, then report its result."
  tools [MyApp.UnstableLookup]

  middleware [
    {BeamWeaver.Agent.Middleware.ToolRetry, tools: ["lookup_order"], max_retries: 1, initial_delay: 10},
    {BeamWeaver.Agent.Middleware.ToolChoice,
     choice: fn request ->
       if Enum.any?(request.messages, &(&1.role == :tool)), do: :none, else: :required
     end}
  ]
end

{:ok, counter} = Agent.start_link(fn -> 0 end)

try do
  {:ok, result} = MyApp.RetryingLookup.invoke(%{messages: ["Look up A-1001."]}, context: %{counter: counter})
  2 = Agent.get(counter, & &1)
  true = Enum.any?(result.messages, &(&1.role == :tool and BeamWeaver.Core.Message.text(&1) =~ "Thursday"))
after
  Agent.stop(counter)
end

ToolError

on_error: receives the error and tool-call request. Return text for the model, or nil to fail the run. tools: scopes the rule. Invalid arguments and calls to unavailable tools retain their corrective errors. This middleware does not retry. With ToolError outside ToolRetry, it sees the last failure after retries.

Convert the same failed lookup into application-specific feedback. This agent makes only one tool attempt:

defmodule MyApp.ExplainedLookup do
  use BeamWeaver.Agent
  model "openai:gpt-5.4-mini", max_tokens: 256, timeout: 60_000
  system_prompt "Call lookup_order, then explain its result without retrying."
  tools [MyApp.UnstableLookup]

  middleware [
    {BeamWeaver.Agent.Middleware.ToolError,
     on_error: fn _error, _request -> "The order service is temporarily unavailable." end},
    {BeamWeaver.Agent.Middleware.ToolChoice,
     choice: fn request ->
       if Enum.any?(request.messages, &(&1.role == :tool)), do: :none, else: :required
     end}
  ]
end

{:ok, counter} = Agent.start_link(fn -> 0 end)

try do
  {:ok, result} = MyApp.ExplainedLookup.invoke(%{messages: ["Look up A-1001."]}, context: %{counter: counter})
  1 = Agent.get(counter, & &1)

  true =
    Enum.any?(result.messages, &(&1.role == :tool and BeamWeaver.Core.Message.text(&1) =~ "temporarily unavailable"))
after
  Agent.stop(counter)
end

Limits

ModelCallLimit

run_limit: bounds one invocation; thread_limit: spans saved calls on a thread. Neither is configured by default. Thread limits need a reused checkpointer. exit_behavior: :error returns :model_call_limit_exceeded; :end returns an explanatory message, customizable with message:.

A model-turn limit is distinct from token limits, provider attempts inside retry wrappers and graph supersteps. Use a whole-run timeout as well.

A thread budget survives separate invocations when the saver and thread are reused:

defmodule MyApp.TwoTurnAgent do
  use BeamWeaver.Agent
  model "openai:gpt-5.4-mini", max_tokens: 128, timeout: 60_000
  middleware [{BeamWeaver.Agent.Middleware.ModelCallLimit, thread_limit: 2, exit_behavior: :end}]
end

opts = [checkpointer: BeamWeaver.Checkpoint.ETS.new(), config: %{"configurable" => %{"thread_id" => "two-turns"}}]
{:ok, _} = MyApp.TwoTurnAgent.invoke(%{messages: ["Say hello."]}, opts)
{:ok, _} = MyApp.TwoTurnAgent.invoke(%{messages: ["Say goodbye."]}, opts)
{:ok, limited} = MyApp.TwoTurnAgent.invoke(%{messages: ["One more turn."]}, opts)
true = BeamWeaver.Core.Message.text(List.last(limited.messages)) =~ "thread limit (2/2)"

ToolCallLimit

Use run_limit: and/or thread_limit:, optionally for one tool_name:. The run limit cannot exceed a supplied thread limit. The default exit_behavior: :continue rejects excess actions and lets the model explain; :end stops the loop and :error fails it. message: customizes feedback. Use separate instances when individual tools need separate limits.

A zero budget demonstrates the rejection boundary without running the tool handler:

{:ok, agent} =
  BeamWeaver.Agent.build(
    model: "openai:gpt-5.4-mini",
    model_opts: [max_tokens: 256, timeout: 60_000],
    tools: [MyApp.UnstableLookup],
    middleware: [
      {BeamWeaver.Agent.Middleware.ToolCallLimit, tool_name: "lookup_order", run_limit: 0, exit_behavior: :error},
      {BeamWeaver.Agent.Middleware.ToolChoice, choice: :required}
    ]
  )

{:error, error} = BeamWeaver.Agent.invoke(agent, %{messages: ["Look up A-1001."]})
:tool_call_limit_exceeded = error.type

Tools

ToolSelection

Selection changes both model-visible tools and the execution registry.

Model-assisted selection makes an additional call. max_retries: defaults to 1 for malformed selection output. on_parsing_failure: can fail (default), select none/all, or use a supplied list/function. Deliberately choose whether failure should broaden access.

A paused run retains selected tool names. Dynamically created tools from a function are rebuilt on a later model call, not automatically on tool resume; use stable tool entries for actions that must survive review. Agent exclude_tools and tool_descriptions declarations install a selection policy before your explicit one.

An allowlist can expose a read-only lookup while withholding unrelated actions. The selected tool must execute:

read =
  BeamWeaver.Core.Tool.from_function!(
    name: "read_policy",
    description: "Read the refund policy.",
    input_schema: %{"type" => "object", "properties" => %{}},
    handler: fn _, _ -> {:ok, "Refunds take five business days."} end
  )

{:ok, agent} =
  BeamWeaver.Agent.build(
    model: "openai:gpt-5.4-mini",
    model_opts: [max_tokens: 256, timeout: 60_000],
    tools: [read, MyApp.UnstableLookup],
    system_prompt: "Read the refund policy before answering.",
    middleware: [{BeamWeaver.Agent.Middleware.ToolSelection, allow: ["read_policy"]}]
  )

{:ok, result} = BeamWeaver.Agent.invoke(agent, %{messages: ["How long do refunds take?"]})
["read_policy"] = result.messages |> Enum.filter(&(&1.role == :tool)) |> Enum.map(& &1.name) |> Enum.uniq()

ToolChoice

choice: sets :auto, :required, :none, or a provider-supported named choice. It can be a function of the request. when_tool_present: restricts application to matching offered tools. Requiring a tool forever prevents a normal prose answer; apply the requirement only to the turn that needs it.

Choose a tool on the first turn and permit a final answer afterward. This direct call makes the resolved policy visible:

alias BeamWeaver.Core.Message
alias BeamWeaver.Agent.Middleware.ToolChoice
alias BeamWeaver.Agent.ModelRequest

choice =
  ToolChoice.new(
    choice: fn request ->
      if length(request.messages) == 1, do: :required, else: :auto
    end
  )

first_call = ModelRequest.new(messages: [Message.user("Where is my order A-1001?")])
ToolChoice.wrap_model_call(choice, first_call, & &1.tool_choice)
#=> :required

ToolCallNormalization

Agents with tools/middleware already normalize call IDs and JSON argument strings. An explicit instance customizes max_argument_chars: (default 20,000). Normalization repairs representation, not semantics or authorization.

Normalize a provider-shaped call before a tool is looked up. This local example shows both ID repair and argument truncation:

alias BeamWeaver.Core.Message
alias BeamWeaver.Agent.ModelRequest
alias BeamWeaver.Agent.Middleware.ToolCallNormalization
alias BeamWeaver.Agent.ModelResponse

call = %{id: "call 1", name: "add_note", args: ~s({"note": "#{String.duplicate("x", 30)}"})}
response = %ModelResponse{messages: [Message.assistant("", tool_calls: [call])]}
normalization = ToolCallNormalization.new(max_argument_chars: 10)

{:ok, %{messages: [message]}} =
  ToolCallNormalization.wrap_model_call(normalization, ModelRequest.new(), fn _ -> {:ok, response} end)

[call] = message.tool_calls
{call.id, call.args}
#=> {"call_1", %{"note" => "xxxxxxxxxx\n\n... argument truncated at 10 bytes."}}

Prompts and output

DynamicPrompt

prompt: is a string/system message, a function of the request, a function of state and runtime, or an MFA tuple called with the request first. It replaces the base system prompt before other capability prompt sections are added. Validate required caller fields with context_schema.

Use validated context to construct the prompt on each call:

defmodule MyApp.NamedGreeter do
  use BeamWeaver.Agent
  model "openai:gpt-5.4-mini", max_tokens: 128, timeout: 60_000

  context_schema do
    field :customer_name, :string
  end

  middleware [
    {BeamWeaver.Agent.Middleware.DynamicPrompt,
     prompt: fn _state, runtime ->
       "Greet #{runtime.context.customer_name} by name, briefly."
     end}
  ]
end

{:ok, result} = MyApp.NamedGreeter.invoke(%{messages: ["Hello"]}, context: %{customer_name: "Ada"})
true = BeamWeaver.Core.Message.text(List.last(result.messages)) =~ "Ada"

StructuredOutputRetry

Adds the invalid answer and feedback, then retries parsing/validation failures. max_retries: defaults to 1; feedback: is text with {error} or an error function. For the tool strategy, set handle_errors: false if this middleware should own correction rather than the strategy's own retry loop. See Structured Output .

Configure schema correction and execute a real extraction. This input is valid; use the malformed-answer example in Testing to force a retry:

defmodule MyApp.Schemas.Triage do
  use BeamWeaver.Schema

  field :category, :string, enum: ["billing", "shipping", "returns"]
  field :urgent, :boolean
end

defmodule MyApp.SupportAgent do
  use BeamWeaver.Agent

  model "openai:gpt-5.4-mini", max_tokens: 256, timeout: 60_000

  response_schema MyApp.Schemas.Triage

  middleware [
    {BeamWeaver.Agent.Middleware.StructuredOutputRetry,
     max_retries: 2, feedback: "Your answer didn't match the schema: {error}. Answer again."}
  ]
end

{:ok, result} =
  MyApp.SupportAgent.invoke(%{
    messages: ["An urgent shipping issue: my parcel never arrived. Classify as shipping and urgent."]
  })

result.structured_response
#=> %{"category" => "shipping", "urgent" => true}

PII

Detects email, credit-card, IP, MAC-address and URL text. type: chooses a kind; omission checks all built-in kinds. A custom detector: is a regex string or match function. Strategies are :redact, :mask, :hash and :block (:pii_detected).

apply_to_input: defaults to true; apply_to_tool_results: and apply_to_output: default to false. Message-text edits do not scrub tool-call arguments, reasoning, artifacts, structured response maps or raw metadata. stream_transform/1 has a separate scope for emitted events. Guardrails explains those boundaries.

The saved input shows masking before the provider sees it. The number here is a synthetic example:

defmodule MyApp.SupportAgent do
  use BeamWeaver.Agent

  alias BeamWeaver.Agent.Middleware

  model "openai:gpt-5.4-mini", max_tokens: 128, timeout: 60_000

  middleware [
    {Middleware.PII, type: :credit_card, strategy: :mask},
    {Middleware.PII, type: :url, strategy: :block, apply_to_input: false, apply_to_output: true}
  ]
end

{:ok, result} = MyApp.SupportAgent.invoke(%{messages: ["My card is 4111 1111 1111 1111"]})
result.messages |> hd() |> BeamWeaver.Core.Message.text()
#=> "My card is **** **** **** 1111"

Context size

ContextEditing

After trigger: tokens, replace older tool results while keeping keep: recent results. Saved state is unchanged. Options include exclude_tools:, clear_tool_inputs:, placeholder:, token_count_method: and a custom token_counter:. clear_tool_uses/2 applies the transformation directly.

Transform a completed tool exchange locally to inspect exactly which result is hidden:

alias BeamWeaver.Core.Message
alias BeamWeaver.Agent.Middleware.ContextEditing

messages = [
  Message.user("Where are A-1001 and A-1002?"),
  Message.assistant("", tool_calls: [%{id: "call_1", name: "lookup_order", args: %{"order_id" => "A-1001"}}]),
  Message.tool("Order A-1001 shipped on Monday.", tool_call_id: "call_1"),
  Message.assistant("", tool_calls: [%{id: "call_2", name: "lookup_order", args: %{"order_id" => "A-1002"}}]),
  Message.tool("Order A-1002 shipped on Tuesday.", tool_call_id: "call_2")
]

messages |> ContextEditing.clear_tool_uses(trigger: 0, keep: 1) |> Enum.map(&Message.text/1)
#=> ["Where are A-1001 and A-1002?", "", "[cleared]", "", "Order A-1002 shipped on Tuesday."]

Summarization

Replace old saved messages with a summary. model: or summarizer: creates it; trigger: and keep: define the window. Message, token and fractional limits are supported; fractional middleware limits require max_input_tokens:. A list of triggers means any; {:all, triggers} means all. on_overflow: false disables overflow-triggered recovery. Context Management provides an example that actually triggers and checks a summary.

This deliberately small threshold triggers a real summary and verifies that the order fact remains available:

defmodule MyApp.SummarizingAgent do
  use BeamWeaver.Agent

  model "openai:gpt-5.4-mini", max_tokens: 256, timeout: 60_000
  system_prompt "Acknowledge supplied facts briefly. Retain order IDs and delivery dates."

  middleware [
    {BeamWeaver.Agent.Middleware.Summarization,
     model: BeamWeaver.Models.init_chat_model!("openai:gpt-5.4-mini", max_tokens: 512, timeout: 60_000),
     trigger: {:messages, 6},
     keep: {:messages, 2}}
  ]
end

saver = BeamWeaver.Checkpoint.ETS.new()
opts = [checkpointer: saver, config: %{"configurable" => %{"thread_id" => "summary-demo"}}]

for fact <- ["Order A-1001 arrives Thursday.", "The delivery address is Lisbon.", "Contact me by email."] do
  {:ok, _} = MyApp.SummarizingAgent.invoke(%{messages: [fact]}, opts)
end

{:ok, result} = MyApp.SummarizingAgent.invoke(%{messages: ["What is the order ID and delivery day?"]}, opts)
summary = result.messages |> hd() |> BeamWeaver.Core.Message.text()
true = String.starts_with?(summary, "Conversation summary:")
true = summary =~ "A-1001"
true = length(result.messages) < 8
answer = result.messages |> List.last() |> BeamWeaver.Core.Message.text()
true = answer =~ "A-1001"
true = String.downcase(answer) =~ "thursday"

summarize/2 applies the same trigger: and keep: to a message list outside a run. It returns {:ok, %{summary: message, kept: messages}}, :unchanged when the trigger isn't reached or nothing can be summarized, or {:error, error}:

alias BeamWeaver.Core.Message
alias BeamWeaver.Agent.Middleware.Summarization

summarization =
  Summarization.new(
    summarizer: fn old -> "The customer asked about #{length(old)} orders." end,
    trigger: {:messages, 4},
    keep: {:messages, 1}
  )

history = for id <- ["A-1001", "A-1002", "A-1003", "A-1004"], do: Message.user("Where is order #{id}?")
{:ok, %{summary: summary, kept: kept}} = Summarization.summarize(summarization, history)
Message.text(summary)
#=> "Conversation summary:\nThe customer asked about 3 orders."
Enum.map(kept, &Message.text/1)
#=> ["Where is order A-1004?"]

Documented elsewhere