Middleware performance and latency

Paybond agent middleware adds policy enforcement and evidence around side-effecting tool calls. For architects sizing agent workloads or comparing guarded vs unguarded tool latency, the important fact is simple: network round trips dominate — not local policy parsing or evidence JSON construction.

This guide describes the typical online path per guarded tool call, what adds latency, what the dev trace dashboard does not measure, and how to observe middleware overhead honestly in production. For integration mechanics, see Agent middleware. For the trace event model, see Middleware trace.

What middleware adds

Middleware work splits into two buckets:

For fast tools (cache lookups, lightweight API calls), Paybond hops often exceed handler time. For slow tools (human approval, large file uploads, multi-second vendor APIs), middleware overhead is usually a smaller fraction of end-to-end latency.

Per guarded tool call — typical online path

The interceptor in kit/ts/src/agent/interceptor.ts (Python parity in paybond_kit.agent) runs a serial cycle for each registered side-effecting tool:

Phases in order:

1. Authorize — POST /verify through Gateway. Harbor validates the capability token and spend amount; Gateway may apply spend-policy locks and idempotency in Postgres before/after the Harbor forward.
2. Execute — Your tool handler. Paybond does not instrument vendor latency beyond durationMs on the tool_executed trace event.
3. Complete — POST /v1/spend/decisions/{decision_id}/complete marks the spend decision consumed (or released on handler failure). Stripe MPP session mode may add an internal Harbor voucher finalize hop during completion.
4. Evidence — Auto-evidence after a successful handler:
   • Sandbox bind — Gateway-composed guardrails evidence route (simulator lifecycle).
   • Production attach — Local payee + agent-recognition signing, then POST /harbor/intents/{id}/evidence with Gateway recognition verification and fraud hardening before Harbor proxy.

Settlement confirm (released / refunded) is a separate Harbor mutation — not included in per-tool middleware latency unless your integration calls it inline after each tool.

Latency factors

Qualitative overhead model

Without publishing fabricated percentiles:

• Expect roughly one RTT per Paybond phase on healthy hosted infrastructure — often ~20–150ms per hop within the same region, plus handler time.
• Multi-hop serial calls dominate for sub-100ms tools.
• Transient upstream errors can add several seconds of SDK retry backoff on tail paths.
• Signal lifecycle latency (intent outcome → reputation) operates on hours scale — not per-tool-call middleware time.

Authorization cache (split approve / execute)

Some agent frameworks separate tool approval from tool execution (for example Vercel AI SDK human-in-the-loop). Paybond caches a successful authorize result for 120 seconds (AUTHORIZATION_CACHE_TTL_SEC in kit/ts/src/agent/authorization-cache.ts).

The cache key is toolCallId:operation. A cached entry is reused in wrapExecute only when operation, requestedSpendCents, and toolName still match. Expired entries are evicted on the next authorize.

When it helps: approve now, execute seconds later without a second /verify.

When it does not apply: single-shot wrapExecute without a prior authorizeToolCall always calls /verify once. Mismatched spend or renamed tools miss the cache.

What dev trace shows — and what it does not

paybond dev trace and traceSink events label middleware phases, not network profiling.

The durationMs on tool_executed is your local handler, not Paybond authorize or evidence latency. Use the trace dashboard to debug whether each phase ran and in what order — see Middleware trace — not to size production p99.

Hosted sandbox smoke may print trace_url and console dossier links; those surfaces still omit per-hop network timing until you add your own instrumentation around Kit calls.

Production observability

Kit runs in your infrastructure. Paybond observes middleware from the Gateway and Harbor side:

Recommended partner pattern:

1. Wrap guarded tool entry/exit in your APM (OpenTelemetry, Datadog, etc.) with intent_id, operation, and tool_call_id attributes.
2. Alert on Gateway→Harbor SLIs from Production operations v1 §3.2.
3. Treat predicate_passed: false as a business outcome — not a retryable transport error (Disputes and evidence).

Reducing perceived latency

Practical patterns that preserve guardrails:

• Fewer side-effecting tools — consolidate paid vendor calls behind one guarded operation when policy allows.
• Read-only tools outside the registry — do not register passthrough tools that never spend.
• Pre-validate evidence — paybond policy validate-evidence catches schema issues before production attach signing.
• Healthy idempotency keys — reuse stable keys per toolCallId so retries do not multiply Harbor work.
• Regional co-location — run agents in the same region as your Gateway deployment when possible.
• Offline rehearsal — paybond dev loop --offline for policy and trace UX without network variance.

Where to go next