When a provider degrades - node sync lag, gas spikes, partial outages - transactions stack up in the mempool. This agent correlates 5 data sources simultaneously and presents a ranked routing recommendation for human approval.
Concept stage. Built as part of a payment processing AI talk. Click any step to see what it does.
Human approval before any routing change. The agent handles correlation and recommendation across 5 sources. The engineer makes the final call. Both outcomes feed the provider scoring model.
Monitors the transaction queue for payments stuck in pending state beyond a defined threshold, typically 5 minutes. A transaction that entered the mempool and has not been confirmed indicates a problem at the network or provider level. This is the trigger that starts the entire pipeline.
The threshold is configurable. A lower value catches problems faster but increases noise. 5 minutes is a reasonable starting point for most crypto payment use cases.
Three Prometheus queries run together: mempool wait time (average time from submission to confirmation), node sync lag (how many blocks behind the current provider is), and block confirmation delay (time between submission and first block confirmation).
These three metrics characterise whether the problem is with the provider specifically or with network load overall. A synced provider on a congested network looks different from a degraded provider on a healthy network, and they require different responses.
Health checks against each configured RPC provider, typically Infura, Alchemy, and a self-hosted local node. Each endpoint is queried for current response latency, recent error rate, and sync status.
The point is not just is our current provider broken but what are the alternatives right now. All providers are checked simultaneously so the agent has a comparison table to work from, not just a single data point about the failing provider.
Current gas prices from public feeds for each target network. High gas prices can cause transactions to stall even when the provider is healthy. The transaction is valid but will not be picked up until the fee meets market rate.
This data informs two things: whether increasing the gas fee (replace-by-fee) would resolve the stuck transactions independently of a provider switch, and whether gas conditions on alternative networks make routing there viable.
Programmatic check of each provider's public status page for open incidents or ongoing degradations. A provider may show acceptable latency on its health endpoint but have a known partial degradation on its status page, particularly for region-specific or transaction-type-specific issues.
Historical incidents from the past 24 hours are also pulled. A provider that resolved an incident 4 hours ago is a different risk profile than one with a clean 30-day history.
The only inference step in the pipeline. The agent receives the full data set - Prometheus metrics, provider health comparisons, gas price feeds, status page summaries - and produces a ranked recommendation.
The output is not a data dump. It is a structured assessment: which provider to route to, why, what the identified risks are, which providers were ruled out and on what basis. The recommendation includes evidence rather than just a conclusion.
Human confirmation via Slack. Two paths.
Gateway routing config is updated to point at the recommended provider. The stuck transactions are flagged for RBF review in a follow-up message. Decision is logged with full context.
The engineer's reasoning is captured and stored as a labeled training signal. If they route to a provider not in the recommendation, that becomes a high-value counter-example; the hardest cases to handle well are the ones where the obvious recommendation was wrong.
Gateway routing config is updated to point at the recommended provider. The stuck transactions are flagged for RBF review in a follow-up message. Decision is logged with full context.
The engineer's reasoning is captured and stored as a labeled training signal. If they route to a provider not in the recommendation, that becomes a high-value counter-example; the hardest cases to handle well are the ones where the obvious recommendation was wrong.
Every routing decision is logged with: the data snapshot that triggered the alert, the full ranked recommendation, the outcome (approved provider or denial reason), the approving engineer's identity, and a timestamp.
This creates the audit trail needed for postmortems and compliance, and the labeled dataset needed to improve provider scoring over time. Both approved and denied decisions are equally important records.
Approved and denied decisions update the provider scoring model. An approved switch that resolves the incident increases that provider's reliability score for similar conditions. A denied switch with a reasoning label teaches the model about cases where its recommendation would have been wrong.
Over time, particularly for edge cases like partial degradation or time-of-day patterns in provider reliability, the scoring improves. Early recommendations will be rougher; later ones will be better calibrated.