Back to Blog
Brokerage

When Your LP Goes Dark: A Practical Liquidity Incident Runbook for Brokers

Amira KhalidAmira Khalid
May 3, 20266 min read24 views
When Your LP Goes Dark: A Practical Liquidity Incident Runbook for Brokers

Liquidity incidents aren’t rare—they’re just rarely documented well. One LP feed stalls, spreads blow out, or a single bad tick prints across your symbols, and suddenly you’re juggling execution risk, client trust, and potential regulatory scrutiny.

This post is a practical broker runbook for three common scenarios—LP outages, price spikes, and stale quotes—focused on roles, logs, and client comms. The goal isn’t to “avoid all incidents” (you can’t); it’s to respond consistently, preserve evidence, and reduce disputes.

1) Define the incident types and trigger thresholds (before you need them)

Most liquidity incidents become messy because the team debates definitions mid-firefight. Put clear categories and triggers in writing, aligned across dealing, ops, support, and compliance.

Common incident types to define:

  • LP outage / disconnect: FIX session down, bridge disconnect, or liquidity venue unreachable.
  • Stale quotes: market data timestamp drift, frozen bid/ask, or last price not updating while other sources move.
  • Price spike / bad tick: outlier price print that is not corroborated by other LPs/venues and causes abnormal fills, stops, or margin events.

Practical trigger examples (tune to your setup and instruments):

  • Stale quote trigger: no update for X ms/seconds on a “fast” instrument during active session, or last-update time exceeds your normal baseline by Yx.
  • Outlier trigger: top-of-book mid deviates by more than N pips or N standard deviations vs. median of other feeds, or spread widens beyond a defined percentile.

Document what happens at each trigger: alert only, throttle, symbol-by-symbol protection, or full “trade protect” mode. Keep it jurisdiction-aware—execution policies and client disclosures vary, so check local regulations and align with your legal/compliance advisors.

2) Assign roles: who decides, who executes, who communicates

In liquidity incidents, speed matters—but so does avoiding contradictory actions (e.g., dealing disables a symbol while support tells clients “everything is fine”). Define a small incident cell with clear authority.

A simple role model that works for many brokers/prop firms:

  • Incident Commander (IC): usually Head of Dealing or Trading Ops lead. Owns decisions and timeline.
  • Liquidity/Bridge Engineer: validates connectivity, FIX sessions, bridge health, routing, and failover.
  • Risk Owner: monitors exposure, A/B routing behavior, margin impacts, and hedging status.
  • Client Comms Lead: usually Support/CS manager. Owns status page/email/in-app messaging.
  • Compliance/QA Reviewer: ensures actions follow execution policy, disclosure, and recordkeeping.

Two practical rules:

  1. One decision-maker for “pause trading / symbol protect / switch LP / roll back config.”
  2. One voice to clients. Internal chat can be noisy; external messaging must be consistent.

3) The first 10 minutes: stabilize execution before you investigate

Your first objective is to stop the bleeding—reduce the chance of erroneous fills, cascading stop-outs, and avoidable disputes. Investigation is second.

A tight “first 10 minutes” checklist:

  • Confirm scope: which symbols, which server, which account groups, which LP(s).
  • Check if this is market-wide volatility vs. a single-feed issue (cross-check against backup feeds).
  • Apply controls (pick the least disruptive that reduces harm):
    • Switch routing to backup LP(s) / venue(s)
    • Increase markups/filters temporarily (within policy)
    • Enable max spread / max deviation / outlier filters
    • Throttle or reject orders that exceed defined slippage bands
    • Place affected symbols into close-only or trade protect mode
  • Freeze configuration changes: log who changed what, and when.

If you run hybrid execution, verify what’s happening on both sides:

  • Are A-book orders failing over cleanly?
  • Is B-book pricing still valid (or are you internalizing against bad ticks)?
  • Is hedging automation firing incorrectly due to corrupted prices?

4) Evidence pack: the logs you’ll wish you had during disputes

Client complaints and regulator questions usually arrive after the incident—when memory is fuzzy. Your runbook should define an evidence pack you can assemble quickly and consistently.

Minimum log set (store with timestamps in UTC and keep retention aligned to policy):

  • Market data logs: tick-by-tick or aggregated quotes, including LP source, timestamps, and sequence gaps.
  • Bridge/aggregator logs: LP connections, reject codes, last look outcomes (if applicable), routing decisions, failover events.
  • Platform logs (MT4/MT5/cTrader/etc.): order lifecycle, execution reports, dealer interventions, symbol settings changes.
  • Risk logs: exposure snapshots, hedging actions, margin changes, stop-out events, and risk limit breaches.
  • Config audit trail: who changed markups, filters, symbol trading status, group settings, or routing rules.

A practical tip: define a single “incident folder” naming convention, e.g. INC-YYYYMMDD-<type>-<scope>, and require the IC to assign an incident ID early. That ID should appear in internal tickets, client comms drafts, and post-mortem notes.

5) Scenario playbooks: LP outage, stale quotes, price spikes

Treat these as three separate playbooks. They overlap, but the right control differs.

A) LP outage (disconnect / no liquidity)

  • Immediate action: fail over to other LPs, or reduce instruments to those with reliable liquidity.
  • Watch for: partial connectivity (market data up, execution down), asymmetric liquidity, or one-sided quotes.
  • Client impact focus: execution delays, re-quotes, widened spreads.

B) Stale quotes (frozen feed / timestamp drift)

  • Immediate action: stop publishing stale prices to clients (filter by age), switch to backup feed.
  • Watch for: stale feed still “looks tight,” attracting toxic flow and causing losses/disputes.
  • Client impact focus: fills at prices that were not available in the market at that time.

C) Price spikes / bad ticks (outliers)

  • Immediate action: enable outlier detection (median-of-feeds checks), widen filters, or temporarily disable affected symbols.
  • Watch for: stop-loss cascades, margin calls, and “free money” arbitrage patterns.
  • Client impact focus: stop-outs and triggered pending orders on erroneous prints.

For all three, decide in advance how you handle contested fills. Some brokers will review and adjust clearly erroneous executions; others will follow strict “as executed” rules. Whatever your approach, it must match your execution policy and client terms—and again, check local regulations.

6) Client communications: fast, factual, and consistent (with templates)

Silence creates tickets; vague statements create accusations. Your comms should be timely, factual, and avoid over-promising. Don’t speculate about LP names or “fault” until you confirm.

A simple comms cadence:

  • T+0–15 min: acknowledge issue + scope + next update time.
  • T+30–60 min: status update + mitigations applied (e.g., failover, symbol protection).
  • Resolution: confirm stability + what happens next (review process, dispute channel).

Template: first notice (short)

  • “We’re currently investigating an execution/price feed issue affecting [symbols / platform] starting at [UTC time]. Mitigations have been applied and we will provide an update by [UTC time].”

Template: resolution notice (short)

  • “The issue affecting [symbols / platform] has been resolved as of [UTC time]. If you believe your execution was impacted, please contact support with ticket ID + order IDs for review under our execution policy.”

Operationally, ensure support has:

  • a one-page internal FAQ (what to say / not say)
  • a list of known affected symbols and time windows
  • a clear escalation path to the incident cell

The Bottom Line

A liquidity incident runbook is less about perfect prevention and more about repeatable control: clear triggers, a single decision-maker, fast stabilization steps, and an evidence pack that stands up in disputes.

If you can’t reconstruct “what happened” from logs in under an hour, your next incident will cost more than it should.

If you want help tightening your liquidity ops stack—risk controls, platform management, and audit-ready workflows—talk to Brokeret at /get-started.

Share:TwitterLinkedIn