Replaying Kalshi Order Fill Events to Validate Automated Strategy Logic

Testing a Kalshi bot before going live requires a stateful replay environment that reconstructs captured order fill sequences. Kalshi cannot retroactively replay missed WebSocket deltas, and fill-related status transitions are non-monotonic. Validating automated strategy logic therefore means testing its response to the cumulative sequence of book changes, fills, and conflicts.

Why Kalshi's WebSocket architecture makes fill replay a structural requirement, not an optimization

A WebSocket reconnect can leave a gap in the event sequence that Kalshi cannot retroactively replay. Reconnecting does not recover the missed causal history. For strategy validation, that gap separates knowing an order’s current state from knowing how the strategy would have responded to each preceding change.

The lifecycle itself creates a second constraint. Orders have distinct states, including resting, canceled, and executed, but related trigger status does not move only forward. An order-anchored trigger returns to pending_on_entry after closing its current quantity because the anchor order can still accumulate fills. Strategy logic must distinguish that reactivation from a new position, while also respecting the rule that reusing a client_trigger_id for a different position returns a 409.

The event representation also affects faithful reconstruction. Kalshi’s API Changelog records that, on October 22, 2026, user_orders began truncating created_time, last_update_time, and expiration_time to milliseconds without rounding. The changelog’s October 29, 2026 update adds a clock field to order book snapshot and delta messages that is identical across parallel connections. These details govern timestamp precision and cross-connection matching, so a replay cannot treat event metadata as incidental.

Why stateless mocks cannot exercise fill-state transitions at all

Consider a strategy that creates an order, polls until it is filled, and then attempts to cancel it. In a stateful environment, creation establishes an order object, subsequent fills update that object, and cancellation encounters its filled state. The correct response is a 409 conflict. A stateless mock instead returns whatever response its fixture assigns to the cancellation request, without deriving that response from the preceding fills.

A canned 409 can exercise code that handles the response while leaving untested whether the response follows from the order’s accumulated state. The distinction is between recognizing an error and validating the conditions that produce it. If earlier calls do not change what later calls encounter, the test never exercises the full create-to-conflict sequence.

That limitation extends to fill counts and trigger reactivation. Both depend on what happened before the current request, which a stateless fixture does not remember. A suite can therefore pass while leaving the production state sequence untested. The failure is silent because individual responses may look correct even when the test has no model connecting them.

What fill data is available for capture and what its limits are

Kalshi’s live REST tier exposes individual fills through GET /portfolio/fills, while older fills require access to the historical tier. Plans for capture must take that boundary into account from the start. A later request for fill history is not a substitute for preserving the book and status context around an earlier decision.

Kalshi BackTest provided historical order book snapshots at ~100ms resolution for BTC, ETH, SOL, DOGE, and XRP crypto markets, replayable by timestamp. New registrations and purchases are now closed. Existing users retain their current access, but paid access ends at the current billing period end and subscriptions will not renew. That closure makes it unavailable as a durable source for a new capture plan.

Predexon offers Kalshi markets, trades, and orderbook history over REST on a normalized cross-venue schema. Its tick-level data is downloadable as compressed Parquet and finalized daily through the previous day. A free tier is available without a credit card.

Trades describe matches. Historical books also contain displayed prices and sizes at levels that never traded, so trade-only validation systematically overestimates fill probability in thin markets. Sam Marriott’s SSRN paper, posted April 2026, documents the difficulty of reconstructing full Kalshi limit order books from WebSocket streams, including KDB+/q implementation challenges requiring non-standard reconstruction patterns. Durable capture consequently requires deliberate infrastructure rather than opportunistic access to whichever records remain available.

How to structure a fill-event capture pipeline that a replay environment can consume

To build that infrastructure, the capture model should preserve the full causal chain around each order. It must include the resting bid/ask ladder and available size at each level when the order was placed, followed by fill events as they arrived on user_orders and the resulting status at each step. The final outcome is not enough for reconstruction. The model should keep trades, candles, current book snapshots, and recorded historical books separate so that displayed liquidity is not mistaken for execution.

Cross-connection reconciliation uses the clock field on orderbook_snapshot and orderbook_delta as a shared reference because it is stable across connections, while seq is connection-local. Observation timestamps should also be kept separate from market settlement fields. Otherwise, a resolved outcome can leak into a row used to reconstruct an earlier decision point.

Native yes/no price and size fields should use integer or exact decimal handling without floating-point conversion so that price units do not drift over long replays. Series, event, and market tickers should be kept with the raw book because a title-only join can silently mix different expiration windows when contracts roll or new thresholds appear.

Recorded responses contain the full response body, including balances, positions, and any PII present in the account. The fixture directory must always be covered by .gitignore.

What a stateful Kalshi simulator must maintain to replay fill sequences correctly

A replayable dataset supplies events, but the simulator must turn those events into persistent state. It must remember created order objects and update them as fills arrive, so later requests encounter the consequences of earlier actions. Conflict responses must be based on that stored state. This includes a 409 for canceling a filled order and for reusing a client_trigger_id for a different position, rather than unconditional responses attached to request fixtures.

Trigger state needs its own faithful lifecycle. When an order-anchored trigger closes its quantity and returns to pending_on_entry, the simulator must represent that return. Otherwise, status-reading logic sees a simplified lifecycle that excludes reactivation. Correct order objects alone do not cover that behavior.

The book at the start must also look like the setting that was captured. An empty sandbox cannot exercise slippage logic or fill-probability estimation because there is no realistic resting liquidity for the strategy to encounter. Pre-seeded book data puts that liquidity in place before replay starts. The simulator must also support injected latency, 429s, and 503s during the fill sequence so tests can exercise rate-limit handling and recovery from transient failures.

Behavioral fidelity is not a one-time implementation task. The October 2026 changes span message fields, timestamp handling, and portfolio endpoint behavior, each of which can alter what the strategy observes. A simulator that is not re-verified against relevant live API changes can drift while its own tests continue to pass.

How do you test a Kalshi bot with a replay suite?

A test author should seed the simulator with the captured book state immediately before order placement, submit the order as the strategy would in production, and then feed captured fill events into the simulator in sequence. Assertions belong at each state transition. Each case should name an order_id and include the status history for that order. That anchor ties a failure to an exact event in the trace.

The partial-fill case should keep the order in its own stable partially_filled state before it reaches filled to test whether the strategy handles incomplete execution. The trigger-reactivation case should then replay a return to pending_on_entry. Its assertion should establish that the strategy does not interpret reactivation as a new fill requiring a new position.

The fill-then-cancel case tests a different decision. After replaying the fill, the test should assert that the strategy handles the resulting 409 as a non-retriable conflict. These cases test responses to state as well as response parsing. They also keep separate the handling of incomplete execution, reactivation, and an action that the order’s current state no longer permits.

For CI, the replay suite should run on every pull request. The simulator runs entirely offline and needs no live credentials, exchange connection, or rate-limit budget. Live verification belongs in a separate drift check after any changelog entry touching order, fill, or portfolio endpoints, or on a scheduled cadence where production secrets are available.

How rate-limit changes on the portfolio endpoints affect replay test design

The changelog records that GET /trade-api/v2/portfolio/balance is now metered in its own rate-limit bucket, separate from the Read budget, with call rates that differ by tier. A strategy that frequently checks balance during a fill sequence must therefore be tested against independent bucket depletion.

Order polling has a separate behavioral change. GET /portfolio/orders now accepts up to 100 comma-separated tickers in event_ticker, up from 10, and filtering on events with no markets has changed from returning every order to returning no orders. These changes affect tests that poll across more than one event. A simulator that models the relevant filters and rate-limit buckets lets engineers exercise backoff and retry behavior without consuming live API budget or triggering limits on authenticated accounts.

What fill replay enables for RL-based strategy agents on Kalshi

An RL-based strategy agent encounters the same order and trigger lifecycles as deterministic strategy code. If its training environment omits the return to pending_on_entry or accepts cancellation after a fill without the appropriate conflict, the learned policy is calibrated against a simplified world. The stateful fill model must sit beneath the learning interface. It determines which actions change state and which encounter exchange constraints.

Gymnasium’s step API returns (observation, reward, terminated, truncated, info) and provides a standard substrate for RL environment prototyping. It is not itself a training framework. A separate library, such as RLlib, CleanRL, or Stable-Baselines3, runs training, while the environment must supply the fill model.

A stateful Kalshi simulator provides a suitable inner layer because it is deterministic, maintains controllable state, and enforces conflict rules. It can also scale horizontally by running across parallel training containers without live credentials or rate-limit budgets. Captured book conditions and fill sequences give those environments state transitions they can replay. The training interface changes how actions are selected, but not the exchange constraints those actions face.

Fault injection should be a first-class training feature. Latency spikes, 429s, and partial-fill hangs expose the policy to disrupted execution. An agent that has never encountered a 429 during a fill sequence will not have learned robust recovery from that condition. Stateful replay makes those interruptions part of the same controlled sequence as the fills, so recovery behavior can be trained and evaluated without substituting a simpler lifecycle for Kalshi’s own.