Observable Assertion Layers Serious Quants Run Against a Kalshi Bot Before First Live Fill

Testing a Kalshi bot before its first live fill means proving the stateful envelope around every order. Use demo to verify API plumbing, a controlled stateful simulator to assert order-lifecycle correctness and fault handling, and live capital to evaluate strategy only after those checks pass. A backtest cannot substitute for assertions about restarts, fills, signing, market closures, and ambiguous order acknowledgments.

Why the plumbing layer decides whether a Kalshi bot survives first contact with live capital

BotCookery’s 12-trade build lost money despite a majority win rate. Fees were absent from the original algorithm and wiped out its gains, leading the project’s postmortem to recommend including fees from day one. Winning trades were not enough to produce a gain. The same distinction applies to execution, where sound signal logic cannot prevent a bot from buying an already-filled ticker again or mistaking a timed-out acknowledgment for a rejected order.

Restart behavior, fill acknowledgment, market-close handling, authentication, and retry decisions depend on what happened before the current request. They need tests that preserve that history.

What Kalshi's API envelope actually looks like before a bot touches it

Kalshi has separate production and demo environments, each with its own credentials. Credentials do not transfer between the two environments. Order creation uses POST /portfolio/events/orders, called Create Order V2. The legacy /portfolio/orders endpoint entered a deprecation path with an initial commitment of no earlier than May 6, 2026, later revised to no earlier than May 21, 2026.

The V2 order body requires ticker, side, count, price, and self_trade_prevention_type. The side is bid or ask, count is a string, and price is a fixed-point dollar string. Optional fields include time_in_force, client_order_id, post_only, reduce_only, and cancel_order_on_pause. The March 2026 format transition is a concrete schema boundary. Legacy fields were deprecated March 5 and removed from REST and WebSocket payloads on March 12, 2026, with prices represented as strings such as "0.6500" and quantities using count_fp strings such as "10.00".

The order book’s two arrays contain YES bids and NO bids, so the implied YES ask is 1.00 - best_NO_bid. Without NO bids, there is no implied YES ask. Rate limiting uses independent Read and Write token buckets, with per-request costs that vary by endpoint. GET /account/endpoint_costs supplies non-default costs, while documented 429 responses omit Retry-After, leaving backoff to the client.

Authentication supports RSA-PSS/SHA-256 and Ed25519, with the signing path determined by key type. The signed string concatenates a millisecond timestamp, HTTP method, and full URL path excluding query parameters. The exact path is part of the signature. A wrong path or included query string can therefore produce an authentication failure that resembles a credential problem. A trading key needs read and write::trade scopes, while omitting write::transfer restricts its ability to transfer funds.

Why stateless mocks cannot exercise the bugs that actually appear in a Kalshi bot

BotCookery’s rebuild log shows that the bot bought a filled order again. The original rule treated any ticker with a resting order as already ordered, which addressed restarts but missed fills. Once filled, the order was no longer resting. On the next loop, the ticker appeared untouched and the bot bid again. The repair persisted ordered tickers in state.json and skipped tickers for which GET /portfolio/positions showed a position.

A positions fixture that remains unchanged after an order fills cannot test whether the bot reconciles its next decision with the resulting position. Static mocks give the same response each time, while conditional mocks change the response based on the request shape but may not keep track of past requests. Stateful mocks retain data between requests, allowing create-then-read-then-cancel behavior. The distinction is between validating a request and validating a workflow.

That distinction imposes a cost on the test suite. Stateful tests introduce dependencies, require cleanup, and can increase runtime compared with fast, parallel stateless checks. But assertions about one request at a time cannot test restart reconciliation and fill suppression. Kalshi’s demo is stateful, but it does not provide controlled fill injection or scenario replay and requires real environment credentials. Kalshi also warns that demo market prices and behavior may not reflect real markets, limiting what a demo result establishes.

The eight assertion categories a Kalshi bot's stateful test layer must cover

Fill acknowledgment and reorder suppression. Assert that a resting order’s transition to filled does not cause another order on the same ticker in the next loop. The simulator must remove the resting order and reflect the fill in positions. Otherwise, absence from resting orders can masquerade as permission to buy. BotCookery’s persisted ticker record and position cross-check address that exact failure.

Restart and position reconciliation. Assert that a cold restart reads existing positions and resting orders before any placement decision. The bot must not interpret restart as a clean slate. Without reconciliation, it can add to positions it already holds, leaving the order group fill limit as the only backstop.

Market-close handling. Assert that an active-to-closed transition stops new placements and reconciles resting orders against the closed market. A rejection from a closed market must not send the bot into an error loop.

Auth signing correctness. Assert the timestamp, method, and full path composition against a known key before integration tests run. The signed string must leave out query parameters. The unit check must exercise RSA-PSS/SHA-256 or Ed25519 as appropriate, while environment checks establish that credentials and destination host match. Without these checks, a signing defect can look like invalid credentials.

Schema correctness after the March 2026 change. Assert that prices are fixed-point dollar strings and quantities use the required string format, including count_fp fields. The assertion must fail if an outgoing order has integer-cent values. Otherwise, a fixture in the old format can pass requests that the live API rejects.

Rate-limit and retry behavior. Assert that an injected 429 makes the bot wait before a retry is allowed and that each call has a limit on retries. BotCookery’s hardened policy retries GETs only, never POSTs. It also ends the loop early after network failures on two consecutive tickers. Without these constraints, retry logic can prolong a failing loop or repeat a placement whose outcome is uncertain.

Order group and hard-limit enforcement. Assert that every order has a fresh UUID client_order_id that is never reused and the order group ID it should have. When fills exceed the configured contracts limit within a rolling 15-second window, all resting orders in the group must cancel. The daily contract counter must survive restarts as well. A counter that resets cannot enforce a daily allowance.

Assert that the implied YES ask equals 1.00 - best_NO_bid. With no NO bids, the bot must skip trading. Otherwise, an empty side of the book can become a fabricated executable price. The test must cover both a populated book and the no-bid case.

How do you test a Kalshi bot in CI with a simulator

The simulator must keep track of the full order lifecycle from one request to the next. It needs to keep resting orders, mark an order filled on demand, and let position queries read the position that the fill created. State must remain controllable throughout the test run. That control makes it possible to restart the bot against an existing position or close a market between two calls on demand.

The CI environment also needs to be local and credential-free. Production or demo API keys in CI variables introduce an attack surface that a local simulator does not require. This setup also lets the tests run offline. This setup separates authentication correctness, tested against a known key, from possession of credentials capable of reaching an exchange account.

Schema fidelity requires its own maintenance discipline. A simulator must track the live API’s wire format through continuous verification, including checking that an OpenAPI document is current. The March format change illustrates the risk. If both bot and simulator accept the same obsolete representation, a passing integration test establishes agreement between them and leaves compatibility with Kalshi unverified.

Fault controls and seeded data complete that environment. Responses such as 429s, 503s, and delays must be selectable per request, so a fault can be followed by normal behavior. Seeded markets should include active, closed, and no-bid cases, with positions already held. Empty state leaves reconciliation largely untested. BotCookery’s rebuild used “live demo markets with a mock broker — no keys, and no order was sent anywhere,” describing a request-logging, response-producing, state-persisting pattern that a verified simulator formalizes.

Fault injection scenarios specific to the Kalshi order lifecycle

Authentication failure during an active session. Inject a 401 Unauthorized with a realistic error envelope after orders have already been placed. Assert that the bot stops repeating failed authentication, surfaces a needs-reauth state, and logs the cause. Earlier successful requests do not cover this case.

A 429 burst during write-heavy activity. Inject 429 responses on the first two POST /portfolio/events/orders calls, then allow the third. Assert exponential backoff and exactly one eventual placement, without automatic POST retries. These are distinct placement decisions. The test must establish both the wait and the absence of duplication, because recovery from throttling is not successful if it creates an extra position.

A partial or empty order book. Inject a book with no NO bids for the selected ticker. Assert that the bot produces no implied YES ask and skips the ticker without error. Neither zero nor 1.00 is an acceptable substitute.

A market closing during an active loop. Inject the active-to-closed transition between the price-fetch call and placement. Assert that the resulting rejection, such as a 400 or equivalent, is logged and that processing moves to the next ticker without an error loop. This scenario tests the gap between observing a tradable market and attempting to trade it.

A slow or timed-out order acknowledgment. Inject a configurable delay of 8–20 seconds in the order-creation response. Assert that the bot places no second order while it waits for acknowledgment, follows its timeout, and logs the ambiguous outcome without assuming the order was rejected. This is the highest-risk fault path in the set. Retrying a timed-out POST can silently double a position, making it worse than a missed trade. The required outcome is a known, logged, recoverable state rather than a second action built on an unproven assumption.

The right sequence — demo for plumbing verification, simulator for assertion coverage, live for strategy

The testing workflow begins with demo for plumbing verification. That stage checks signing, host selection, accepted order bodies, and WebSocket reconnection, but not strategy edge or controlled fill handling. Next, the stateful simulator runs the assertion coverage and fault scenarios in CI on every commit, and the signing unit tests run before the integration tests. Ongoing checks for API changes keep those repeatable results in step with the exchange interface.

Live capital comes last, after the envelope has passed those checks. The first fill should confirm how execution works, with restart and acknowledgment defects already caught by the tests. Strategy evaluation also needs fees in the model from day one, as the 12-trade result demonstrated.