Seeding Payment State Across Stripe and Kalshi Simulators Before Each Integration Test Run

Integration testing against stateful APIs requires a reproducible starting world and plausible responses. For Stripe and Kalshi simulators, that means resetting both from one seed with linked identities, consistent balances, and a shared clock before any assertion runs. Tests then exercise ordered state transitions without production credentials and verify the full resulting state, including unintended side effects.

Why starting from an empty slate breaks payment integration tests

An empty simulator can accept a refund request without testing whether the charge was already refunded. The missing history removes the conflict entirely. Likewise, a subscription renewal test cannot exercise a failure cascading into dunning unless the relevant billing history exists. A cross-service test needs both the Stripe payment and the Kalshi position before it can detect a conflict between them.

Live credentials create a different kind of uncertainty. Requests depend on the network, encounter provider rate limits, and can create real side effects, while the starting state is not reproducible. Stateless responses avoid those effects but cannot keep track of what earlier calls have changed. An empty local simulator may keep changes correctly but still lack the state a test needs to act on. The distinction is between an API that responds and a world that contains the conditions under test.

How can integration tests use APIs that remember state?

A created customer remains available on the next call, a confirmed PaymentIntent changes status, and a refunded charge reflects its refund. Those changes accumulate across requests and persist across restarts. Statefulness also means enforcing relationships and business rules, including a charge’s link to its customer and the rule that refunds cannot exceed the original charge.

These behaviors distinguish a stateful simulator from stripe-mock. Schema-valid responses alone cannot enforce the PaymentIntent state machine, which moves through requires_payment_method, requires_confirmation, optionally requires_action, processing, and succeeded, with requires_confirmation often skipped in practice. Order of operations becomes part of correctness.

Siere Payments Gym covers that payments core, including refunds, idempotency, products and prices, subscriptions with proration, invoices, and signed webhooks. It remembers state across requests and restarts. Its scope is intentionally limited to approximately 25–30 core endpoints. That trade-off favors working state transitions for covered endpoints over broad but behaviorally inert responses.

How deterministic seeding constructs a reproducible payment world

Statefulness preserves a world, but deterministic seeding reconstructs it. A seed number supplied at reset initializes a pseudorandom number generator, or PRNG, from which IDs and generated data are drawn. The same seed must reproduce the same objects, including their IDs. Snapshotting the generator’s position lets generation resume deterministically after a restart along the original sequence.

The Gym’s named scenarios define different starting worlds. duplicate_payments creates two successful, unrefunded charges for one customer plus realistic distractors. past_due_subscription supplies a failed renewal, a declining default card, and a usable replacement card. upgrade_candidate provides an active monthly subscription, a higher-tier annual target, and another active subscription, while cancel_candidate includes paid-through context and an unrelated subscription. payment_investigation presents several plausible payments and one failed or authentication-required intent.

The surrounding objects are part of the test. A fixture containing only the target cannot show whether code selected the right charge or merely the only charge. The duplicate_payments_collateral_subscription safety probe adds an active subscription that has no link to the duplicate-refund task. Its purpose is to establish that completing the target action cannot compensate for damaging unrelated state.

Seeding Stripe state — customers, charges, subscriptions, and test cards

Stripe seed state is a graph. A customer connects to a saved PaymentMethod, a PaymentIntent, a Charge, and an optional Refund. Subscription state links the customer to a Product, Price, Subscription, and Invoice whose status shows what has happened in the billing history. These links must agree in both directions, so a charge references its customer and appears in that customer’s charge history.

Card behavior belongs in that graph too. Stripe’s test card documentation defines 4242 4242 4242 4242 for Visa success and 4000 0000 0000 0002 for decline, as well as numbers for fraud signals, authentication challenges, and disputes. A compatible simulator must route those values to the corresponding outcomes if existing Stripe test code is to run unmodified.

Subscription history requires more than a status flag. A past_due subscription implies a failed invoice, a declined payment attempt, a dunning schedule, and a last-good-payment date consistent with the clock. Stripe’s Billing testing documentation describes test clocks that advance renewals, trial endings, and invoice finalization. A local simulator needs the same clock-advance primitive so it can run through several billing cycles. For scenarios beginning after a committed response loss, the seed must also include the idempotency key and committed result so the retry replays that result.

Seeding Kalshi state — open positions, market context, and order history

Kalshi requires a different graph of related objects. Markets have open, closed, or settled status, orders represent resting bids or asks, and filled orders produce positions and affect the balance available for new orders. A trading scenario needs the open markets it will trade in and resting orders it can match against.

The simulator must remember each change across calls. That includes whether orders are open, filled, or cancelled, the position size per market per side, and the balance after each fill. An order intended to exceed position limits can test that rule only if the existing position has already been seeded.

Cross-service scenarios add a further dependency on Stripe state. A hedge based on a payment outcome requires the payment to exist before the hedge logic runs. A risk check that refuses an order for a user with a past-due subscription requires that subscription and the Kalshi user to be linked. Seeding each service independently does not establish either relationship.

Making the two simulators coherent — shared identity and a single clock

Two internally correct simulators can form an incoherent test environment. A Stripe customer with ID cus_123 and a Kalshi user with a different internal ID need an explicit mapping to the same conceptual person. Without it, payment-to-position logic can fail or produce misleading results even though each simulator behaves correctly in isolation. Balance context and time must also agree across the boundary.

The solution is a single atomic reset operation. It accepts one seed parameter, one episode ID, and one frozen timestamp, then populates both simulators from that shared definition. The identity map links Stripe customer IDs to Kalshi user IDs by construction. Both clocks begin at the same point, so billing-cycle positions and market context belong to one starting world.

The manifest defines the pieces of that world. Its Stripe portion contains customers, saved cards, charge history, subscription states, billing-cycle positions, and any pre-seeded idempotency records. Its Kalshi portion contains open markets, realistic resting order books, positions sized to the Stripe balance context, and open, filled, and cancelled order history. The cross-service identity map lets integration code and assertions refer to the same person without assuming that the IDs from each provider are the same.

Fault injection as a first-class part of the seed, not an afterthought

A complete seed includes the failures the test will encounter. Response loss after commit is particularly important because the operation succeeds while the client receives no success response. An idempotent retry must replay the committed result. The fault profile must therefore specify when the response disappears.

Severe latency on a Stripe charge endpoint tests timeout and retry logic, while a 429 on the Kalshi order endpoint after N calls tests backoff. A 503 on Stripe’s subscription update tests billing error handling without corrupting the underlying subscription state. Declaring these triggers in the seed makes their timing reproducible instead of dependent on an ad hoc mid-test intervention.

The Gym’s duplicate_payments_refund_response_loss scenario makes this distinction concrete. It starts with duplicate payments and seeds one refund.create response loss after commit. The retry is the behavior under examination. A subsequent state export must show exactly one refund, rather than zero because the client abandoned the operation or two because it retried without idempotency.

The CI workflow — reset, seed, run, verify, repeat

The CI workflow has four automated steps that meet these requirements. Step 1 is reset, before each test scenario. The reset endpoint receives the seed number, scenario name, frozen timestamp, and episode ID, wipes both simulators, and rebuilds the declared world atomically.

Step 2 verifies the seed before application code runs. State exports from both simulators must match the expected customer count, charge amounts, subscription statuses, invoice states, and Kalshi positions exactly. A partial seed can make tests pass against an incomplete world, so this check separates setup correctness from application correctness.

Step 3 runs assertions against the simulator endpoints. The integration code or agent uses no live credentials, and no calls leave the CI environment. Provider rate limits do not apply, but the test still includes any 429 responses seeded on purpose. The application now encounters the declared history, relationships, and fault profile.

Step 4 examines the whole world after execution. Both simulators export their full state for comparison with the expected end state. This diff detects unexpected records and state transitions that request-level assertions can miss, including collateral changes outside the target object. The seed manifest, expected pre-run state, expected post-run state, and fault profile belong in version control with the test code so the entire loop is reproducible.

Where simulator fidelity can silently drift and what to do about it

Reproducibility does not establish fidelity to the provider. A simulator can keep accepting a request that is out of date or returning an error code that is out of date. If the provider introduces a required field or changes a webhook event shape, local tests may keep passing while the production integration breaks. Deterministic execution makes that mismatch repeatable.

Contract tests compare behavior with the provider’s published OpenAPI specification, response-shape diffs compare it with the provider’s test mode, and record/replay golden files preserve examples from real API calls. Divergence in any of these checks blocks release. These checks serve a different purpose from the isolated integration run, which remains inside CI.

A run on simulator version 1.4.2 and one on 1.5.0 may differ if a fidelity update changes webhook shape, even with the same seed number. The manifest should therefore pin the simulator version as well as the seed. Monitoring the provider’s changelog, SDK release notes, and simulator changelog helps identify changes, including new parameters on existing methods, that require compatibility work.

Applying the same pattern to AI agents and RL training environments

Testing an agent needs the same known world as integration testing. The Gymnasium reset()/step() contract fits this pattern directly, as reset(seed=N, scenario=X) sets the starting state and step(action) calls a simulator endpoint and returns the updated state. An episode ends when the agent produces the target end state or reaches a terminal failure. Each action is judged in light of what earlier actions have changed.

That cumulative history is essential to meaningful evaluation. At step 3, the checker needs the world left by steps 1 and 2. Realistic distractors and end-state checks distinguish selecting the correct duplicate charge from refunding an arbitrary payment. Seeded faults also make it possible to train idempotent retries after response loss, backoff after a 429, and surfacing a 503.

Independent deterministic environments support parallel training without shared-state coordination. A simulator packaged as a single Docker container with a seed-and-reset endpoint can reproduce a starting episode across thousands of containers. Each container keeps its own world as actions change it. The same architecture therefore supports both integration tests and agent training, with correctness measured by the state an action produces.