# Rust sparse replay traversal, 2026-10-04

MemoryStore, FileStore and QuickFixFileStore inherited a replay loop that looked up
every integer in a requested resend range, including absent sequences. Four HA
wrappers also inherited that loop. This work bounds missing-run traversal by the
retained ring slots or ordered QuickFIX index entries and forwards that behavior
through the wrappers. The session's callback ordering and cancellation contract
stays the same; the normal save path has no new allocation or bookkeeping.

Implementation: `a5f367e3d` after rebase; measured as `567b5bd95`, based on `347d2743b`.
Full-suite engine tree: `934c02c60dd0400d8ebdafe70db55afdd2c07bca`.
The final Rust tree is unchanged: `9380fa2e9ffb25ab5e300cfb04fa8e79f07c56a6`.
Upstream changed the shared GapFill fixture during this batch; the fixture source
checker and its runtime Rust test were checked again after rebase in both editions.
Retained commands, probes, logs and independent reviews:
`~/archive/libhft/reviews/2026-10-04-codex-review-replay-1004/` on BEELINK2 and hp.

## Correctness

Sixteen new regression methods compare complete callback streams with the
reference walk, including every callback cancellation point, original payloads
and timestamps, nonmonotonic saves, collisions, empty administrative records,
reopen, clear, zero/inverted intervals and saturated unsigned endpoints. Logical
lookup counters check bounded work before the tests enter huge ranges. Coverage
includes ConfiguredStore, generic LearnedStore, both QuickFIX profiles, and all
four HA wrappers through standby, promotion, binding and release.

The public session probe requests replay across a million-number sparse range
from each configured memory/file/QuickFIX backend. Before and after the change it
produces exactly the same 646 bytes: GapFill, application, GapFill, application,
GapFill. The probe checks checksums, PossDup, original sending time, payload,
sequence endpoints and the unchanged next outbound number, 1000005. All four
backend outputs are equal. The before/after result files have SHA-256
`d6a99eaaaf6cf3bc172d36d988a9cad9711f262799f6cd1ca97554518516a5bf`.

Validation: the Rust library with venue codecs passed 1,315 pro tests on BEELINK2
(Rust 1.93.1) and 1,165 open tests on hp (Rust 1.92.0): 2,480 total, zero failed
or ignored. Strict Clippy (`--lib --tests --features venue -- -D warnings`) passed
both editions. Heavy jobs were serialized per host with parallelism capped at eight.
The four changed engine files pass the publication scan with zero findings.

## Measurements and limits

On hp, Rust 1.92.0 release builds, one host lease at a time, `-j8`, warm stores,
one warmup followed by seven timed calls. The sparse fixture retains four
six-byte records at the end of a requested range; memory/file have 16 slots.
Each timed call asserts four messages, one gap and an unchanged content checksum.
For range 1 through 10,000,000, medians were:

| Backend | Before | After |
|---|---:|---:|
| Memory | 54,827,681 ns | 127 ns |
| Async file | 55,439,408 ns | 143 ns |
| QuickFIX C++ layout | 43,125,971 ns | 56 ns |
| QuickFIX/J layout | 43,548,402 ns | 57 ns |

These are store callback timings, including clock overhead, not complete FIX
recovery latency. Small sub-microsecond differences between backends are not a
ranking. The important result is removal of work proportional to the numeric
span. All three tested spans (1,000; 100,000; 10,000,000) report identical event
counts before and after. Memory's worst case remains proportional to capacity
times the number of gaps; this is not a universally linear sparse-ring walk.

The initial separate-process dense sample moved from 31.1 to 32.8 microseconds.
A follow-up paired both paths in one binary using the unchanged public
`replay_range` as the reference, alternating order over 21 measured pairs of
1,000 calls each. A 4,096-message dense replay measured 29,519 ns reference versus
28,246 ns candidate; four messages measured 33 versus 32 ns, and one message plus
a tail gap measured 18 versus 18 ns. No code change was justified by the initial
dense sample. These warm microbenchmarks do not establish an end-to-end gain on
ordinary contiguous traffic.

A second paired probe uses the session's first-event callback pattern: false from
the first message or gap. It alternates 22 measured pairs of 100,000 calls after
two warmup pairs and checks the exact first event and cancellation on every call.
All five cases pass. Dense-first medians were 21.765 ns reference versus 17.768 ns
candidate; a four-number gap measured 35.264 versus 25.309 ns. Ring-boundary,
adjacent-gap and resume cases also showed no regression. These timings include
result checks, a six-byte payload copy and indirect-call overhead; they exclude
FIX decoding, transport sends and complete session recovery.

## Follow-up review

A separate source-only audit found analogous per-sequence traversal in C++, Java
and .NET classic resend paths. C++'s session also bypasses the public store replay
API; Java has separate single-session and acceptor-client loops; .NET's one-output
per-tick replay still coalesces missing sequences in an unbounded inner loop.
Those findings are not runtime reproductions or fixes in this batch. Exact source
references, callback lifetime constraints and owner coordination are retained in
`evidence/cross-flavour/CLASSIC_REPLAY_AUDIT.txt` in the archive.