# .NET FIX recovery rejects oversized journal payloads before reading

The .NET managed session loaded an imported QuickFIX record before checking
whether its recovery buffer could encode it. Both compatible-layout stores
grow their read arrays to the indexed payload size, so an otherwise rejected
record could allocate and read far more than the session's configured buffer.
The correction checks existing index metadata before loading the payload.
Public store reads, journal formats and ordinary send/receive paths are unchanged.

## Reproduction

The unchanged production revision is `af9fea4fe`; the new wire fixture is
retained in the baseline patch and source manifest. SDK 8.0.131 and a freshly
built default native library run four real TCP managed-session cases: the
QuickFIX C++ and QuickFIX/J layouts, each recovered through either
ResendRequest or NextExpectedMsgSeqNum(789) on Logon.

Each journal has two bounds-valid index entries pointing at one 4 MiB sparse
payload, followed by a small valid application message. The oversized payload
contains a short FIX prefix and zero padding, not a valid complete FIX frame.
Its contents do not affect the expected result: its declared length already
exceeds the 1024-byte recovery work buffer, which reserves 64 bytes for replay
fields. The outbound checkpoint is 6 and inbound checkpoint is 1.

All four baseline cases complete the wire assertions, then fail the owner
allocation budget of less than 1 MiB. Allocation counts include only calls to
the session owner's `RunOnce`, not client parsing or journal preparation.

| Layout | Recovery trigger | Baseline owner allocation in bytes | Corrected owner allocation in bytes |
|---|---|---:|---:|
| QuickFIX C++ | ResendRequest | 4236920 | 42504 |
| QuickFIX C++ | Logon tag 789 | 4236088 | 41760 |
| QuickFIX/J | ResendRequest | 4218008 | 23680 |
| QuickFIX/J | Logon tag 789 | 4217880 | 23552 |

The corrected measurements come from the nine-case run at `2da9c66fa` after
integrating the new acceptor-port setting and shared configuration-range
fixtures. All nine regressions pass with zero skips; the source manifest stays
unchanged through the run. The four wire cases no longer allocate the rejected
payload, and the five metadata cases perform their measured queries without
current-thread allocation.

The wire checks require one gap fill across sequences 2 and 3, the retained
application at 4 with its original Text and SendingTime, and a trailing gap
fill. The session remains logged on, and a subsequent TestRequest gets a fresh
Heartbeat at sequence 7. Tag-789 recovery includes the newly sent Logon in its
trailing gap; explicit recovery ends before that Logon.

## Correction

The existing internal replay index exposes an optional payload length. Its
default of minus one preserves lookup and validation for stores without size
metadata. QuickFIX C++ and QuickFIX/J return the size already held in their
offset dictionaries, without reading the body or adding append bookkeeping.
The session reuses its cached index capability and rejects known oversized
payloads before `TryGet`. The actual returned span still passes the existing
framing, checksum, application-type and headroom checks.

Headroom validation now subtracts the reservation from the work-buffer length,
avoiding addition overflow on a process-sized returned span. No public API,
native ABI, session field or lock is added. Recovery does add a metadata call
and, for the two QuickFIX layouts, a dictionary lookup before a valid payload
read. This is a reduction in unnecessary recovery allocation and I/O, not an
ordinary-message or NIC latency benchmark.

## Regression coverage

The four wire cases deliberately use a safe 4 MiB payload. Public store readers
still return that payload after recovery; those reads occur outside the owner
allocation measurement. Five metadata cases cover empty records, duplicate
replacement, reset, unavailable lengths and a recovered `int.MaxValue` length
queried without loading its sparse payload. Existing retained-index
implementations can keep the unknown-length default.

The C++, Java and Rust compatible-layout readers have different bounds:
C++ rejects records larger than its fixed message capacity while recovering
the index; Java validates against its message capacity and checks the
destination size before reading; Rust checks the recovered size before making
its payload vector. This finding concerns the .NET managed recovery path.

## Process sized recovery

A standalone ProjectReference probe uses a read-only source archive at
`af9fea4fe` for the baseline and the session worktree at `dda01b1da` for the
correction. The same wire fixture drives both builds. Four controls use
4096-byte payloads; four large cases use `int.MaxValue`, covering both journal
layouts and both recovery triggers. Each child verifies its 256 MiB GC heap
bound before entering recovery and has an external ten-second watchdog.

The unchanged runtime passes all four controls and loses the peer in all four
large cases before completing recovery. The corrected runtime passes all
eight cases without peer loss or timeout. The four corrected large cases
allocate 24184 to 43368 bytes on the owner, not a process-sized array. The
metadata-only `int.MaxValue` unit cases and this real-session drive are
separate checks: the latter proves gap-fill completion, retained application
delivery, logged-on state and the next fresh Heartbeat sequence.

The body files are sparse scratch under the build-root temporary directory;
the fixture never materializes the huge public-read payload. The heap bound
applies only to these probe children, not the full suites or a deployed engine.

## Development and open qualification

The complete run at `dda01b1da` uses SDK 8.0.131, runtime 8.0.31 and
VSTest 17.8.0. Default and open native libraries are freshly built with GCC
toolset 14 and export 209 hftnet entry points each. Open compilation verifies
144 sources without 67 private sources and six refused feature boundaries.

| Check | Passed | Failed | Skipped |
|---|---:|---:|---:|
| Complete development suite | 2568 | 0 | 4 |
| Complete open suite | 2337 | 0 | 3 |
| Development native pool with two workers | 3 | 0 | 0 |
| Open native pool with two workers | 3 | 0 | 0 |
| New wire and metadata regressions | 9 | 0 | 0 |
| New regressions plus sparse/index and imported-admin recovery | 151 | 0 | 0 |

The separate pool checks close the suites' pool-only broadcast skip. Remaining
distinct skips are multicast loopback, the refusal test for a non-Linux
platform, and the development-only opt-in live bypass receiver. No native
availability test skips. Hardware latency and non-Linux refusal are outside
this Linux functional qualification.

All nine selected conformance guards run and pass with zero skips, including
the native ABI/export table, cross-binding contracts, public-package checks
and all open .NET tool projects. The runner reports a selected 9-of-362 slice,
not a complete conformance-suite result. Both public scans have zero findings,
including all six changed engine files. Source, SDK-helper, driver and
baseline-archive manifests remain unchanged throughout the complete run.
The run holds an isolated-core lease excluding shared work.

## Publication integration

After rebasing onto the timer-ceiling validation and standalone Rust export
fixes, the bounded supplement at `f4e293b74` passes 298 development and 288
open tests with zero failures or skips. It covers every SessionSettings
fixture, native option mapping, required native loading, the new regressions,
sparse/index recovery and imported administrative records. All nine selected
publication guards pass again, and the six changed engine files have zero
public-scan findings.

The source comparison permits only the changed SessionSettings parser and
guide, its new shared-range fixture, and additional public-export self-tests.
Replay code, native runtime, C++ sources and the other .NET runtime files stay
byte-identical to the complete qualification. The supplement's source and
driver manifests remain unchanged. `publication-integrated/` retains the
comparison and results. The complete-suite verdict remains attributed to
`dda01b1da`; the focused supplement is not relabelled as another full run.

## Final receive and configuration integration

The complete rerun at `25224b067` integrates the subsequent configuration
ranges, half-window schedule refusal, native option mapping and ordered
Session Reject delivery. Its authority is `qualification-latest/`.

| Check | Passed | Failed | Skipped |
|---|---:|---:|---:|
| Complete development suite | 2582 | 0 | 4 |
| Complete open suite | 2350 | 0 | 3 |
| Development native pool with two workers | 3 | 0 | 0 |
| Open native pool with two workers | 3 | 0 | 0 |
| New wire and metadata regressions | 9 | 0 | 0 |
| Focused recovery selection | 151 | 0 | 0 |

All nine selected guards pass again, and both public scans have zero findings.
The standalone baseline again passes its four controls and loses the peer in
all four large cases; the corrected build passes all eight without timeout.
The skip identities and feature-boundary counts are unchanged. Native sources
are unchanged, so the verified default/open libraries are reused with their
freshness guards enabled. Source and driver manifests remain unchanged through
the full rerun. Earlier counts keep their earlier revisions above.

## CheckLatency and native configuration integration

The complete run at `1dd631649` integrates the later CheckLatency/MaxLatency
validation, Logout-reply clock check and C++ configuration bounds through
`ea555357b`. Its authority is `checklatency-integration/`. Default and open
native libraries are rebuilt after the shared C++ header change; both retain
209 exports.

| Check | Passed | Failed | Skipped |
|---|---:|---:|---:|
| Complete development suite | 2595 | 0 | 4 |
| Complete open suite | 2363 | 0 | 3 |
| Development native pool with two workers | 3 | 0 | 0 |
| Open native pool with two workers | 3 | 0 | 0 |
| New wire and metadata regressions | 9 | 0 | 0 |
| Focused recovery selection | 151 | 0 | 0 |

All nine selected guards pass, both public scans have zero findings, and the
same four development/three open skips remain. The separate native-pool runs
again close the pool-only suite skip. The baseline probe passes its four
controls and loses the peer in all four process-sized cases; the corrected
probe passes all eight without peer loss or timeout under the verified
256 MiB child heap limit. Source, driver, SDK-helper and baseline-archive
manifests remain unchanged throughout the run. This is functional recovery
qualification, not a hardware-latency measurement.

Before that full rerun, `publication-final/` at `3c11e6f6e` passed 307
development and 298 open selected tests with no failures or skips, plus all
nine selected guards. Its equivalence report has an empty `changed_inputs`
list against the `25224b067` qualification: the selected C++, .NET, tool and
publication inputs were byte-identical. The earlier complete-suite counts
remain attributed to their own revisions.

## Evidence retention

Evidence lives under
`/home/yann/archive/libhft/dotnet-oversized-replay-20261012/`.
`baseline/` retains the four expected allocation-budget failures, native build,
toolchain, source pin, patch and unchanged input manifest. `fixed-focused/`
stopped before tests at the native-library freshness guard after the
acceptor-port rebase. `qualification/` freshly builds that library and passes
all nine regressions; its driver then stops because normal-verbosity VSTest
prints a different summary from the one its count check expects. The driver
uses the count-bearing minimal-verbosity format for the complete retry.
`qualification-complete/` is the authority for the complete suites, guards and
`process-sized-probe/` results above.
The retained
`evidence/dotnet-oversized-replay-20261012/run-checks.sh` composes the existing
default/open native builds, complete .NET suites, separate native-pool checks
and nine selected conformance guards with the new regression selection and
public scan.