# Rust audit output: account for completed records before a write failure

## Reproduction and source contract

The source baseline is `421acd7ec`; its audit implementation is unchanged from
`cf0622267`. The original writer formats up to 64 records into one buffer and
uses `File::write_all`. If a short write succeeds and the next syscall fails,
`write_all` reports the error without reporting the successful prefix. The
writer then counts the whole batch as failed, including complete records
already in that prefix. The public failure counter promises a count of records,
so this overstates the audit gap.

The baseline public-API tests produce 3 passes and 7 failures. For example, a
three-record batch with two complete records written before ENOSPC reports three
failures instead of one. The tests verify actual output bytes and that the
entire intended batch reached the overridden syscall before checking counts.
The successful-prefix and Interrupted behavior follow Rust's primary
[Write contract](https://doc.rust-lang.org/std/io/trait.Write.html#tymethod.write)
and [write_all contract](https://doc.rust-lang.org/std/io/trait.Write.html#method.write_all).

## Repair

The writer records each formatted record's end offset in a fixed 64-entry stack
array, including its terminating newline. Its write loop tracks bytes accepted
by successful writes and retries Interrupted without changing that progress.
A hard error or zero-byte write ends the batch; only record ends beyond the
accepted byte offset add to `write_failures`. A partially written record counts
once, as do the remaining records which were not written. Embedded payload
newlines do not create record boundaries.

Successful batch output, producer ownership, slot handoffs, file append mode,
batch size, sync cadence, counter carry and the public API retain their existing
behavior. The extra bookkeeping occurs only in the private writer, with no
producer allocation, lock or syscall.

## Fault coverage

Ten external integration tests exercise 75 deterministic syscall scenarios via
`FileLog`'s public API. The override is confined to the Linux x86-64 test binary
and one canonical fixture pathname. A parked seed write lets the producer queue
all records before the next drain, proving tests drive a three-record or maximum
64-record batch rather than depending on thread timing.

Cases include EIO, ENOSPC and zero-byte writes before progress, at complete
record boundaries, within a record, and just before the final newline. Every
complete-record boundary in a maximum batch is checked. EINTR before and after
short progress must preserve the byte offset; successful retries finish with
zero failures and no duplicated bytes. Literal epoch timestamps and independently
assembled payloads check exact preexisting bytes, accepted prefix, later batch
and reopened append cycle. Frames include empty, SOH, embedded newline and an
oversized frame; truncation counts once even when its write fails. Refused enable
and healthy reopen retain the cumulative counters. Fixture directories delete
on drop, and guard teardown releases parked writers before log teardown.

## Verification

The full-suite candidate is `d6a398e5b`. All 341 Rust file hashes match between
the publishing worktree and HP. Cargo/Rust 1.92.0 runs offline at one job, with
test threads capped at one and absolute session `LIBHFT_BUILD_ROOT`/Cargo target
paths. Test/debug profiles omit debug symbols; optimization and debug assertions
retain their normal test settings.

| Gate | Result |
|---|---|
| Baseline fault integration, Rust/Cargo 1.93.1 on BEELINK2 | 3 passed, 7 failed; all seven fail at the record-count assertion |
| Full default Cargo suite, pinned candidate on HP | 1,097 passed, 0 failed, 1 existing ignored builders doctest |
| Full open Cargo suite, same candidate on HP | 952 passed, 0 failed, the same 1 ignored doctest |
| New external audit fault regressions | 10 passed in each edition, included in the full suites |
| Integrated `rust-engine-tests` conformance stage | Passed; 1 ran, 0 skipped, 0 failed |
| Integrated `rust-open-edition-tests` conformance stage | Passed; 1 ran, 0 skipped, 0 failed |
| Strict Clippy 1.93.1, all targets, integrated source | Passed in both editions with `-D warnings` |
| Final audit/journal integration selection | 23 passed in each edition, 0 failed, 0 ignored |
| Final strict Clippy 1.93.1, all targets | Passed in both editions with `-D warnings` |
| Rustfmt on final new test; public-export scan on owned audit files | Passed; 0 scrub findings |

The integrated source is `e9f938d4c`, based on `40c9e96fd`. It adds the separately
published sparse mmap replay, learned-store forwarding and FIXP refusal-count
fixes. The audit runtime remains byte-identical; the test harness additionally
releases its fault mutex before reporting barrier errors, so its panic hook can
write to stderr safely. All 343 Rust file hashes match HP. The two selected
conformance stages run separately and retain their own reports. Each report is
PARTIAL/exit 2 because only one of the 358 configured stages was selected; these
are two successful Rust stages, never a claim that the whole monorepo suite ran.
The runner does not retain successful Cargo stdout, so the explicit full-suite
counts above belong to the earlier pinned candidate, not these stage reports.

The final integration pin is `9fd0a76da`. A later rebase adds only two Rust paths:
the independently published classic journal reset/append-cursor repair and its
four-case fault integration file. Both owned audit files remain unchanged. All
344 Rust file hashes match HP. The final public-API selection includes the three
ownership, ten write-fault, six journal recovery and four journal reset cases.
Clippy also compiles every target on that integrated source.

Own heavy jobs run serially on each host, with a guard checking active compiler
and suite processes plus at least 8 GiB available memory. Verification moved to
HP after BEELINK2's shared build load. Foreign jobs interrupt only our owned
process group; interrupted/superseded attempts are retained and are not green
gates. Full-suite totals count only the last completed attempt in each log.
Conformance was split by stage to retain each completed report during later
interruptions. New temporary fixtures delete on normal teardown.

The candidate bundle, source manifests, exact patches, baseline audit/test files,
commands, reports and logs are kept in
`~/archive/libhft/rust-audit-write-progress-20261004`. `publication.json` records
final source/publication pins; `SHA256SUMS` verifies the kept evidence.


## Limits and remaining work

This repairs failed-record accounting. The existing append policy can leave a
torn prefix after a hard write error, and later batches resume appending; these
tests check those bytes explicitly rather than calling that prefix a valid
record. The counter is not a crash durability guarantee. Periodic/final sync
errors are still outside its record-count contract. A coordinated torn-output
recovery/fencing policy and sync-error observability remain separate work.
No disk format or other engine behavior changes here. Inventory parity scores
stay unchanged; actual production endpoint and venue/FIXP certification remain
pending.