# Buffer and codec review — 2026-10-07

Base: `9a9d4d7f8`. Session: `codex-review-buffers-1007`. Parallel reviewers
examined C++ rings/queues/numeric helpers, Java codecs/builders/queues and Rust
wire/builders/views/transport buffering. The root examined SOFH framing and
ran all builds/tests under the shared host lease, at most 16 jobs. Build outputs
stayed under the session build root; Windows source snapshots and outputs were
under `C:\Temp\build\codex-review-buffers-1007`. No mirror was edited.

## Corrections

C++ and Java SOFH encoders previously wrote the framing header before copying
the payload. When input and output shared storage, that could overwrite source
bytes and then copy a corrupted payload. C++ also used memcpy on overlapping
ranges. Encoding now moves the payload before writing the header, with memmove
in C++ and arraycopy in Java. The exact reserved-header placement skips copying
entirely. Bounds/format refusals still change no bytes. Both formats, empty and
small/large payloads, overlap in both directions, disjoint ranges, exact placement
and JNI views of shared direct storage are covered. Rust's safe slice API cannot
express aliased input and mutable output, so it needs no equivalent change.

Java's native SOFH entry wrapper now refuses read-only output before JNI can
write through its direct address. Read-only input remains accepted. MessageEncoder
also refuses read-only direct output before consuming the pending message, as
its heap path already did. Previously JNI wrote through the read-only view, while
pure Java threw ReadOnlyBufferException after consuming the message. Both
providers now throw IllegalArgumentException with unchanged bytes, position,
limit and pending message, allowing a writable retry.

JNI MessageEncoder construction now converts header strings to standard UTF-8,
matching the public Java contract. GetStringUTFChars had produced modified UTF-8:
supplementary characters became two encoded surrogates, changed wire bytes and
could exceed a native CompID limit after passing Java's 35-byte check. The new
bounded construction-only helper uses stack storage and preserves Java's '?'
replacement for unpaired surrogates. JNI signatures are unchanged. Regressions
cover ASCII/BMP controls, supplementary characters, malformed surrogates and
12/35-byte header limits.

Both Java MessageBuilder.putData overloads now preflight two field slots before
encoding or appending either field. With one slot left, the old method appended
the LENGTH field and then threw while appending DATA, leaving a malformed body
available for serialization. Tests cover zero/one/two available slots, repeated
refusal, reuse of the remaining slot and reset recovery.

C++ FixUtils::atoi64 promoted the billion multiplier only to long, which is
32 bits on Windows LLP64. The intermediate uint32 product wrapped in the
10–18 digit cases before any wider addition: 4294967296 parsed as zero and
19999999999 as 11410065407. Nine LL suffixes promote those products before
multiplication, preserving the unrolled implementation and shorter-width paths.
The 27-case regression table covers every affected width, 32-bit crossings,
leading zeroes, signed maximum and the existing 19-digit saturation behavior.

## Reproduction

- Linux SOFH regression fails against unmodified production code at the full
  buffer comparison. Java reproduces both formats in pure and JNI paths.
- The focused Java baseline runs 49 tests: 10 failures, two errors, no skips.
  Five expose SOFH overlap/read-only output; seven expose classic codec
  read-only/UTF-8 behavior and partial DATA admission. The other 37 pass.
- Windows MSVC numeric baseline has 14 failing cases; the Linux numeric baseline
  passes. The corrected Windows source snapshot includes the tuning kernel
  headers pulled in by FixUtils. The first snapshot failed compilation because
  those headers were missing; that setup failure is retained separately and is
  not counted as defect evidence.

## Verification

The focused rebuilt-JNI gate passes all 49 cases with zero failures/errors/skips.
Both Windows MSVC binaries (test_utils and test_fixp_sofh) pass. Their source
snapshots, manifests and logs are retained. Linux C++11 utilities and SOFH pass,
as does SOFH under AddressSanitizer plus UndefinedBehaviorSanitizer.

The full C++ open-edition gate passes 147/147 binaries with zero reported
skips, plus layering and simulator checks. Its edition selection excludes
83 private binaries; this is not a claim that those binaries ran.
Java development passes 3546 unique cases and Java open passes 3104, each with
zero final failures, errors or skips. Native loading was required in both,
with separate freshly built HFTJAVA_PRO=1 and HFTJAVA_PRO=0 libraries.
The ordinary forks intentionally skip three counters-disabled cases, which
all pass in their configured forks. Final JUnit aggregation keys by class and
case name; the 49 focused reruns are not added to the unique suite counts.
The generated verification-summary.json retains these counts.

All eleven final engine files pass the export scan with zero findings, and
whitespace checks pass. All eleven exact source hashes still match after the
final fetch/rebase; no incoming engine source changed the tested tree.
Independent review found no blocking issues in overlap ordering, bounds/refusal
atomicity, JNI views, UTF-8 conversion, builder admission or read-only retry.

No new lock or allocation is added to frame encoding. Exact payload placement
now avoids copying, and the JNI Unicode conversion is constructor-only. No
measured latency improvement is claimed. The bounded Rust review found no
confirmed defect; it made no source changes and ran no Rust suite. No .NET or
reserved C++ FIXP runner/flow/liveness/finalization path was edited.

## Publication and evidence

SOFH: `f211aa879`; Java codecs/builders: `79f59b4e5`; C++ decimal:
`ecac08fac`. Publication is only to private `atrekes/libhft-devel` after
fetch/rebase and verification of the exact tested source.

Evidence is retained at
`~/archive/libhft/reviews/2026-10-07-codex-review-buffers-1007/` locally and on HP,
including baseline/final logs and JUnit reports, Windows source snapshots,
final source hashes, scripts, published patch, record and a review-history bundle.
Every retained file is checked against MANIFEST.sha256 before cleanup. Session
worktree/build roots and Windows scratch are removed after verified archival.