# hp managed and Rust validation

This review found two runtime defects: .NET Native discarded TCPDirect peer EOF,
and Rust FAST byte-vector encoders omitted their length prefix in optimized builds.
Both have deterministic failing-before/passing-after regressions. The work also
strengthens benchmark freshness and allocation checks. It does not establish a
latency improvement or complete the all-flavour hardware matrix.

## Runtime fixes

The .NET native transport now forwards TCPDirect reads to `ReadMsgOrEof` instead
of the EOF-discarding `ReadMsg` wrapper. The native regression exercises the actual
exported session owner: idle EAGAIN, peer FIN, reset, teardown and reconnect policy.
It passed with TLS compiled both out and in. This uses a controlled TCPDirect API,
not a physical NIC; hardware validation remains separate.

Rust's mandatory and nullable FAST byte-vector encoders called `encode_u64` inside
`debug_assert!`. Optimized compilation removed that side effect: the payload
overwrote the prefix and the reported extent included unwritten trailing bytes.
Encoding is now unconditional, retaining the transactional capacity checks. New
tests cover empty values, nullable present values, multi-byte prefixes and nonzero
output offsets. The optimized production FAST module passed all 71 tests, including
the generated codecs. A new conformance stage compiles these actual modules with
`debug-assertions=no`; debug Cargo coverage remains intact.

Neither runtime fix adds a lock or allocation.

## Test and harness changes

Java's runtime-jar freshness check now considers `src/main` and the Maven pom,
not `src/test`. The behaviour regression proves that test-only edits do not mark
a current runtime jar stale, while main-source and pom edits still do.

The Java simulator allocation tests retain exact-zero assertions. They now warm
the same measurement loops for longer and explicitly skip when allocation
counting is unsupported or disabled. A deliberately allocating sink is a positive
control, ensuring the counter observes real escaped allocations. A failing trace
captured 1,688 bytes in 54 String/byte-array allocation events at the test's
measurement-loop entry. Startup/class-resolution work is an inference from that
trace and the warmup experiments, not a directly recorded compilation event.

The SBE ring's test-only producer and consumer yield when blocked. Both previously
landed on one isolated CPU, making the tiny ring depend on scheduler quanta. The
same 200,000-message ordering/torn-slot test passed on a single CPU after the change;
production ring operations are unchanged.

The .NET benchmark publish wrapper disables persistent build servers, keeping its
work inside the CPU lease. The TLS test now checks send acceptance directly rather
than waiting for a report after an unchecked failed send.

## Completed validation and limits

The refreshed .NET full suite ran 1,404 tests: 1,401 passed and three skipped.
The TLS allocation test passed in five fresh test runs and in that full rerun.
One earlier full refresh hit its five-second round-trip deadline while both peers
were logged on. Its cause remains unresolved; the passing rerun is not a fix claim.

The three skips were multicast loopback, the opt-in live bypass receiver, and the
native pool broadcast case requiring multiple acceptor threads. The refreshed
pool case then passed separately with `LIBHFT_ACCEPTOR_THREADS=2`, with no skip.
The ordinary full-suite skips must not be counted as live NIC coverage.

Full Rust release validation passed 708 tests, with no failures or ignored tests,
and the Rust benchmark binary rebuilt successfully. The optimized FAST gate
passed 71 tests; its seven self-tests, 149 conformance-manifest tests and 29
build-root tests also passed.

Java completed 204 current source test classes: 1,917 tests, no failures, errors
or skips. This is composite coverage, not a single successful full-suite command:
the outer 30-minute build/test lease expired after Rust and Java rebuilds and most
Java tests. The outstanding mode-3 and counters-disabled forks were then run
explicitly through their configured Maven executions. The report collector checks
current source class names and timestamps; it excludes nine stale reports for
removed or renamed classes, rather than counting old results as current coverage.
The allocation tests also passed in five fresh default-JIT JVMs, synchronous C2,
C1-only and OpenJDK 21.

The Java live allocation test initially placed its busy client and server on CPU
8. It progressed after a manual placement intervention inside this session's
lease: the verified client LWP was pinned to CPU 8 and its server LWP to CPU 9.
The retained thread dump records that intervention. This run does not prove that
the test automatically selects distinct CPUs; that harness improvement remains
separate from the production review.

## Physical Rust smoke

The refreshed Rust binary passed one single-session application exchange on each
transport, with 200 warmup and 200 measured messages per arm. Both ports were up,
the server/client pins were disjoint (9 and 14), and VMA's audited internal thread
used CPU 13. Physical TX deltas were:

| Transport | Physical NIC | TX delta |
|---|---|---:|
| kernel | Mellanox | 418 |
| Onload | Solarflare | 409 |
| TCPDirect | Solarflare | 408 |
| VMA | Mellanox | 811 |
| SocketXtreme | Mellanox | 810 |

Evidence is in `~/libhft-bench-runs/smoke-hp-rust-20261002-r3/`. These are functional
checks only: the lease was `isol`, with another session running shared-core work.
No latency values from these short samples are published or compared.

The preceding `r2` sitting passed Onload logically but failed the physical guard:
its immediate post-sample counter showed only three packets. During the final
verifier's three pings the counter advanced by 408. A repeat of the same binary
and recipe, waiting three seconds outside the timed sample before the final
counter read, yielded 409 without reducing the 100-packet threshold. This supports
delayed MAC-statistics visibility, not an Onload framing or routing failure. The
[upstream Solarflare MAC-statistics implementation](https://github.com/torvalds/linux/blob/master/drivers/net/ethernet/sfc/mcdi_port_common.c)
programs periodic updates at 1,000 ms; this is supporting context, not proof of the
exact update interval of the installed driver. The final read precedes the final
verifier's pings; a few residual startup-verifier packets are negligible against
the unchanged threshold. Preserve both sittings and allow statistics to settle
when testing similarly short exchanges.

Java and .NET physical smoke still requires staging their verified bundles with
the password-gated installer. A SHA256 manifest and recoverable staging script
are prepared; they have not been executed as root. Two obsolete Java classpath
jars will be moved into the installation backup, not deleted.

## Provenance and evidence

Work is in the locked `codex-libhft-hp` worktree of `atrekes/libhft-devel`, with
`LIBHFT_BUILD_ROOT=/home/yann/build/codex-libhft-hp/libhft`. Foreground work uses
`host-cpu-slot`; functional builds/tests and Rust NIC smoke use `isol`. Any future
latency measurements require `isol-no-shared`. Native managed bundles include
TCPDirect and VMA support.
These are profile-free functional checks, not a tuned latency sitting.

Rust used rustc 1.92.0; the native JNI rebuild used GCC 14 with `-O3`, LTO and
`-march=native`. Java checks use OpenJDK 17, with allocation-test variants also
covering OpenJDK 21. The .NET SDK is 10.0.112, targeting the project's .NET 8 runtime.

The refreshed runtime sources are pinned at
`a7c57a04b1dafdc4bb28bf888a8d5a2bcd775e53`, with later test/tool-only changes.
The pre-publication source snapshot is retained on
`origin/s/codex-libhft-hp-verify-20261002` at
`5cf500da594839347d0eff4f8cad497954df84eb`; this preserves the tested revisions
across the final rebase. Artifact SHA256s are:

```text
libhftjava.so  855281af68ea07d2699c35d7c5e67cf1663a2ed5f8a19bceefb8f5eda6b5d733
libhftnet.so   b08018891ea1bdc2df6fc84196c076caad1bd08c73ea340eaaaa8f06e08efba5
rust_pure     91d660b9c39762320224f6acace4ed4b37396d42ee7f25cc4c627809cffed61c
```

Retained evidence is under `~/archive/libhft/nic-20261002-managed-rust/`:
the before/after FAST logs, allocation traces, .NET full-run logs and Java XML
reports in `java-complete-a7c57a04b/`, plus staging and tested-source manifests.
The earlier complete Java run at `79e53ac78` had 1,912 tests and no
failures, errors or skips; it does not substitute for the current refresh.