# The pre-warmup capture on SCAN: installed and verified

SCAN, 2026-10-16/17. Every load-bench client now runs its first ten cycles before
warmup with capture on, and its result file carries the graded capture
(docs/bench/BENCH_RESULTS.md, "Message capture happens before warmup"). This
sitting installed the clients that write the capture and measured one closed
single-session kernel nos_er cell per flavour.

## What was installed

Built from 208e972a2 and installed with `libhft-bench-install`. The replaced
versions were kept in `~/.cache/libhft/bench-rollback-20261016` on the host.

| Artifact | Built by | New md5 | Replaced |
|---|---|---|---|
| `bench-run` | the checkout | `e85785b5` | |
| `bench_roundtrip` (untuned, as before) | `make libhft-bench-roundtrip` | `41837d53` | `8813683d` (2026-10-11) |
| `dotnet-fix/native` (with `libhftnet_*.so`) | `private/ops/scripts/build-dotnet-bench.sh` | `ProviderSmoke.dll` `024f2c45` | client of 2026-09-12, library of 2026-10-02 |
| `dotnet-fix/libhft-net` | `private/ops/scripts/build-dotnet-bench.sh` | `LibhftNet.Bench.dll` `65bea677` | |
| `java-fix/libs` (with `libhftjava_*.so`) | `private/ops/scripts/build-java-bench.sh` | `fixbench.jar` `6e7a08c7` | jars of 2026-10-10 |

Not installed:
- `bench_roundtrip_zf` and `bench_roundtrip_vma_sx` stayed the backport builds
  that day; they were replaced on 2026-10-17 (below).
- `rust_pure` was being rebuilt with PGO by another session at the time; it
  captures once a build from 8a0ed62e1 or later is installed.

## The capture

Two sittings, kernel 5.14.0-687.44.1:
- `claude-preflight-verify-20261016`: all five flavours;
- `-r2`: the four managed ones again.

In the first sitting `bench.sh` did not pass the switch to the Java clients.
Its probe of `fixbench.jar` used `grep -q` under `pipefail`, so `unzip` died
of SIGPIPE and a capable jar read as an old one; it is fixed in this commit.

Every capture graded pass, 10 cycles, preflight sink `9f930503e0ebe96`
(`expected_sink(nos_er, 10)`):
- cpp, dotnet-native and dotnet-pure in both sittings;
- java-pure and java-jni in the second.

Each result file below carries its capture in `functional` (about 3.8 KB) and a
2000-point CDF.

## Closed kernel/single, us

The rows published: cpp's from the first sitting, the others from the second.

| Flavour | p50 before | p50 now | p99 before | p99 sitting 1 | p99 sitting 2 |
|---|---|---|---|---|---|
| cpp | 17.68 | 18.04 | 20.41 | 19.78 | |
| java-jni | 23.16 | 22.05 | 25.65 | 34.20 (WARN) | 27.94 |
| dotnet-native | 20.53 | 21.00 | 24.38 | 28.23 | 27.88 |
| dotnet-pure | 27.25 | 29.05 | 30.36 | 39.42 (WARN) | 30.88 |
| java-pure | 27.83 | 30.02 | 33.45 | 42.31 | 45.15 |

The rows before were measured on 2026-09-12 (2026-10-16 for cpp).

## The higher p99, A/B on 2026-10-17

`claude-preflight-ab-20261017`, closed kernel/single, OLD then NEW, twice:
- OLD: the replaced trees, with Java jars of 2026-10-10, the .NET client of
  2026-09-12 and its library of 2026-10-02;
- NEW: the installed trees.

| Cell | OLD p99 | NEW p99 |
|---|---|---|
| java-pure | 37.82, 38.42 (both PARTIAL) | 44.87, 43.51 |
| java-jni | 33.80, 29.18 | 28.64, 32.29 |
| dotnet-native | 28.36, 28.43 | 35.19, 24.35 |

- **dotnet-native's rise is the host:** the old build measures the same,
  about 28 us, and both are above its 2026-09-12 row.
- **java-jni:** no difference.
- **java-pure** is the only cell where NEW is higher in every run. Its seven
  NEW runs across the three sittings average about 43 us, against 38 for OLD.

`claude-preflight-volatile-ab-20261017` tested the one change of this work on
pure Java's measured path: the two raw-publication flags in `PureJavaSession`,
made `volatile`. Two `libhft-java` jars were built from 5ef3ff9bd, A as
committed and B with the fields plain, and run A then B, three times:

| Jar | p99 | p50 |
|---|---|---|
| A, volatile | 40.48, 36.46, 46.57 (mean 41.2) | mean 29.9 |
| B, plain | 36.60, 43.40, 45.30 (mean 41.8) | mean 29.4 |

The flags cost nothing measurable, and one build varies by about ±5 us between
runs. The capture's other cost was ruled out above: the first sitting ran
without it. What remains of java-pure's difference from OLD is about 4 us,
within twice that run-to-run spread. If it is real, it comes from the pure-Java
engine changes between 2026-10-10 and 208e972a2, not from the capture.
Settling it would take more runs per arm than these. After both A/Bs the
installed trees are again the ones in the table at the top.

## The cpp bypass builds, 2026-10-17

- **Built:** `bench_roundtrip_zf` (`0fd4353b`) and `bench_roundtrip_vma_sx`
  (`7650531d`), from ec4a8a364 (`make -C engine/cpp zf_session vma_session`).
- **Replaced:** the stamped backport builds `96d93000` and `440df2af`, kept
  with the others.
- **Sitting:** `claude-preflight-verify-20261017-cpp-bypass`, closed single.
  Both captures graded pass, 10 cycles.

| Cell | p50 before | p50 now | p99 before | p99 now |
|---|---|---|---|---|
| cpp/tcpdirect | 4.88 | 4.68 | 5.01 | 4.79 |
| cpp/socketxtreme | 5.76 | 5.68 | 6.85 | 6.74 |


## Through the open v4 driver, 2026-10-17

`claude-preflight-open-verify-20261017`, one sitting per cell, published by
`run-open-matrix.py --schedule-version 4` at 50 000/s. Both cells passed the
cell-pinning and open-schedule audits with the capture flag on the client
command line: a path for cpp, the bare switch for Java. Every capture graded
pass, 10 cycles.

| Cell | service p50 | service p99 | response p99 | status |
|---|---|---|---|---|
| cpp/kernel/single, before | 17.755 | 27.908 | 31.590 | ok |
| cpp/kernel/single, now | 16.998 | 21.864 | 23.198 | ok |
| java-pure/kernel/single, before | 36.796 | 53.267 | 55.031 | ok |
| java-pure, attempt 1 | 35.085 | 43.335 | 46.252 | partial, latency-variation |
| java-pure, attempt 2 (published) | 39.024 | 50.308 | 54.870 | partial, latency-variation |

java-pure is unstable from run to run on the new build in open loop as well
(service p50 CV 9.8% in attempt 2). The table keeps the driver's latest
publication, attempt 2.

## rust-pure, 2026-10-17

- **Built and installed:** `rust_pure` (`402d2de7`) from 4c8af281e, by
  `private/ops/scripts/build-rust-bench.sh` with target-host CPU, no tune profile and
  no PGO, the same build path as the `d6ede28d` it replaces (kept with the
  others).
- **Conditions:** on the user's word, the two cells ran on the isolated CPUs
  alone (`host-cpu-slot --pool isol`). Another session's HA run held the shared
  CPUs at the time, so unlike every other row here they are not like for like.
- **Captures:** both graded pass, 10 cycles.

| Cell | p50 before | p50 now | p99 before | p99 now | status |
|---|---|---|---|---|---|
| closed kernel/single | 18.33 | 18.20 | 31.15 | 25.97 | ok |
| open v4 kernel/single (service) | 18.284 | 18.252 | 28.144 | 30.148 | partial, latency-variation |

In the open cell p99.9 rose from 32.2 to 47.3 us and max from 184 to 275, the
tail the overlapping work would add.

The open cell was measured again with the whole host quiet
(`claude-preflight-open-verify-20261017-rust-pure-quiet`), and that row replaces
the overlapped one. Its capture graded pass.

| Run | service p99 | p99.9 | max | run-to-run CV of p99 | status |
|---|---|---|---|---|---|
| before (d6ede28d) | 28.144 | 32.160 | 184.1 | 4.9% | ok |
| overlapped (402d2de7) | 30.148 | 47.315 | 275.4 | 18.2% | partial |
| quiet host (402d2de7) | 27.103 | 35.989 | 142.3 | 22.5% | partial, latency-variation |

On the new builds two open cells are partial on run-to-run variation where the
old builds' rows were complete: rust-pure (p99 CV 4.9% before, 22.5% now) and
java-pure (above). cpp's stays complete. Each measured run has its own warmup,
and the ten preflight cycles run once, before all of them, so the capture is an
unlikely cause, and an A/B confirms it.

`claude-preflight-onoff-ab-20261017-r2` and `-r3` ran the installed `rust_pure`
(402d2de7), open v4 kernel/single, collection-only, five rounds with the capture
on and off. The off arms used `run-open-matrix --no-preflight-capture` and wrote
no capture. The first attempt (`claude-preflight-onoff-ab-20261017`) set
`BENCH_PREFLIGHT_CAPTURE=0`, which the driver strips, so all four of its runs
had the capture.

| Arm | service p99 | mean p99 | mean p50 | stable runs |
|---|---|---|---|---|
| on | 35.35, 27.35, 26.84, 28.63, 23.99 | 28.4 | 18.90 | 2 of 5 |
| off | 25.01, 23.41, 28.82, 28.98, 31.25 | 27.5 | 18.89 | 3 of 5 |

The capture costs nothing measurable. The partial verdicts come with and
without it, so the instability belongs to the new build on this host; the new
builds also carry other sessions' engine changes.

Every flavour and every C++ build on SCAN now writes the capture.

## The builds themselves, A/B 2026-10-17/18

Old and new builds, the capture off in both arms, interleaved, nothing published:

| Cell | Arm | p99 by run | mean p99 | mean p50 |
|---|---|---|---|---|
| rust-pure open v4 kernel/single (service) | OLD d6ede28d | 24.80, 25.65, 30.22 | 26.9 | 18.35 |
| | NEW 402d2de7 | 26.39, 26.37, 23.91 | 25.6 | 18.59 |
| java-pure closed kernel/single | OLD (2026-10-10 jars) | 40.61, 49.63, 39.06, 42.88 | 43.1 | 29.70 |
| | NEW (208e972a2) | 40.52, 36.90, 37.11, 40.35 | 38.7 | 28.56 |

Sittings `claude-rust-build-ab-20261017` and `claude-java-build-ab-20261018`.

**Neither build regressed.** The earlier java-pure gap (37.8 and 38.4 on the old
build) rested on two runs. rust-pure's old build is partial in two of its three
open runs, as the new one is, although its 2026-10-13/14 row was complete. The
higher tails and the partial verdicts of these sittings belong to the host as it
was on those days: the same builds vary as much from run to run as they differ
from each other.