# TX queue counter experiment

A C++ and Java prototype reduced the cost of TX queue bookkeeping on an isolated
core. No production queue source or published latency result has changed. This
experiment measures serial enqueue and drain calls into a synthetic sink, not
producer/consumer threads running concurrently or a network round trip.

## Candidate and measurements

Each queue already documents exactly one producer and one consumer. Its queued
and refused counters have only the producer as writer, and its written counter
only the consumer. The candidate replaces atomic read-modify-write increments
with atomic loads and stores: relaxed in C++, `get` plus `lazySet` in Java.
Queue head/tail publication, fault handling, payload copies and observer reads
are unchanged. No locks or non-atomic shared counters are introduced.

The queued job obtained an `isol-no-shared` lease on hp at 2026-10-03 00:19:08
UTC, and measurements were pinned to CPU 9. This excludes other cooperating
foreground jobs; the archive does not contain a complete audit of every
background process. GCC 14.2.1 compiled with `-O3 -march=native`; Java used
OpenJDK 17.0.20.1 with a fixed 128 MiB heap and `ActiveProcessorCount=1`.

Five paired comparisons alternated baseline/candidate order. Each process used
three warm-up batches and eleven timed batches of two million 64-byte frames.
Every run verified 28 million queued and written frames, equal sink checksums,
an empty queue, no refusal and no send fault. The table is the median of the
five process medians, in nanoseconds per enqueue plus drain.

| Language | Baseline | Candidate | Reduction |
| --- | ---: | ---: | ---: |
| C++ | 21.169 | 9.375 | 55.7% |
| Java | 40.249 | 27.834 | 30.8% |

The existing C++ queue regression binary and all nine Java queue tests passed
for both variants. These checks do not establish production latency gains or
exhaustive concurrent observer behavior. In particular, a sink with no actual
I/O magnifies bookkeeping's contribution to the measured total.

## Evidence and decision

`~/archive/libhft/tx-queue-counter-experiment-20261003/` contains both source
variants, the benchmark and regression sources, the invocation script and the
complete output. The baseline matches the production queue sources at the
experiment's worktree HEAD, `b80005257`:

- C++ SHA-256: `ab1e9f4aa1796625e5439a375f9e8d76b22bc52499a9a79b9367f8cd2dfdd388`.
- Java SHA-256: `657f4ad971c07f6b22b7abf2ce39e344c3af9168a8f858dcee21ae20eff22267`.

Keep the candidate out of production until real transport checks have run.
Do not apply these percentages to the public round-trip matrix. The archived
script records the original session paths; reproducing it elsewhere requires
pointing those paths at the archived inputs and a new build directory.

## Concurrent verification

At source checkpoint `e154e0791`, a separate producer, consumer and statistics
observer exercised both archived variants. Each language and variant ran four
one-million-frame success cases and four cases that deliberately failed the
sink after 500,000 writes. Variable-length frames used a small 1,024-byte slab
to force wrapping and backpressure. Every accepted payload was checked for
sequence, length and contents. Observers checked each counter for monotonicity;
the independently published counters were not treated as an atomic snapshot.
Final counts were checked after thread joins. Injected failures latched, and
100 subsequent drain attempts never called the sink again.

All 32 optimized-build cases passed: **24 million successful frame writes**
across both languages and variants. Another 16 smaller C++ cases passed with
AddressSanitizer and UndefinedBehaviorSanitizer, accounting for 1.2 million
successful frame writes. This is not a ThreadSanitizer result and does not
establish a network latency gain.

The first unpinned attempts timed out without detecting a payload fault. All
three threads had remained on the same isolated CPU despite an affinity mask
allowing CPUs 8–17. Isolated CPUs do not automatically spread this workload.
Explicitly pinning the producer, consumer and observer to CPUs 8, 9 and 10
removed that test-placement problem. The corrected runs used an `isol` lease,
overlapping unrelated shared-core work: they are functional stress checks,
not quiet-host measurements. Java used the verified JNI library only for
one-time affinity calls; both queue implementations remain the archived ones.

The archive now also retains the stress sources, invocation script, source and
pinning-library hashes, the earlier diagnostic logs and the complete corrected
output. Production queue sources remain unchanged. The earlier serial timing
numbers have not been replaced by timings from these concurrent runs.

A caller search at this checkpoint also found no production instantiation of
these queue utilities in the C++, Java or .NET engines: only their definitions
and tests. Optimizing them would therefore not accelerate the current default
benchmark path. Wiring a dedicated TX queue into a runner is a separate change
with the ordering and durability trade-offs described by each utility.