test: isolate the OOM harness per iteration; re-land the ReleaseSafe promotion

aaabb6a promoted CI's ReleaseSafe lane to the full suite and b4873d8 backed it
out, because `L4 Join proxy relay rolls back state under OOM injection` failed
under ReleaseSafe on both amd64 tiers. Root-caused: THE ROLLBACK CODE WAS NEVER
AT FAULT. All 34 injected failures rolled back cleanly, no leak, no swallowed
OOM. The harness was unsound.

`std.testing.checkAllAllocationFailures` shares ONE backing allocator across
every fail-index iteration, coupling the runs through residual heap state.
`FailingAllocator` bumps `alloc_index` only in `alloc`; `resize`/`remap` bump a
separate counter it never fails. So a buffer that needs a fresh `alloc` on a
cold heap can grow IN PLACE on a warm one -- one allocation disappears, the last
fail index is never reached, and a deterministic function is reported as
`NondeterministicMemoryUsage`.

Measured on ubuntu-latest under ReleaseSafe, via a throwaway branch because the
CI log printed no verdict:

    reference allocs = 35
    ANOMALY fail_index=34 induced=false alloc_index=34 resize_index=6
    anomalies = 1

Exactly one anomaly, at the LAST index only -- for every earlier index the
injected failure fires before that allocation is reached. Six identical plain
runs all allocated exactly 35, so the impl itself is deterministic. It is
platform-specific because whether growth happens in place is an allocator and
heap-layout decision: linux-aarch64 (verified in a container) and macOS never
reproduce it.

`harness.checkAllAllocationFailuresIsolated` gives every iteration a pristine
DebugAllocator. That removes the coupling and keeps per-iteration leak
detection; with a fresh heap each time an unreached fail index is a real signal
rather than an artifact. The previous arrangement could silently under-test
rollback paths on any platform where growth happens in place -- which is the
more important half of this finding.

With that fixed the promotion returns: CI runs `zig build test
-Doptimize=ReleaseSafe`, green on all three tiers.

A correction to the record while diagnosing this: I claimed an emulated amd64
container reproduced the failure. It did not -- that container died with
`rosetta error: bss_size overflow`, i.e. Apple's x86 emulation failing to run
the Zig compiler, and I read exit 133 as a reproduction. The architecture
correlation is real, but it rests on the CI runs and the aarch64 container, not
on that.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Author: nullstyle

.github/workflows/ci.yml

@@ -442,8 +442,8 @@ jobs:
       # fuzz-smoke and two transport files, so most of the RPC runtime -- peer,
       # caps, promises, vat, integration -- is still never executed with safety
       # checks on in an optimized build.
-      - name: Run ReleaseSafe hardening tests
-        run: zig build test-release-safe --summary all
+      - name: Run the full suite under ReleaseSafe
+        run: zig build test -Doptimize=ReleaseSafe --summary all
 
       # ReleaseFast, not for speed but because it is the only mode that does NOT
       # poison a freed pointer. A use-after-free reached from a destructor is

CHANGELOG.md

@@ -104,6 +104,57 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   that can carry RPC today — see the Evented note above. The compile check is
   kept as a cheap cross-check that the evented selector still builds.
 
+- **CI runs the full suite under ReleaseSafe**, not the ten-binary
+  `test-release-safe` subset. That subset covered message, codegen, framing,
+  fuzz-smoke and two transport files — leaving most of the RPC runtime (peer,
+  caps, promises, vat, integration) never executed with safety checks on in an
+  optimized build. Ablation-verified both ways: a deliberately failing assertion
+  in `tests/rpc/peer/rpc_peer_test.zig` leaves `zig build test-release-safe`
+  **green** and turns `zig build test -Doptimize=ReleaseSafe` red. The narrow
+  step remains for a quick local pass and for the nightly job.
+
+- **The OOM-injection harness was unsound, and the ReleaseSafe lane found it on
+  its first run.** `L4 Join proxy relay rolls back state under OOM injection`
+  failed under ReleaseSafe on both amd64 tiers while passing on macOS and
+  linux-aarch64. The rollback code was never at fault — all 34 injected failures
+  rolled back cleanly, with no leak and no swallowed OOM.
+
+  `std.testing.checkAllAllocationFailures` shares **one** backing allocator
+  across every fail-index iteration, which couples the runs through residual
+  heap state. `FailingAllocator` bumps `alloc_index` only in `alloc`, while
+  `resize`/`remap` bump a separate counter it never fails — so a buffer that
+  needs a fresh `alloc` on a cold heap can grow *in place* on a warm one. One
+  allocation disappears, the last fail index is never reached, and a perfectly
+  deterministic function is reported as `NondeterministicMemoryUsage`. Measured
+  on ubuntu-latest: reference run 35 allocations; iteration 34 made 34
+  allocations and 6 resizes with no failure induced — exactly one anomaly, at
+  the last index only, because for every earlier index the injected failure
+  fires first. It is platform-dependent because in-place growth is an
+  allocator/heap-layout decision.
+
+  `harness.checkAllAllocationFailuresIsolated` gives each iteration a pristine
+  `DebugAllocator`, which removes the coupling and keeps per-iteration leak
+  detection. With a fresh heap every time, an unreached fail index becomes a
+  real signal instead of an artifact. Worth stating plainly: the previous
+  arrangement could silently under-test rollback paths on any platform where
+  growth happens in place.
+
+- **A lane that actually executes an `std.Io` backend selector.** The
+  `evented-check` job ran `zig build -Dio-backend=evented check`, which verified
+  nothing that plain `zig build check` did not already: `-Dio-backend` is a
+  `[]const u8` compared at *runtime* by `io_backend.parseKind`, so all three
+  arms of `Backend.init` are semantically analysed in every configuration.
+  Ablation-proven — breaking the `.evented` arm turns plain `zig build check`
+  red with no flag passed at all.
+
+  What was missing was any lane that *executes* a selector. `just
+  check-selector` / `zig build -Dio-backend=threaded e2e-self` now runs the RPC
+  e2e over an explicitly chosen backend, in CI and in `just ci`. Its own
+  ablation: making `parseKind("threaded")` return null leaves the old compile
+  check **green** and turns the new lane red. `.threaded` is the only selector
+  that can carry RPC today — see the Evented note above. The compile check is
+  kept as a cheap cross-check that the evented selector still builds.
+
 ### Known issues
 
 - **`L4 Join proxy relay rolls back state under OOM injection` fails under

tests/rpc/peer/rpc_join_readiness_test.zig

@@ -2857,7 +2857,12 @@ fn crossPeerJoinRelayOomImpl(allocator: std.mem.Allocator) !void {
 }
 
 test "L4 Join proxy relay rolls back state under OOM injection" {
-    try std.testing.checkAllAllocationFailures(std.testing.allocator, crossPeerJoinRelayOomImpl, .{});
+    // Isolated backing allocator per iteration. `std.testing.
+    // checkAllAllocationFailures` shares one across every fail index, and that
+    // coupling reported this deterministic impl as NondeterministicMemoryUsage
+    // under ReleaseSafe on both amd64 tiers while passing on macOS and
+    // linux-aarch64. See the helper for the measured evidence.
+    try harness.checkAllAllocationFailuresIsolated(crossPeerJoinRelayOomImpl, .{});
 }
 
 test "L4 JoinResult send failure drains pending direct Accept state" {

tests/rpc/peer/three_party_handoff_harness.zig

@@ -78,3 +78,55 @@ pub fn expectNoCrossPeerProxyLinks(peer: *const Peer) !void {
     try std.testing.expectEqual(@as(usize, 0), peer.cross_peer_proxy_links.items.len);
     try std.testing.expectEqual(@as(usize, 0), peer.cross_peer_join_relay_links.items.len);
 }
+
+/// `std.testing.checkAllAllocationFailures`, but with a pristine backing
+/// allocator per iteration.
+///
+/// The std version shares ONE backing allocator across every fail-index
+/// iteration, which couples the runs together through residual heap state.
+/// That is enough to make it report `NondeterministicMemoryUsage` for code
+/// that is in fact perfectly deterministic: `FailingAllocator` bumps
+/// `alloc_index` only in `alloc`, while `resize`/`remap` bump a separate
+/// counter it never fails, so a buffer that needs a fresh `alloc` on a cold
+/// heap can grow in place on a warm one. One allocation disappears, the final
+/// fail index is never reached, and the run is reported as nondeterministic.
+///
+/// Measured on `crossPeerJoinRelayOomImpl` (ubuntu-latest, ReleaseSafe): the
+/// reference run made 35 allocations; iteration 34 made 34 allocations and 6
+/// resizes, with no failure induced. Every one of the 34 injected failures
+/// rolled back correctly — the rollback code was never the problem. macOS and
+/// linux-aarch64 did not reproduce it, because whether growth can happen in
+/// place is allocator- and platform-dependent.
+///
+/// Giving each iteration its own allocator removes the coupling, and keeps
+/// per-iteration leak detection.
+pub fn checkAllAllocationFailuresIsolated(
+    comptime test_fn: anytype,
+    extra_args: anytype,
+) !void {
+    const needed = blk: {
+        var dbg: std.heap.DebugAllocator(.{}) = .init;
+        defer std.debug.assert(dbg.deinit() == .ok);
+        var fa = std.testing.FailingAllocator.init(dbg.allocator(), .{});
+        try @call(.auto, test_fn, .{fa.allocator()} ++ extra_args);
+        break :blk fa.alloc_index;
+    };
+
+    for (0..needed) |fail_index| {
+        var dbg: std.heap.DebugAllocator(.{}) = .init;
+        defer std.debug.assert(dbg.deinit() == .ok);
+        var fa = std.testing.FailingAllocator.init(dbg.allocator(), .{ .fail_index = fail_index });
+        if (@call(.auto, test_fn, .{fa.allocator()} ++ extra_args)) |_| {
+            // A fresh heap every iteration means the alloc sequence is the
+            // same one the reference run took, so an unreached fail index is
+            // a real signal rather than an artifact.
+            if (fa.has_induced_failure) return error.SwallowedOutOfMemoryError;
+            return error.NondeterministicMemoryUsage;
+        } else |err| switch (err) {
+            error.OutOfMemory => {
+                if (fa.allocated_bytes != fa.freed_bytes) return error.MemoryLeakDetected;
+            },
+            else => |e| return e,
+        }
+    }
+}