perf: RTS test64 GC tests allocate 192MB per seed (should be much smaller)

[ggreif]

Problem

The incremental GC unit tests (test64) allocate 3 * PARTITION_SIZE = 192 MB per GC test invocation, where PARTITION_SIZE is 64 MB (designed for production canisters). With 100 random heap seeds, this means ~19.2 GB of allocation for testing tiny heaps (~180 small objects each).

This makes test64 orders of magnitude slower than the wasm32 GC tests, which scale heap size proportionally to the actual object count.

Evidence

• test32-non-incremental: runs 3 GC variants × 100 seeds in seconds
• test64: runs 1 GC variant (incremental) × 100 seeds in ~40+ minutes
• The difference is entirely due to heap_size_for_gc returning 3 * PARTITION_SIZE (192 MB) regardless of object count

Suggested fix

Use a smaller partition size for tests, e.g.:

#[cfg(test)]
pub const PARTITION_SIZE: usize = 256 * 1024; // 256 KB for tests

#[cfg(not(test))]
pub const PARTITION_SIZE: usize = 64 * 1024 * 1024; // 64 MB for production

Or parametrise heap_size_for_gc to scale with the actual heap content, similar to the non-incremental variants.

Context

Found while investigating RTS test parallelism (#6029). The parallel test framework works correctly, but the speedup is limited because test64 dominates wall clock time.

Relevant code:

• rts/motoko-rts-tests/src/gc/heap.rs:345-349
• rts/motoko-rts/src/gc/incremental/partitioned_heap.rs (PARTITION_SIZE)

[ggreif]

Update: reducing PARTITION_SIZE to 2 MB (via a small_partitions cargo feature) did NOT speed up the tests. The heap allocation (192 MB Box<[u8]>) is nearly instant on the host — the bottleneck is in the incremental GC algorithm itself, not the memory footprint.

The incremental GC does per-partition sweeps, bitmap management, write barriers, and multi-pass collection that is inherently expensive regardless of partition size. The 100 random heap seeds × the incremental GC's evacuation/compaction passes are the real cost.

Profiling the GC test loop (e.g. with perf or samply on the wasmtime process) would identify the actual hotspot. Possible candidates:

• Bitmap allocation/iteration per partition
• Write barrier bookkeeping in the remembered set
• Evacuation pass scanning all partitions even when only a few objects exist
• The test heap construction itself (MotokoHeap::new and generate)

[alexandru-uta]

I;m not really sure testing with a smaller partition tells us much. I'd rather have similar tests with prod environments; perhaps in test we should even have a plethora of partition sizes (but including 64MB), just to make sure we catch size-related bugs.

Also, CI seems fast enough for now (haven't seen more than 20 mins)..

[ggreif]

Funny we had asked the same questions. A seed of 20000 for the 20000-tree caused a 50-min build, if I read that correctly. In the other PR I am switching to randomised (but still deterministic) seeds to explore configurations. This will be important as we start adding (cautiously) more types of heap objects. That some of those seeds exposed a max-heap calculation bug (but not a bug in the GC).

[ggreif]

I think this can be closed. I also haven't seen speedups due to limiting the heap sizes.

[ggreif]

Also, CI seems fast enough for now (haven't seen more than 20 mins)..

That's probably when you don't touch the RTS, and it is coming from the pre-built cache.