Add native THD packed ASR encoders

[pzelasko]

Summary

• add a native THD packed-sequence API for Transformer and MoE ASR encoders
• make PEE execute packed experts layer-synchronously with grouped QKV/output projections, grouped attention, grouped dense FFNs, and ragged grouped_mm top-k MoE where supported
• integrate packed encoder outputs through Perception, SALM, chunk microbatching, context parallelism, activation checkpointing, and FSDP2
• preserve historical padded behavior, strict loading of existing checkpoints, and state-dict compatibility
• add a benchmark harness, design document, aggregate performance report, and Nsight Systems kernel/API analysis

This is a dependent PR based on #16048 and targets its head branch, moe_pee_ggemm_encoder_chunking.

Motivation

The existing packed path uses FlexAttention's sparse masks but still materializes padded BHSD activations. Native THD execution avoids that memory cost. PEE additionally needs experts to remain parallel; the production path therefore groups compatible experts per layer, while serial THD remains only as a numerical oracle and compatibility fallback.

Performance

BF16 measurements on an RTX PRO 6000 Blackwell:

• Transformer: 2.37x inference / 2.13x training speedup, with 49.7% / 59.0% lower incremental peak memory
• MoE: 1.41x inference / 1.39x training speedup, with 53.3% / 58.1% lower incremental peak memory
• PEE grouped training: 1.56x faster than legacy and 1.17x faster than serial THD, with 15.0% lower incremental peak memory than legacy
• PEE grouped top-k: 72.8% lower incremental peak memory than legacy
• Nsight Systems: grouped PEE reduces kernel count by 44.3% and CUDA API calls by 53.0% versus serial THD

The committed benchmark report contains the complete methodology, aggregate measurements, numerical preflight, source hashes, and profiler queries. Raw generated JSON trials remain local to keep the review diff focused.

Compatibility

• existing Transformer, MoE, and PEE checkpoints load strictly before and after packed execution
• the new path adds no state-dict keys
• historical padded configuration defaults are unchanged
• unsupported or heterogeneous custom expert configurations use correct packed fallbacks
• CPU/autograd and unsupported grouped-kernel configurations use differentiable reference/fallback implementations

Validation

• 263 focused tests pass on the exact rebased commit
• coverage includes BHSD-vs-THD outputs and gradients, causal/full attention, positional encodings, empty rows/ranks, checkpointing, grouped PEE launch structure, MoE routing, legacy checkpoints, fusion, chunking, context parallelism, and distributed FSDP2
• BF16 numerical comparisons pass rtol=0.03, atol=0.03; relative L2 differences are approximately 0.004-0.005
• all changed Python files pass Black, isort, and source compilation
• YAML/JSON/report/profiler artifacts parse and the benchmark CLI smoke test passes

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[pzelasko]

/ok to test 4a4f436

[pzelasko]

/ok to test c0fc91f

[pzelasko]

/ok to test 4b242b5

[pzelasko]

/ok to test a86b4a0