Skip to content

Complete SVE coverage of ARM SIMD kernels; add an SVE2 dispatch level #1697

Description

@xiaofan-luan

Current state

The ARM runtime dispatch in src/simd/hook.cc is incomplete and only knows a single SVE level:

  1. SVE coverage gaps (hook.cc:386-427, the block itself carries // ToDo: Enable remaining functions on SVE):
    • ivec_inner_product / ivec_L2sqr fall back to the NEON versions on SVE machines.
    • fp16_vec_inner_product_batch_4 / fp16_vec_L2sqr_batch_4 are not assigned in the SVE branch at all — on SVE machines they keep their default (ref/scalar) values while the NEON branch does assign _neon versions. Likely an oversight; SVE machines run scalar fp16 batch_4 today.
  2. No SVE2 anywhere: no +sve2 compile option, no SVE2 intrinsics, and supports_sve() (hook.cc:151) only probes HWCAP_SVE.
  3. Latent armv9 hazard: the SVE flag ladder prefers -march=armv9-a+sve+bf16 (CMakeLists.txt:218, cmake/libs/libfaiss.cmake:266) when the compiler accepts it. armv9-a implies SVE2, so the compiler is allowed to emit SVE2 encodings (e.g. via autovectorization inside those TUs) — but runtime gating only checks SVE1. A binary built with an armv9-capable compiler can in principle fault on an SVE1-only core (Graviton3). The compile flag level and the runtime probe level must match.

Proposal

  1. Finish SVE coverage: implement/wire the missing ivec_* and fp16 batch_4 SVE kernels (or at minimum assign the NEON versions instead of leaving ref).
  2. Introduce a SIMD level enum for ARM (NEON < SVE < SVE2) detected once via getauxval(AT_HWCAP/AT_HWCAP2) (HWCAP_SVE, HWCAP2_SVE2), replacing the boolean supports_sve().
  3. Split compile options accordingly: SVE variant TUs built with an armv8 +sve march (safe on Graviton3), a separate SVE2 variant TU set built with armv9-a/+sve2 and only selected when HWCAP2 reports SVE2 (Graviton4 / Axion / Cobalt). Keep check_cxx_compiler_flag ladders so old toolchains (gcc < 12 lacks armv9-a) skip the SVE2 tier gracefully instead of breaking the build.
  4. Adopt SVE2-specific instructions selectively, only where they measurably win. Note from a first pass: dense distance kernels gain little from SVE2 itself (dot/popcount are SVE1); realistic candidates are compaction-style postprocessing (e.g. top-k threshold filtering in sindi_simd_sve.cc batch_insert — svcompact is even SVE1) and any future set-intersection kernels (svmatch/svhistcnt). The main value of the SVE2 tier is letting the compiler use SVE2 encodings safely + a place to land such kernels.

Related: #1696 (runtime dispatch / -march=native policy).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions