Current state
The ARM runtime dispatch in src/simd/hook.cc is incomplete and only knows a single SVE level:
- SVE coverage gaps (
hook.cc:386-427, the block itself carries // ToDo: Enable remaining functions on SVE):
ivec_inner_product / ivec_L2sqr fall back to the NEON versions on SVE machines.
fp16_vec_inner_product_batch_4 / fp16_vec_L2sqr_batch_4 are not assigned in the SVE branch at all — on SVE machines they keep their default (ref/scalar) values while the NEON branch does assign _neon versions. Likely an oversight; SVE machines run scalar fp16 batch_4 today.
- No SVE2 anywhere: no
+sve2 compile option, no SVE2 intrinsics, and supports_sve() (hook.cc:151) only probes HWCAP_SVE.
- Latent armv9 hazard: the SVE flag ladder prefers
-march=armv9-a+sve+bf16 (CMakeLists.txt:218, cmake/libs/libfaiss.cmake:266) when the compiler accepts it. armv9-a implies SVE2, so the compiler is allowed to emit SVE2 encodings (e.g. via autovectorization inside those TUs) — but runtime gating only checks SVE1. A binary built with an armv9-capable compiler can in principle fault on an SVE1-only core (Graviton3). The compile flag level and the runtime probe level must match.
Proposal
- Finish SVE coverage: implement/wire the missing
ivec_* and fp16 batch_4 SVE kernels (or at minimum assign the NEON versions instead of leaving ref).
- Introduce a SIMD level enum for ARM (NEON < SVE < SVE2) detected once via
getauxval(AT_HWCAP/AT_HWCAP2) (HWCAP_SVE, HWCAP2_SVE2), replacing the boolean supports_sve().
- Split compile options accordingly: SVE variant TUs built with an armv8
+sve march (safe on Graviton3), a separate SVE2 variant TU set built with armv9-a/+sve2 and only selected when HWCAP2 reports SVE2 (Graviton4 / Axion / Cobalt). Keep check_cxx_compiler_flag ladders so old toolchains (gcc < 12 lacks armv9-a) skip the SVE2 tier gracefully instead of breaking the build.
- Adopt SVE2-specific instructions selectively, only where they measurably win. Note from a first pass: dense distance kernels gain little from SVE2 itself (dot/popcount are SVE1); realistic candidates are compaction-style postprocessing (e.g. top-k threshold filtering in
sindi_simd_sve.cc batch_insert — svcompact is even SVE1) and any future set-intersection kernels (svmatch/svhistcnt). The main value of the SVE2 tier is letting the compiler use SVE2 encodings safely + a place to land such kernels.
Related: #1696 (runtime dispatch / -march=native policy).
Current state
The ARM runtime dispatch in
src/simd/hook.ccis incomplete and only knows a single SVE level:hook.cc:386-427, the block itself carries// ToDo: Enable remaining functions on SVE):ivec_inner_product/ivec_L2sqrfall back to the NEON versions on SVE machines.fp16_vec_inner_product_batch_4/fp16_vec_L2sqr_batch_4are not assigned in the SVE branch at all — on SVE machines they keep their default (ref/scalar) values while the NEON branch does assign_neonversions. Likely an oversight; SVE machines run scalar fp16 batch_4 today.+sve2compile option, no SVE2 intrinsics, andsupports_sve()(hook.cc:151) only probesHWCAP_SVE.-march=armv9-a+sve+bf16(CMakeLists.txt:218,cmake/libs/libfaiss.cmake:266) when the compiler accepts it. armv9-a implies SVE2, so the compiler is allowed to emit SVE2 encodings (e.g. via autovectorization inside those TUs) — but runtime gating only checks SVE1. A binary built with an armv9-capable compiler can in principle fault on an SVE1-only core (Graviton3). The compile flag level and the runtime probe level must match.Proposal
ivec_*and fp16batch_4SVE kernels (or at minimum assign the NEON versions instead of leaving ref).getauxval(AT_HWCAP/AT_HWCAP2)(HWCAP_SVE,HWCAP2_SVE2), replacing the booleansupports_sve().+svemarch (safe on Graviton3), a separate SVE2 variant TU set built witharmv9-a/+sve2and only selected when HWCAP2 reports SVE2 (Graviton4 / Axion / Cobalt). Keepcheck_cxx_compiler_flagladders so old toolchains (gcc < 12 lacksarmv9-a) skip the SVE2 tier gracefully instead of breaking the build.sindi_simd_sve.ccbatch_insert —svcompactis even SVE1) and any future set-intersection kernels (svmatch/svhistcnt). The main value of the SVE2 tier is letting the compiler use SVE2 encodings safely + a place to land such kernels.Related: #1696 (runtime dispatch / -march=native policy).