Repository-wide audit and 5.0.0 completion: SIGILL in shipped wheels, KyberSlash divisions, broken SVE2 kernels, and the gates and lanes that could not fail #2317
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Copyright (C) 2025-2026 Steel Security Advisors LLC | |
| # SPDX-License-Identifier: Apache-2.0 | |
| name: CI - Build and Test | |
| on: | |
| push: | |
| branches: [ main, develop, 'feature/**', 'fix/**' ] | |
| pull_request: | |
| branches: [ main, develop ] | |
| workflow_dispatch: | |
| # Least privilege, matching every other workflow in this directory. This file | |
| # was the one without a `permissions:` block, so its jobs — including the | |
| # `security` job that runs bandit, pip-audit and the Bandit severity gate — | |
| # inherited whatever the repository or organisation default is. Where that | |
| # default is "read and write", those jobs held write access to contents, | |
| # issues and packages for no reason at all. | |
| permissions: | |
| contents: read | |
| # Collapse overlapping runs on the same ref; feature branches and PR | |
| # heads cancel in-flight runs, main is preserved. | |
| concurrency: | |
| group: ci-build-test-${{ github.workflow }}-${{ github.ref }} | |
| cancel-in-progress: ${{ github.ref != 'refs/heads/main' && github.event_name != 'schedule' }} | |
| jobs: | |
| # C Library Build and Test | |
| c-library: | |
| name: C Library (${{ matrix.os }}, ${{ matrix.compiler }}) | |
| runs-on: ${{ matrix.os }} | |
| timeout-minutes: 15 | |
| strategy: | |
| fail-fast: false | |
| matrix: | |
| os: [ubuntu-latest, macos-latest] | |
| compiler: [gcc, clang] | |
| include: | |
| - os: ubuntu-latest | |
| compiler: gcc | |
| cc: gcc-13 | |
| cxx: g++-13 | |
| - os: ubuntu-latest | |
| compiler: clang | |
| cc: clang-18 | |
| cxx: clang++-18 | |
| - os: macos-latest | |
| compiler: gcc | |
| cc: gcc-12 | |
| cxx: g++-12 | |
| - os: macos-latest | |
| compiler: clang | |
| cc: clang | |
| cxx: clang++ | |
| steps: | |
| - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | |
| - name: Install dependencies (Ubuntu) | |
| if: matrix.os == 'ubuntu-latest' | |
| run: | | |
| sudo rm -f /etc/apt/sources.list.d/microsoft-prod.list /etc/apt/sources.list.d/azure-cli.list || true | |
| .github/scripts/apt-install.sh cmake libssl-dev ${{ matrix.cc }} ${{ matrix.cxx }} | |
| - name: Install dependencies (macOS) | |
| if: matrix.os == 'macos-latest' | |
| run: | | |
| brew install cmake gcc@12 | |
| - name: Configure CMake | |
| run: | | |
| mkdir -p build | |
| cd build | |
| # Set compiler path | |
| if [ "$RUNNER_OS" = "macOS" ]; then | |
| # For GCC on macOS, use full path | |
| if [ "${{ matrix.compiler }}" = "gcc" ]; then | |
| CC_PATH=$(brew --prefix gcc@12)/bin/gcc-12 | |
| CXX_PATH=$(brew --prefix gcc@12)/bin/g++-12 | |
| else | |
| CC_PATH=${{ matrix.cc }} | |
| CXX_PATH=${{ matrix.cxx }} | |
| fi | |
| else | |
| CC_PATH=${{ matrix.cc }} | |
| CXX_PATH=${{ matrix.cxx }} | |
| fi | |
| cmake .. \ | |
| -DCMAKE_C_COMPILER=${CC_PATH} \ | |
| -DCMAKE_CXX_COMPILER=${CXX_PATH} \ | |
| -DCMAKE_BUILD_TYPE=Release \ | |
| -DAMA_BUILD_SHARED=ON \ | |
| -DAMA_BUILD_STATIC=ON \ | |
| -DAMA_BUILD_TESTS=ON \ | |
| -DAMA_ENABLE_AVX2=ON | |
| - name: Build | |
| # PR #326 follow-up: re-emit per-command verbose output on | |
| # failure so the actual compiler diagnostic surfaces in the | |
| # job log without needing local repro. The parallel-j4 form | |
| # is the fast happy path; the serial-verbose form runs only | |
| # if the fast path returned non-zero AND propagates that | |
| # non-zero exit so the step still fails. Diagnoses the | |
| # otherwise opaque "Process completed with exit code 2" | |
| # surfaced on the macos-latest-clang lane that's been red | |
| # since `58e7a2d` introduced the dispatch cache code. | |
| shell: bash | |
| run: | | |
| set -o pipefail | |
| # Capture the BUILD's exit status, not the `if` statement's. In bash | |
| # an `if` whose condition fails and whose branches do not run returns | |
| # 0, so the previous `if cmake …; then exit 0; fi; rc=$?` form always | |
| # recorded rc=0 and the final `exit "$rc"` reported success — a failed | |
| # compile or link went green whenever ctest did not independently | |
| # notice (e.g. the shared library failing to link while the tests link | |
| # the static one). Assign from the command itself. | |
| cmake --build build --config Release -j4 && exit 0 | |
| rc=$? | |
| echo "::group::Re-running build with --verbose to surface the failing command" | |
| # `--clean-first` discards the partial build state from the | |
| # failed parallel pass so the verbose pass actually re-runs | |
| # the failing compile (otherwise cmake/ninja/make caches the | |
| # failure and just exits non-zero without re-emitting it). | |
| cmake --build build --config Release --verbose --clean-first -j1 \ | |
| 2>&1 | tee build-verbose.log || true | |
| echo "::endgroup::" | |
| # First few lines of the verbose log are the most useful — | |
| # they show the failing TU and the diagnostic text immediately, | |
| # without the reviewer having to scroll through every | |
| # successful compile that ran before it. | |
| if [ -s build-verbose.log ]; then | |
| echo "::group::First failure block from verbose log" | |
| grep -m1 -B0 -A40 'error:' build-verbose.log || tail -80 build-verbose.log | |
| echo "::endgroup::" | |
| fi | |
| exit "$rc" | |
| - name: Test | |
| # `--output-on-failure` + `--verbose` together print the full | |
| # stdout/stderr of every failing test directly into the CI log, | |
| # plus a one-line per-passing-test trace. Without `--verbose` | |
| # ctest swallows the diagnostic output of an exit-code-only | |
| # failure, which is how the `C Library (*, clang)` lanes | |
| # previously reported only "Process completed with exit code 8" | |
| # without ever revealing which 8 tests failed. PR #326 | |
| # follow-up: every failing C test now self-diagnoses in the | |
| # job log. | |
| run: cd build && ctest --output-on-failure --verbose | |
| # --------------------------------------------------------------------- | |
| # AMA_USE_NATIVE_PQC=OFF configuration guard | |
| # --------------------------------------------------------------------- | |
| # ON is and remains the default (top-level CMakeLists.txt) and the only | |
| # configuration setup.py will build — INVARIANT-7 forbids a cryptographic | |
| # fallback, so the shipped wheel always carries the native PQC backends. | |
| # OFF is nonetheless a supported CMake configuration for downstream | |
| # packagers who take PQC from elsewhere, and it is exactly the kind of | |
| # cell that silently rots when nothing builds it: before this job existed, | |
| # `-DAMA_USE_NATIVE_PQC=OFF` left dispatch code referencing four Kyber/ | |
| # Dilithium NTT reference symbols defined only in the gated PQC source | |
| # group. | |
| # | |
| # The observable failure is toolchain-dependent, and the guard catches it | |
| # either way. On this Linux lane GNU ld does NOT set --no-undefined on the | |
| # shared library, so libama_cryptography.so links with those four symbols | |
| # left UNDEFINED; what actually fails is every EXECUTABLE that links it — | |
| # the always-built test binaries and the non-PQC examples — with | |
| # "undefined reference to ama_dilithium_ntt_generic_ref". On macOS/Windows | |
| # the shared-library link itself would fail. The build step below fails on | |
| # the first of these, and the ctest step then proves the tests that survive | |
| # the gate (including test_agent_binding, which needs no PQC symbol) still | |
| # pass rather than merely compile. Fail-closed: no continue-on-error. | |
| c-library-no-native-pqc: | |
| name: C Library (AMA_USE_NATIVE_PQC=OFF configuration guard) | |
| runs-on: ubuntu-latest | |
| timeout-minutes: 15 | |
| steps: | |
| - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | |
| - name: Install dependencies | |
| run: | | |
| sudo rm -f /etc/apt/sources.list.d/microsoft-prod.list /etc/apt/sources.list.d/azure-cli.list || true | |
| .github/scripts/apt-install.sh cmake gcc-13 g++-13 | |
| - name: Configure CMake (native PQC disabled) | |
| run: | | |
| cmake -B build-nopqc \ | |
| -DCMAKE_C_COMPILER=gcc-13 \ | |
| -DCMAKE_CXX_COMPILER=g++-13 \ | |
| -DCMAKE_BUILD_TYPE=Release \ | |
| -DAMA_USE_NATIVE_PQC=OFF \ | |
| -DAMA_BUILD_SHARED=ON \ | |
| -DAMA_BUILD_STATIC=ON \ | |
| -DAMA_BUILD_TESTS=ON \ | |
| -DAMA_BUILD_EXAMPLES=ON | |
| - name: Build | |
| run: cmake --build build-nopqc --config Release -j4 | |
| - name: Test | |
| run: cd build-nopqc && ctest --output-on-failure | |
| - name: Assert the default is still ON | |
| # A regression that flipped the default would make the job above green | |
| # for the wrong reason (it would test the default path twice). Assert | |
| # CMake's RESOLVED value from a default configure (no -D override), not | |
| # a regex over the source line: a source regex both false-passes on a | |
| # trailing '... ON)' comment and false-fails on CMake-valid ON spellings | |
| # (TRUE / 1 / lowercase on). This reads the value CMake actually | |
| # computed, so it is immune to how the option line is written. | |
| shell: bash | |
| run: | | |
| set -o pipefail | |
| cmake -B build-default-probe \ | |
| -DCMAKE_C_COMPILER=gcc-13 \ | |
| -DCMAKE_CXX_COMPILER=g++-13 \ | |
| -DCMAKE_BUILD_TYPE=Release >/dev/null | |
| resolved=$(cmake -LA -N build-default-probe | grep '^AMA_USE_NATIVE_PQC:' || true) | |
| echo "resolved default: ${resolved:-<absent>}" | |
| if [ "$resolved" != "AMA_USE_NATIVE_PQC:BOOL=ON" ]; then | |
| echo "::error::AMA_USE_NATIVE_PQC no longer defaults to ON (resolved: ${resolved:-<absent>})" | |
| exit 1 | |
| fi | |
| echo "AMA_USE_NATIVE_PQC default is ON." | |
| # Python Package Build and Test | |
| python-package: | |
| name: Python ${{ matrix.python-version }} on ${{ matrix.os }} | |
| runs-on: ${{ matrix.os }} | |
| timeout-minutes: 20 | |
| strategy: | |
| fail-fast: false | |
| matrix: | |
| os: [ubuntu-latest, macos-latest, windows-latest] | |
| python-version: ['3.10', '3.11', '3.12', '3.13', '3.14'] | |
| steps: | |
| - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | |
| - name: Set up Python ${{ matrix.python-version }} | |
| id: setup-python | |
| uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 | |
| continue-on-error: true | |
| with: | |
| python-version: ${{ matrix.python-version }} | |
| # First-party retry path: actions/setup-python intermittently fails on | |
| # GitHub-hosted Windows runners (toolcache / cache-restore flake; see | |
| # PR #306). Only runs on the first miss, so happy path is unchanged. | |
| - name: Set up Python ${{ matrix.python-version }} (retry) | |
| if: steps.setup-python.outcome == 'failure' | |
| uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 | |
| with: | |
| python-version: ${{ matrix.python-version }} | |
| - name: Install system dependencies (Ubuntu) | |
| if: matrix.os == 'ubuntu-latest' | |
| run: | | |
| sudo rm -f /etc/apt/sources.list.d/microsoft-prod.list /etc/apt/sources.list.d/azure-cli.list || true | |
| .github/scripts/apt-install.sh build-essential python3-dev cmake libssl-dev | |
| - name: Install system dependencies (macOS) | |
| if: matrix.os == 'macos-latest' | |
| run: brew install cmake | |
| - name: Install system dependencies (Windows) | |
| if: matrix.os == 'windows-latest' | |
| shell: pwsh | |
| run: | | |
| # CMake is pre-installed on GitHub-hosted Windows runners. | |
| # Only install via Chocolatey (with retries) if it is missing, | |
| # so a transient Chocolatey CDN outage cannot break the build. | |
| if (Get-Command cmake -ErrorAction SilentlyContinue) { | |
| Write-Host "CMake already available: $(cmake --version | Select-Object -First 1)" | |
| } else { | |
| Write-Host "CMake not found; installing via Chocolatey with retries..." | |
| $maxAttempts = 5 | |
| for ($attempt = 1; $attempt -le $maxAttempts; $attempt++) { | |
| Write-Host "Attempt ${attempt}/${maxAttempts}: choco install cmake..." | |
| choco install cmake --installargs 'ADD_CMAKE_TO_PATH=System' -y 2>&1 | |
| if ($LASTEXITCODE -eq 0) { | |
| Write-Host "Successfully installed cmake" | |
| break | |
| } | |
| if ($attempt -lt $maxAttempts) { | |
| $delay = $attempt * 15 | |
| Write-Host "Install failed, waiting ${delay}s before retry..." | |
| Start-Sleep -Seconds $delay | |
| } | |
| } | |
| if ($LASTEXITCODE -ne 0) { | |
| Write-Error "All ${maxAttempts} attempts to install cmake via Chocolatey failed" | |
| exit 1 | |
| } | |
| # Refresh PATH so cmake is available to subsequent steps | |
| Import-Module "$env:ChocolateyInstall\helpers\chocolateyProfile.psm1" | |
| refreshenv | |
| } | |
| - name: Install Python build dependencies | |
| run: | | |
| python -m pip install --upgrade pip setuptools wheel | |
| pip install "Cython>=3.0.0" "numpy>=1.24.0,<3.0.0" | |
| # Pre-build the native C library so the install step can rely on | |
| # an already-populated ``build/`` directory. Mirrors the | |
| # proven-working pattern from ``ci.yml::test`` and avoids the | |
| # latent setup.py CMakeBuild fragility on Windows / macOS that | |
| # surfaces as "Python X.Y on {os} -- exit 1/2" without the | |
| # pre-build. | |
| - name: Build native C library (Linux / macOS) | |
| if: matrix.os != 'windows-latest' | |
| run: | | |
| cmake -B build -DAMA_USE_NATIVE_PQC=ON -DCMAKE_BUILD_TYPE=Release -DAMA_BUILD_TESTS=OFF -DAMA_BUILD_EXAMPLES=OFF | |
| # Use Python's os.cpu_count() for portability — `nproc` is GNU | |
| # coreutils and not guaranteed on macos-latest. Bare `-j` would | |
| # unbound make's parallelism and OOM the runner. | |
| cmake --build build -j"$(python -c 'import os; print(os.cpu_count() or 1)')" | |
| - name: Build native C library (Windows) | |
| if: matrix.os == 'windows-latest' | |
| run: | | |
| cmake -B build -DAMA_USE_NATIVE_PQC=ON -DCMAKE_BUILD_TYPE=Release -DAMA_BUILD_TESTS=OFF -DAMA_BUILD_EXAMPLES=OFF | |
| cmake --build build --config Release --parallel | |
| - name: Install SoftHSM2 (PKCS#11 test token) | |
| # This job runs with AMA_CI_REQUIRE_BACKENDS=1, which promises every | |
| # backend is provisioned — and SoftHSM2 was not, so the only real | |
| # PKCS#11 coverage in the tree skipped here while the flag claimed | |
| # otherwise. ci.yml gained this step in 5.0.0; this workflow needs it | |
| # for the same reason, because it makes the same promise. | |
| # | |
| # Linux via apt, macOS via Homebrew. The first version of this step | |
| # was Linux-only and the macOS gap was written up as a known | |
| # limitation instead of being closed — but `brew install softhsm` | |
| # ships the same PKCS#11 module, so the limitation was a choice, not a | |
| # constraint. Both platforms now execute the real token lifecycle. | |
| # | |
| # Windows went through the same arc one revision later: it was excused | |
| # as "no maintained package on any runner-available manager", but | |
| # Chocolatey's softhsm.install package (wrapping the Disig | |
| # SoftHSM2-for-Windows MSI) is live on the community feed, so that | |
| # claim was also a choice wearing a constraint's clothes. The next | |
| # step provisions it, and all three OSes run the real token lifecycle. | |
| if: runner.os == 'Linux' || runner.os == 'macOS' | |
| shell: bash | |
| run: | | |
| set -euo pipefail | |
| if [ "${RUNNER_OS}" = "Linux" ]; then | |
| sudo rm -f /etc/apt/sources.list.d/microsoft-prod.list /etc/apt/sources.list.d/azure-cli.list || true | |
| .github/scripts/apt-install.sh --no-install-recommends softhsm2 | |
| test -f /usr/lib/softhsm/libsofthsm2.so | |
| else | |
| brew install softhsm | |
| # Homebrew is prefix-relative (/opt/homebrew on arm64, | |
| # /usr/local on x86-64), so the module path is discovered rather | |
| # than hard-coded — the same mistake as the .so.2 SONAME pin. | |
| test -f "$(brew --prefix)/lib/softhsm/libsofthsm2.so" | |
| fi | |
| command -v softhsm2-util | |
| - name: Install SoftHSM2 (PKCS#11 test token, Windows) | |
| if: runner.os == 'Windows' | |
| shell: pwsh | |
| # Chocolatey's softhsm.install embeds the Disig SoftHSM2-for-Windows | |
| # MSI, which statically links OpenSSL 1.1.1 (EOL) — acceptable ONLY as | |
| # a test token that never executes in or for product code, the same | |
| # posture as the OpenSSL-linked apt/brew builds above. Retry shape | |
| # matches the cmake choco step (transient CDN outages). | |
| run: | | |
| # INSTALLDIR is pinned, not assumed. The Disig MSI parents its | |
| # install directory to TARGETDIR, which Windows Installer resolves | |
| # to ROOTDRIVE — the fixed drive with the MOST FREE SPACE, which on | |
| # GitHub's Windows runners is D:, not C:. The first revision of | |
| # this step asserted C:\SoftHSM2\lib\softhsm2-x64.dll and every | |
| # Windows lane in both workflows failed on a *successful* install. | |
| # Pinning the property makes the location deterministic and keeps | |
| # the constant in HSMKeyStorage.PKCS11_PATHS truthful. | |
| $installDir = 'C:\SoftHSM2' | |
| $maxAttempts = 5 | |
| for ($attempt = 1; $attempt -le $maxAttempts; $attempt++) { | |
| Write-Host "Attempt ${attempt}/${maxAttempts}: choco install softhsm.install..." | |
| choco install softhsm.install -y --no-progress --install-arguments "INSTALLDIR=$installDir" 2>&1 | |
| if ($LASTEXITCODE -eq 0) { | |
| Write-Host "Successfully installed softhsm.install" | |
| break | |
| } | |
| if ($attempt -lt $maxAttempts) { | |
| $delay = $attempt * 15 | |
| Write-Host "Install failed, waiting ${delay}s before retry..." | |
| Start-Sleep -Seconds $delay | |
| } | |
| } | |
| if ($LASTEXITCODE -ne 0) { | |
| Write-Error "All ${maxAttempts} attempts to install softhsm.install via Chocolatey failed" | |
| exit 1 | |
| } | |
| # Discover rather than assume. The pin above should make the first | |
| # candidate exact; searching every fixed drive's two plausible roots | |
| # means a future runner-image or MSI change degrades into a slower | |
| # success instead of a red lane, while a genuine miss still fails. | |
| $roots = @($installDir) | |
| foreach ($fs in (Get-PSDrive -PSProvider FileSystem)) { | |
| $roots += (Join-Path $fs.Root 'SoftHSM2') | |
| $roots += (Join-Path $fs.Root 'Program Files\SoftHSM2') | |
| } | |
| $dll = $null | |
| foreach ($root in ($roots | Select-Object -Unique)) { | |
| $candidate = Join-Path $root 'lib\softhsm2-x64.dll' | |
| if (Test-Path $candidate) { $dll = $candidate; break } | |
| } | |
| # Fail loudly WITH evidence: a missing DLL fails the step rather | |
| # than quietly returning the HSM lane to a skip. | |
| if (-not $dll) { | |
| Write-Error "softhsm2-x64.dll not found after a successful install. Roots searched:`n$($roots -join "`n")" | |
| Get-ChildItem -Path $installDir -Recurse -ErrorAction SilentlyContinue | | |
| Select-Object -First 50 -ExpandProperty FullName | |
| exit 1 | |
| } | |
| $root = Split-Path -Parent (Split-Path -Parent $dll) | |
| Write-Host "SoftHSM2 resolved at $root (module: $dll)" | |
| # TestSoftHSMIntegration constructs HSMKeyStorage("softhsm") with NO | |
| # library_path override, so the module must land somewhere | |
| # PKCS11_PATHS lists. If the INSTALLDIR pin did not take effect the | |
| # install is still "successful" but the lane would fail later with | |
| # an opaque PKCS#11 error, so assert the contract here, where the | |
| # message can name the real location. | |
| if ($root -ne $installDir) { | |
| Write-Error "SoftHSM2 installed to $root, not the pinned $installDir; HSMKeyStorage's PKCS11_PATHS would not find it. The INSTALLDIR property did not take effect." | |
| exit 1 | |
| } | |
| # Publish BOTH bin\ and lib\ to the remaining steps. The MSI appends | |
| # INSTALLDIR\lib to the *machine* PATH, which a running job never | |
| # re-reads — and lib\ is the half that matters to the loader: | |
| # softhsm2-util.exe lives in bin\ but the PKCS#11 module it loads is | |
| # softhsm2-x64.dll in lib\, so publishing bin\ alone made the util | |
| # resolvable and the module not, failing token init with | |
| # "LoadLibraryA failed: 0x0000007E" (ERROR_MOD_NOT_FOUND). The test | |
| # also passes --module explicitly, so this is defence in depth rather | |
| # than the only thing holding the lane up. | |
| Add-Content -Path $env:GITHUB_PATH -Value (Join-Path $root 'bin') | |
| Add-Content -Path $env:GITHUB_PATH -Value (Join-Path $root 'lib') | |
| & (Join-Path $root 'bin\softhsm2-util.exe') --version | |
| if ($LASTEXITCODE -ne 0) { | |
| Write-Error "softhsm2-util --version failed (exit ${LASTEXITCODE})" | |
| exit 1 | |
| } | |
| - name: Install Python dependencies + package (with [dev] extras) | |
| # Use the [dev] extras so hypothesis / pytest-cov / mypy / etc. | |
| # come in via the canonical pyproject.toml declaration rather | |
| # than the legacy requirements-dev.txt. Matches ci.yml::test. | |
| # | |
| # [hsm] supplies PyKCS11: SoftHSM2 alone does not make the HSM lane | |
| # executable, because HSMKeyStorage reaches the token through that | |
| # binding. The token is now provisioned on all three OSes, so the | |
| # extra is unconditional — Windows included, where PyKCS11 builds from | |
| # sdist against the runner's MSVC + swig (ci.yml already installs | |
| # [hsm] on every Windows entry, cp310–cp314, and is green). | |
| run: pip install -e ".[dev,hsm]" | |
| shell: bash | |
| - name: Install differential/interop reference libraries (test-only) | |
| # Independent third-party implementations that let the cross-library | |
| # differential tests (tests/test_differential.py and the PyCA-interop | |
| # tests) actually EXECUTE rather than skip — previously they were dead | |
| # in every CI lane because these libs were never installed. They are | |
| # test-only references, NOT a runtime dependency (INVARIANT-1 forbids | |
| # product code importing them; the invariant lint enforces that only | |
| # ama_cryptography/*.py stays clean). Linux-gated to avoid cross-OS | |
| # wheel flakiness on the primary differential lane. | |
| if: runner.os == 'Linux' | |
| run: pip install cryptography pynacl pycryptodome | |
| # The signed artefact's native digest is specific to the exact build it | |
| # was signed against; this job builds the native library itself, so that | |
| # digest cannot match (across OS by binary format, across Linux by build | |
| # config). Verify the platform-independent .py half, then re-sign the | |
| # native half for this build — the same split ci.yml::test uses and the | |
| # same per-build re-sign setup.py's _run_integrity_signer does per wheel. | |
| - name: Verify committed .py integrity digest is current (platform-independent) | |
| shell: bash | |
| env: | |
| AMA_BUILD_PIPELINE: "1" | |
| run: | | |
| python - <<'PY' | |
| from pathlib import Path | |
| import ama_cryptography as a | |
| from ama_cryptography._self_test import _compute_module_digest | |
| committed = (Path(a.__file__).parent / "_integrity_digest.txt").read_text().strip() | |
| computed = _compute_module_digest() | |
| if committed != computed: | |
| raise SystemExit( | |
| f"FATAL: committed .py integrity digest is stale: " | |
| f"committed={committed[:16]}... computed={computed[:16]}...\n" | |
| "A .py source changed without refreshing the digest. Run:\n" | |
| " AMA_BUILD_PIPELINE=1 python -m ama_cryptography.integrity --update --sign" | |
| ) | |
| print(f"Committed .py integrity digest is current ({committed[:16]}...).") | |
| PY | |
| - name: Re-sign the integrity artefact for this build's native library | |
| shell: bash | |
| # Positive-path only; tamper detection is covered by | |
| # tests/test_native_integrity.py, so re-signing removes no coverage. | |
| env: | |
| AMA_BUILD_PIPELINE: "1" | |
| run: python -m ama_cryptography.integrity --update --sign | |
| - name: Run tests | |
| env: | |
| # Force backend availability check to fail loudly if the | |
| # native C library didn't build / didn't load — otherwise | |
| # tests silently skip and mask backend regressions. | |
| AMA_CI_REQUIRE_BACKENDS: "1" | |
| # UTF-8 encoding for Windows to support Unicode symbols in output | |
| PYTHONIOENCODING: ${{ matrix.os == 'windows-latest' && 'utf-8' || '' }} | |
| PYTHONUTF8: ${{ matrix.os == 'windows-latest' && '1' || '' }} | |
| run: pytest tests/ -v --tb=short --cov=ama_cryptography --cov-report=xml | |
| - name: Upload coverage | |
| if: matrix.os == 'ubuntu-latest' && matrix.python-version == '3.11' | |
| uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0 | |
| with: | |
| token: ${{ secrets.CODECOV_TOKEN }} | |
| files: ./coverage.xml | |
| flags: python | |
| name: python-${{ matrix.python-version }} | |
| # Linting and Code Quality | |
| lint: | |
| name: Lint and Format Check | |
| runs-on: ubuntu-latest | |
| timeout-minutes: 10 | |
| steps: | |
| - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | |
| - name: Set up Python | |
| uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 | |
| with: | |
| python-version: '3.11' | |
| - name: Install dependencies | |
| run: | | |
| # Pinned to requirements-lock.txt so this gate matches the dev | |
| # toolchain and pre-commit (no unpinned ruff, no mypy 1.x/2.x split). | |
| # types-PyYAML keeps tools/check_workflow_commands.py inside | |
| # `mypy --strict`; tests/ imports it, so mypy follows into it. | |
| # pytest is pinned here even though this job runs no tests: | |
| # `pytest.*` is under `ignore_missing_imports`, so without it | |
| # `mypy --strict tests/` types every pytest call as `Any` and its | |
| # verdict changes with whether pytest happens to be installed. | |
| # See the longer note in ci.yml::code-quality. | |
| pip install "black==26.5.1" "ruff==0.16.0" "mypy==2.3.0" \ | |
| "PyYAML==6.0.3" "types-PyYAML==6.0.12.20260724" \ | |
| "pytest==9.1.1" | |
| - name: Lint (ruff) | |
| run: ruff check . | |
| - name: Check formatting (black) | |
| run: black --check --diff . | |
| - name: Type check (mypy --strict) | |
| run: mypy --strict ama_cryptography/ tests/ | |
| continue-on-error: false | |
| # Security Audit | |
| security: | |
| name: Security Audit | |
| runs-on: ubuntu-latest | |
| timeout-minutes: 10 | |
| steps: | |
| - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | |
| - name: Set up Python | |
| uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 | |
| with: | |
| python-version: '3.11' | |
| - name: Install dependencies | |
| run: | | |
| python -m pip install --upgrade pip | |
| pip install pip-audit bandit | |
| # Use the audit-time dependency closure so pip-audit's strict | |
| # mode (below) is reproducible across runner image versions. | |
| pip install -r requirements-lock.txt | |
| # FAIL-CLOSED on the LOCK FILE (INVARIANT-2 / audit Issue 8 + | |
| # Copilot review #322): --strict gates merge on any vulnerable | |
| # package; `--requirement requirements-lock.txt` scopes the audit | |
| # to the pinned dependency closure rather than whatever the | |
| # `pip install pip-audit bandit` step transitively pulled in, | |
| # so the gate is reproducible across reruns. CVE ignores must | |
| # be declared in INVARIANT-14's table with the canonical | |
| # comment, never silently here. | |
| - name: Audit dependencies (strict, lock-file scoped) | |
| run: pip-audit --strict --desc --requirement requirements-lock.txt | |
| # FAIL-CLOSED on Bandit Medium+: any unjustified Medium/High | |
| # finding fails the build. Justified suppressions must carry an | |
| # inline `# nosec` with a tracking ID, validated by | |
| # tools/check_suppression_hygiene.py per INVARIANT-13. | |
| # | |
| # The report is deliberately unfiltered — no `--severity-level` / | |
| # `--confidence-level` — because the gate cross-checks `results` | |
| # against `metrics._totals` to prove nothing was pruned before it | |
| # looked, and applies the thresholds itself. It reads JSON rather | |
| # than grepping the text report, which prints a by-severity and a | |
| # by-confidence tally under the same labels; the previous | |
| # `grep -E '^\s*(Medium|High):\s*[1-9]'` matched the confidence | |
| # one and fired on a tree with zero Medium+ severity findings. | |
| - name: Security scan (bandit, unfiltered JSON report) | |
| run: bandit -r ama_cryptography/ -f json -o bandit-report.json --exit-zero | |
| - name: Bandit severity gate (fail on Medium+) | |
| run: python3 tools/check_bandit_severity.py bandit-report.json | |
| # Benchmark Regression Detection (Wheel Install) | |
| # Required job — fails the workflow on detected regression. | |
| # Environment variance on shared CI runners (25-50%) means baselines | |
| # must account for ctypes-fallback throughput, not Cython-optimized peaks. | |
| # | |
| # NAME DISAMBIGUATION: ci.yml::benchmark-regression also produces a | |
| # "Benchmark Regression Detection" status check via an editable install | |
| # (`pip install -e ".[dev]"`). This job is deliberately the wheel-install | |
| # variant — it builds a real wheel via PEP 517 (`python -m build --wheel`), | |
| # installs that wheel, and runs the benchmark harness from a directory | |
| # outside the source tree so the install under test cannot be shadowed | |
| # by `./ama_cryptography/`. Packaging-time regressions (missing | |
| # MANIFEST.in entries, missing package_data, broken setup.py CMakeBuild | |
| # paths, incomplete pyproject.toml [build-system].requires) only surface | |
| # in this variant — they're invisible to an editable install. | |
| # The two checks must remain independently required in branch rulesets. | |
| benchmark-regression: | |
| name: Benchmark Regression Detection (Wheel Install) | |
| runs-on: ubuntu-latest | |
| timeout-minutes: 25 | |
| needs: [c-library] | |
| steps: | |
| - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | |
| - name: Set up Python | |
| uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 | |
| with: | |
| python-version: '3.11' | |
| - name: Install system dependencies | |
| run: | | |
| sudo rm -f /etc/apt/sources.list.d/microsoft-prod.list /etc/apt/sources.list.d/azure-cli.list || true | |
| .github/scripts/apt-install.sh build-essential cmake libssl-dev | |
| - name: Build wheel via PEP 517 (isolated build env, signed integrity) | |
| # `python -m build --wheel` runs the build in an isolated env that | |
| # only sees pyproject.toml's [build-system].requires — so a missing | |
| # build-time dep (e.g., Cython, cmake, numpy) fails closed here | |
| # rather than silently falling through on a workstation that | |
| # happened to have it. setup.py's CMakeBuild compiles the native | |
| # C library inside the build, so the resulting wheel ships the | |
| # compiled libama_cryptography artifact. | |
| # | |
| # AMA_BUILD_PIPELINE=1 enables setup.py's post-build integrity | |
| # signer (CMakeBuild._run_integrity_signer) so the wheel ships an | |
| # Ed25519-signed integrity artefact, not just the digest-only | |
| # fallback. The signer uses the in-tree ama_ed25519 C kernel via | |
| # ctypes (INVARIANT-1: no PyCA dependency). Tagged releases can | |
| # additionally set AMA_INTEGRITY_TRUST_ANCHOR_PUBKEY_HEX + | |
| # AMA_INTEGRITY_REQUIRE_TRUST_ANCHOR=1 in the release workflow to | |
| # produce trust-anchored wheels — this PR-level job uses a | |
| # per-build ephemeral pubkey (still verified at import time). | |
| env: | |
| AMA_BUILD_PIPELINE: "1" | |
| run: | | |
| python -m pip install --upgrade pip | |
| pip install build | |
| python -m build --wheel --outdir dist/ | |
| ls -lh dist/ | |
| - name: Install built wheel (no source-tree shadow) | |
| # Plain `pip install dist/*.whl` — no `-e`, no fallback to source. | |
| # numpy is only needed to import the optional math_engine extension | |
| # transitively touched by some benchmarks; it's not declared in the | |
| # base wheel's runtime deps because INVARIANT-1 keeps the runtime | |
| # closure dependency-free. | |
| run: | | |
| pip install dist/*.whl | |
| pip install "numpy>=1.24.0,<3.0.0" | |
| - name: Run benchmark regression detection (from temp dir, wheel-only) | |
| # cd OUT of $GITHUB_WORKSPACE so `import ama_cryptography` cannot | |
| # resolve to ./ama_cryptography (the working-tree shadow that | |
| # would silently turn this job into ci.yml's editable variant). | |
| # The site-packages assertion makes the shadow regression noisy: | |
| # if a future commit accidentally re-introduces the editable | |
| # install, this assertion fails the job before benchmarks run. | |
| env: | |
| SRCDIR: ${{ github.workspace }} | |
| run: | | |
| mkdir -p "$RUNNER_TEMP/wheel-bench" | |
| cd "$RUNNER_TEMP/wheel-bench" | |
| python -c " | |
| import ama_cryptography | |
| loc = ama_cryptography.__file__ | |
| assert 'site-packages' in loc, ( | |
| f'wheel install not active — ama_cryptography loaded from {loc!r}; ' | |
| 'editable / source-tree shadow regression. Re-check the install step.' | |
| ) | |
| print(f'OK: ama_cryptography loaded from {loc}') | |
| " | |
| python "$SRCDIR/benchmarks/benchmark_runner.py" \ | |
| --verbose \ | |
| --baseline "$SRCDIR/benchmarks/baseline.json" \ | |
| --output "$SRCDIR/benchmarks/regression_results.json" \ | |
| --markdown "$SRCDIR/benchmarks/regression_report.md" | |
| - name: Upload regression report | |
| if: always() | |
| uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 | |
| with: | |
| name: benchmark-regression-report | |
| path: | | |
| benchmarks/regression_results.json | |
| benchmarks/regression_report.md | |
| dist/*.whl | |
| # Docker Build — gating: the smoke tests at the bottom of this job | |
| # exercise the exact path a downstream consumer hits on `docker pull`, | |
| # so a build/runtime regression here is a customer-visible regression | |
| # and must block merge. Transient Docker Hub auth/rate-limit failures | |
| # are mitigated in-job via: | |
| # * Pre-pulling the BuildKit image with 8 retries + capped exponential | |
| # backoff (~340s of tolerance) | |
| # * Optional Docker Hub login (gated on both DOCKERHUB_USERNAME and | |
| # DOCKERHUB_TOKEN being set, so a half-configured fork degrades | |
| # cleanly rather than failing noisily) | |
| # If you find yourself wanting to re-add continue-on-error here, | |
| # diagnose the underlying flake instead — silencing this gate masks | |
| # exactly the class of regression it exists to catch. | |
| docker: | |
| name: Docker Build | |
| runs-on: ubuntu-latest | |
| timeout-minutes: 20 | |
| # Job-level env so the optional-login step can gate on env.* in `if:`. | |
| # `secrets.*` is not allowed in step-level `if:` expressions, which | |
| # otherwise causes a workflow-validation error before any job runs. | |
| env: | |
| DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }} | |
| DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }} | |
| steps: | |
| - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | |
| - name: Pre-pull BuildKit image (with retries) | |
| run: | | |
| # Pre-pull the BuildKit image used by setup-buildx-action to | |
| # survive transient Docker Hub outages (504, network timeouts). | |
| # | |
| # Widened from 5 attempts / ~150s to 8 attempts / ~340s after | |
| # Docker Hub timeouts blocked PR #370 three times in one day. | |
| # The backoff is capped rather than doubling without bound: past | |
| # ~90s per gap the extra wait buys little and the job's 20-minute | |
| # timeout starts to bind. | |
| # | |
| # This is a retry, not a bypass. If the image genuinely cannot be | |
| # fetched the job still proceeds to buildx setup and fails there | |
| # on a real error. Do NOT add continue-on-error here (INVARIANT-2) | |
| # — the smoke tests below exercise the exact path a downstream | |
| # consumer hits on `docker pull`, so silencing this job hides a | |
| # customer-visible regression. | |
| IMAGE="moby/buildkit:buildx-stable-1" | |
| # Try the mirror BEFORE Docker Hub on each attempt. mirror.gcr.io is | |
| # a pull-through cache that serves library and official images | |
| # without a token, so it is not subject to Docker Hub's | |
| # unauthenticated rate limit — which is the actual failure mode this | |
| # ladder exists for. Retagging makes the pulled image satisfy | |
| # setup-buildx-action's reference either way. | |
| MIRROR="mirror.gcr.io/moby/buildkit:buildx-stable-1" | |
| BACKOFF="10 20 40 60 60 60 90" | |
| attempt=1 | |
| for delay in ${BACKOFF} final; do | |
| echo "Attempt ${attempt}/8: pulling ${IMAGE} via mirror..." | |
| if docker pull "${MIRROR}"; then | |
| docker tag "${MIRROR}" "${IMAGE}" | |
| echo "Successfully pulled ${IMAGE} from the mirror on attempt ${attempt}" | |
| exit 0 | |
| fi | |
| echo "Mirror miss; falling back to Docker Hub for attempt ${attempt}..." | |
| if docker pull "${IMAGE}"; then | |
| echo "Successfully pulled ${IMAGE} from Docker Hub on attempt ${attempt}" | |
| exit 0 | |
| fi | |
| if [ "${delay}" = "final" ]; then | |
| break | |
| fi | |
| echo "Pull failed, waiting ${delay}s before retry..." | |
| sleep "${delay}" | |
| attempt=$((attempt + 1)) | |
| done | |
| echo "::warning::All 8 pull attempts for ${IMAGE} failed over ~340s; buildx setup will retry internally" | |
| - name: Set up Docker Buildx | |
| uses: docker/setup-buildx-action@bb05f3f5519dd87d3ba754cc423b652a5edd6d2c # v4.2.0 | |
| with: | |
| # The base-image pulls (ubuntu:22.04, alpine:3.18) happen INSIDE the | |
| # BuildKit container, which does not share the runner's local image | |
| # cache — so the host-side pre-pull above cannot cover them, and | |
| # they were the part of this job still exposed to unauthenticated | |
| # Docker Hub. Docker Hub timeouts blocked PR #370 three times in one | |
| # day, all of them here. | |
| # | |
| # Routing docker.io through mirror.gcr.io takes those pulls off the | |
| # unauthenticated Docker Hub path entirely. This is a parameter of an | |
| # action already in use, not a new dependency. | |
| # | |
| # Strictly additive: BuildKit falls back to the source registry when | |
| # a mirror does not have an image or does not answer, so this cannot | |
| # make a pull fail that would otherwise have succeeded. It also does | |
| # not weaken the gate — the images are content-addressed, so a mirror | |
| # cannot serve different bytes under the same digest. | |
| buildkitd-config-inline: | | |
| [registry."docker.io"] | |
| mirrors = ["mirror.gcr.io"] | |
| - name: Log in to Docker Hub (optional, avoids rate limits) | |
| # Gate on BOTH secrets: a half-configured fork (e.g. username | |
| # set but token missing) would otherwise reach docker/login-action | |
| # and fail noisily, defeating the "optional" intent of the step. | |
| if: env.DOCKERHUB_USERNAME != '' && env.DOCKERHUB_TOKEN != '' | |
| uses: docker/login-action@371161bbe7024a29a25c5e19bfcbc0804fe9ad2c # v4.5.2 | |
| with: | |
| username: ${{ env.DOCKERHUB_USERNAME }} | |
| password: ${{ env.DOCKERHUB_TOKEN }} | |
| - name: Build Ubuntu image | |
| uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0 | |
| with: | |
| context: . | |
| file: docker/Dockerfile | |
| push: false | |
| load: true | |
| tags: ama-cryptography:ubuntu | |
| cache-from: type=gha,scope=ubuntu | |
| cache-to: ${{ github.event_name == 'push' && 'type=gha,scope=ubuntu,mode=max' || '' }} | |
| - name: Build Alpine image | |
| uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0 | |
| with: | |
| context: . | |
| file: docker/Dockerfile.alpine | |
| push: false | |
| load: true | |
| tags: ama-cryptography:alpine | |
| cache-from: type=gha,scope=alpine | |
| cache-to: ${{ github.event_name == 'push' && 'type=gha,scope=alpine,mode=max' || '' }} | |
| - name: Test Ubuntu image | |
| run: | | |
| docker run --rm ama-cryptography:ubuntu python3 -c "import ama_cryptography; print('OK')" | |
| - name: Test Alpine image | |
| run: | | |
| docker run --rm ama-cryptography:alpine python3 -c "import ama_cryptography; print('OK')" | |
| ci-gate: | |
| name: Build and Test Gate | |
| # Single aggregating status check for this workflow (see the matching gate | |
| # in ci.yml). `needs:` is workflow-local, so this gate covers only the jobs | |
| # defined in ci-build-test.yml. Branch protection should require this | |
| # context instead of the individual job names. | |
| # | |
| # This list must name EVERY other job in this workflow. A job absent from | |
| # it still runs and still reports its own red X, but branch protection | |
| # only evaluates the gate context — so an absent job cannot block a merge. | |
| # `c-library-no-native-pqc` was exactly that: it guards the | |
| # AMA_USE_NATIVE_PQC=OFF configuration, the build that commit f3dd0c2 on | |
| # this branch had to repair after it broke undetected. It ran green-or-red | |
| # into the void while the gate stayed green. tools/check_gate_coverage.py | |
| # (INVARIANT-31) now fails the PR that omits a job here. | |
| if: always() | |
| needs: | |
| - c-library | |
| - c-library-no-native-pqc | |
| - python-package | |
| - lint | |
| - security | |
| - benchmark-regression | |
| - docker | |
| runs-on: ubuntu-latest | |
| timeout-minutes: 3 | |
| steps: | |
| - name: Fail unless every required job succeeded | |
| # STRICT. Every job in this workflow runs unconditionally — none carries | |
| # a job-level `if:` — so on the authoritative head-commit run each MUST | |
| # be `success`. `skipped` now fails the gate too: on this workflow it can | |
| # only mean a `needs:` dependency failed (already red) or a future job's | |
| # `if:` drifted so it silently stopped running — exactly the gap a | |
| # lenient gate hides. `cancelled` also fails, but only ever appears on a | |
| # run superseded by a newer push or a manual abort, neither of which | |
| # gates the latest head commit. Matrix jobs are fail-fast:false and | |
| # timeouts report as `failure`. | |
| if: ${{ contains(needs.*.result, 'failure') || contains(needs.*.result, 'skipped') || contains(needs.*.result, 'cancelled') }} | |
| run: | | |
| echo "::error::Build and Test Gate failed — a required job did not succeed." | |
| echo "Dependency results: ${{ join(needs.*.result, ', ') }}" | |
| exit 1 | |
| - name: All required jobs succeeded | |
| run: echo "Build and Test Gate passed — all required jobs are green." |