Skip to content

ci: fetch the aarch64 musl toolchain from GitHub instead of musl.cc - #1913

Merged
YamatoSecurity merged 1 commit into
mainfrom
ci/aarch64-musl-toolchain-source
Jul 29, 2026
Merged

ci: fetch the aarch64 musl toolchain from GitHub instead of musl.cc#1913
YamatoSecurity merged 1 commit into
mainfrom
ci/aarch64-musl-toolchain-source

Conversation

@Shirofune-Security

Copy link
Copy Markdown
Collaborator

The release workflow cannot complete right now. upload (ubuntu-latest, aarch64-unknown-linux-musl) dies before it reaches cargo build:

curl: (28) Failed to connect to musl.cc port 443 after 136096 ms: Couldn't connect to server
gzip: stdin: unexpected end of file
tar: Child returned status 1
##[error]Process completed with exit code 2.

musl.cc is a single volunteer-run host with no CDN or SLA, and it has stopped answering — this is not a one-off blip; the step fails the same way on re-run. Because the matrix is fail-fast, that one job also cancels the other eight targets, and upload-all-platforms / all-packages-zip are then skipped, so the whole release is blocked.

Change

Fetch the toolchain from the cross-tools/musl-cross GitHub release instead. It publishes aarch64-unknown-linux-musl.tar.xz built from the same musl-cross-make recipe, and GitHub Releases are already a hard dependency of this workflow — so this adds no new point of failure rather than trading one flaky host for another.

The tool prefix differs between the two builds (aarch64-unknown-linux-musl-gcc vs. musl.cc's aarch64-linux-musl-gcc), so CC/CXX/AR, the $GITHUB_PATH entry and the linker in .cargo/config.toml are updated to match. The reasoning in the existing comment is kept: this still has to be a real musl cross toolchain, because linking with the glibc gcc-aarch64-linux-gnu produces a binary killed at startup (#1332).

Hardened the fetch while touching it:

  • -f so an HTTP error page can never be piped in as if it were the archive (the old -sSL had no -f).
  • --retry 5 --retry-all-errors --retry-delay 5 --connect-timeout 30 to ride out a transient blip instead of failing a release on one bad request.
  • Download to a file, then extract. The old curl | tar turned a network failure into gzip: stdin: unexpected end of file, which reads like archive corruption rather than "the host is down".
  • Pinned sha256. This toolchain compiles and links a binary we publish, so it is worth verifying rather than trusting whatever an unpinned URL returns.

Also set fail-fast: false on the upload matrix. upload-all-platforms needs every target anyway, so cancelling the survivors buys nothing and costs a full re-run just to learn whether the other eight were fine.

Verification

Checked against the real artifact rather than assumed:

  • the pinned sha256 (90282c46…) matches the published aarch64-unknown-linux-musl.tar.xz.sha256;
  • extracting it produces $HOME/aarch64-unknown-linux-musl/bin/aarch64-unknown-linux-musl-{gcc,g++,ar,ranlib,strip}, exactly the paths and names the step now references;
  • the workflow YAML parses, and the matrix still lists all nine targets.

The toolchain binaries are x86_64-hosted, which is right for ubuntu-latest but means the cross-build itself can only be exercised on the runner — worth watching that job on the next release run.

The `aarch64-unknown-linux-musl` release job pulled its cross toolchain
with `curl -sSL https://musl.cc/... | tar -xz`. musl.cc is a single
volunteer-run host with no CDN, and it has stopped answering — every
release run now dies at that step with

    curl: (28) Failed to connect to musl.cc port 443 after 136096 ms
    gzip: stdin: unexpected end of file

before reaching `cargo build`, taking the whole release with it.

Switched to the cross-tools/musl-cross GitHub release, which publishes
`aarch64-unknown-linux-musl.tar.xz` from the same musl-cross-make recipe.
GitHub Releases are already a hard dependency of this workflow, so this
adds no new point of failure. The tool prefix differs between the two
builds (`aarch64-unknown-linux-musl-gcc` rather than
`aarch64-linux-musl-gcc`), so CC/CXX/AR, the `$GITHUB_PATH` entry and the
`linker` in `.cargo/config.toml` are updated to match.

Hardened the fetch while touching it: `-f` so an HTTP error page can never
be mistaken for the archive, `--retry 5 --retry-all-errors --connect-timeout
30` to ride out a transient blip, download-to-file so a failed fetch reports
as a curl error rather than a puzzling gzip one, and a pinned sha256 — this
toolchain compiles and links a published release binary, so it is worth
verifying rather than trusting an unpinned URL.

Also set `fail-fast: false` on the upload matrix. This failure cancelled the
other eight targets mid-build; since `upload-all-platforms` needs them all,
a cancelled matrix means re-running everything just to learn whether the
rest were fine.

Verified against the real artifact: the pinned sha256 matches the published
`.sha256`, and extracting it yields
`$HOME/aarch64-unknown-linux-musl/bin/aarch64-unknown-linux-musl-{gcc,g++,ar}`
exactly as the step now references. The toolchain binaries are x86_64-hosted
(correct for `ubuntu-latest`), so the cross-build itself can only be
exercised on the runner.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@YamatoSecurity
YamatoSecurity self-requested a review July 29, 2026 00:16
@YamatoSecurity
YamatoSecurity merged commit 8aa010e into main Jul 29, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants