Skip to content

shellcheck integration deadlocks intermittently: all goroutines in futex_wait, both ends of a pipe held by actionlint #704

Description

@intellect-energy-lab

Summary

actionlint -shellcheck=... intermittently deadlocks and never exits. The process stays
alive at 0% CPU indefinitely — it does not crash, time out, or print anything, so from
CI it looks like a slow lint rather than a hung one.

It hung twice on the same repository, blocking that repo's CI for 6h and then
4h51m (both ended only by GitHub's default 360-minute job timeout / manual cancel).
Runs of the same commit at other times pass in ~13s.

Diagnosis from the live process

Captured with /proc before killing it:

State:    S (sleeping)
Threads:  24        <- all of them parked in futex_wait_queue
VmRSS:    5188 kB
children: none      <- shellcheck had already exited

$ ls -l /proc/<pid>/fd
0  -> pipe:[744820175]
1  -> pipe:[744820176]
2  -> pipe:[744820177]
13 -> pipe:[744821390]
14 -> pipe:[744821390]     <- same pipe, BOTH ends held by actionlint
...

Every thread is in futex_wait_queue, shellcheck is gone, and fds 13/14 are the read
and write ends of the same pipe, both still open in actionlint. A read on that pipe can
never see EOF while the process itself holds the write end.

What I ruled out

I initially suspected #702 (stdin written to StdinPipe before Start, blocking once the
script exceeds pipe capacity), but the numbers do not fit:

hypothesis check result
script larger than pipe capacity largest run: block is 10,304 bytes; pipe capacity 64 KiB ruled out
pipe buffer shrunk by kernel limits 28 open pipes vs pipe-user-pages-soft = 16384 ruled out
something about the workflow content same commit lints clean locally in seconds, exit 0 ruled out
low CPU count on the runner (2 cores) 5 runs under taskset -c 0,1 locally — all pass ruled out

So it looks like a race in the concurrent shellcheck handling rather than a deterministic
size/content trigger. It reproduces only on one particular environment, and only sometimes.

Environment

  • actionlint v1.7.12 (latest at time of writing), official linux_amd64 release tarball
  • shellcheck 0.10.0
  • self-hosted GitHub Actions runner, container on Kubernetes, 2 CPU limit
  • 7 workflow files, 22 run: blocks total

Why this is worth more than "it hangs sometimes"

The failure is invisible to every layer that could stop it: the process is alive and
healthy to the runner, the container is healthy to the orchestrator, and GitHub just waits
out its default 360-minute limit. Without an explicit job timeout, a single occurrence
blocks a repository's entire CI queue for hours.

Happy to gather more (goroutine dump via SIGQUIT, strace) if it recurs — I have kept
the environment intact.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions