Skip to content

#nosec suppression is nondeterministic (identical source, same binary) #1740

Description

@crudo

#nosec suppression is nondeterministic: the same finding is suppressed on some runs and reported on others (identical source, same binary)

reproduce.sh

Summary

Running gosec repeatedly over unchanged source produces different results.
A G101 finding that is covered by a #nosec G101 directive is correctly
suppressed on most runs, but leaks through as a reported issue on a minority of
runs — no source change, no flag change, same gosec binary.

This makes gosec unusable as a hard CI gate: a job that runs gosec ./... and
fails on any HIGH finding flakes red at random.

Environment

  • gosec: v2.29.0 (also reproduces on @latest built from source)
  • Go: go1.26.8 (also seen on 1.22/1.23)
  • OS: reproduced on macOS (arm64) and Ubuntu (GitHub-hosted ubuntu-latest)

Minimal reproduction

Self-contained package (stdlib only, compiles, go vet-clean):

package repro

import (
	"fmt"
	"net/url"
	"os"
)

// #nosec G101 -- example connection strings in help text are not real credentials
var serveCmd = &struct {
	Use, Long string
	RunE      func(args []string) error
}{
	Use: "serve",
	Long: `Config file format:
  databases:
    db1:
      url: "postgres://user:pass@host:5432/db1"
    db2:
      url: "postgres://user:pass@host:5432/db2"
`,
	RunE: func(args []string) error {
		cfgPath := os.Getenv("CFG")
		raw, err := os.ReadFile(cfgPath) // #nosec G304 -- operator-supplied path
		if err != nil {
			return fmt.Errorf("read: %w", err)
		}
		_ = raw
		u, _ := url.Parse(os.Getenv("DB_URL"))
		if u != nil && u.User != nil {
			u.User = url.UserPassword(u.User.Username(), "x")
		}
		return nil
	},
}

func main() { _ = serveCmd }

Run gosec ~50–60 times and count the G101 findings each run:

for i in $(seq 1 60); do
  gosec -fmt json -out out.json ./... >/dev/null 2>&1 || true
  python3 -c "import json;print(len([x for x in json.load(open('out.json'))['Issues'] if x['rule_id']=='G101']),end=' ')"
done; echo

A ready-made script (reproduce.sh) that generates the package, runs gosec N
times, and exits non-zero when the outcome varies is attached.

Expected

Every run yields the identical result. The #nosec G101 directive covers the
declaration, so the G101 finding should be suppressed on every run (count 0).

Actual

The G101 count varies run to run over identical source, e.g.:

1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0

0 = directive honored (correct); 1 = same directive ignored, finding leaks.
Leak rate is roughly 5–15% per run and rises with more concurrent CPU load.

On the real-world codebase where we first hit this, the same file flapped
between 9 and 10 total findings across 50 CI runs — one suppressed G101
appearing/disappearing — which is what led us here.

What appears to matter (from bisecting)

The bug only surfaces with a specific combination; simplifying the file hides it:

  1. The finding is anchored to an outer node, not the trigger line. The
    #nosec G101 is on the outer var declaration, but the credential-looking
    strings are several lines deeper inside the Long raw string. gosec reports
    the G101 issue at the declaration's line range (e.g. 10-37), not at the
    string's own line. Suppression then depends on range-matching, not an exact
    line hit.
  2. There is more than one #nosec directive in the file (here G101 + G304).
    With a single directive the result is stable; adding the second directive is
    what makes the G101 suppression start flapping.
  3. It is line-position sensitive. Inserting or removing lines (even
    comments) shifts the outcome — sometimes to stable-suppressed, sometimes to
    stable-leaked, sometimes to flapping. This points at nondeterministic
    ordering/iteration when matching findings against suppression ranges, rather
    than a fixed off-by-one.

Taken together this looks like the suppression pass matching issues against
#nosec ranges using a data structure whose iteration order (or concurrent
population) is not deterministic, so which range "wins" for a given
outer-node-anchored finding varies between runs.

Impact

  • gosec cannot be used as a blocking CI check: builds fail at random.
  • Suppressions that developers added and verified locally silently stop working
    on some runs, so the reverse (a real finding randomly suppressed) is also
    plausible.

Repro assets attached

  • reproduce.sh — generates the package and runs the N-times comparison; exits
    1 when nondeterministic.

Few runs

./reproduce.sh
gosec: Version: dev Git tag:  Build date:
go:    go version go1.26.8 darwin/arm64
runs:  50

Per-run count of the G101 finding that #nosec should suppress (expect all 0):
  0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 1 0 0 0 1 0 0 1 1

NONDETERMINISTIC: outcomes across identical source = { 0 1 }
The suppressed G101 leaked on 7/50 runs.
 ./reproduce.sh
gosec: Version: dev Git tag:  Build date:
go:    go version go1.26.8 darwin/arm64
runs:  50

Per-run count of the G101 finding that #nosec should suppress (expect all 0):
  0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

NONDETERMINISTIC: outcomes across identical source = { 0 1 }
The suppressed G101 leaked on 3/50 runs.
 ./reproduce.sh
gosec: Version: dev Git tag:  Build date:
go:    go version go1.26.8 darwin/arm64
runs:  50

Per-run count of the G101 finding that #nosec should suppress (expect all 0):
  0 0 0 0 0 0 0 1 0 0 0 0 1 0 0 1 0 0 1 0 1 1 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0

NONDETERMINISTIC: outcomes across identical source = { 0 1 }
The suppressed G101 leaked on 8/50 runs.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions