Skip to content

e2e: parallelize WaitForExporters to reduce test runtime#895

Open
mangelajo wants to merge 1 commit into
mainfrom
parallelize-wait-for-exporters
Open

e2e: parallelize WaitForExporters to reduce test runtime#895
mangelajo wants to merge 1 commit into
mainfrom
parallelize-wait-for-exporters

Conversation

@mangelajo

Copy link
Copy Markdown
Member

Motivation

WaitForExporters was running sequentially despite its comment saying "in parallel." For the typical call with 3 exporters, each invocation stacked delays:

  • 2s outer exporterPostDelay sleep
  • 3× (2s inner sleep + kubectl wait + Eventually polling)
  • Minimum ~8s per call

This function is called 13+ times in e2e_test.go alone, plus additional calls in the compat test files. The sequential waiting adds up to a significant portion of the total e2e test runtime.

Change

Parallelize the WaitForExporter calls inside WaitForExporters using sync.WaitGroup:

  • Each WaitForExporter call already has its own exporterPostDelay (2s) sleep, so the duplicate outer sleep is removed
  • All exporters are waited on concurrently via goroutines
  • GinkgoRecover() ensures assertion panics from MustRunCmd/Eventually inside goroutines are properly caught by the Ginkgo test framework

Before (sequential, ~8s minimum for 3 exporters)

WaitForExporters(a, b, c)
  sleep 2s
  WaitForExporter(a)  // sleep 2s + kubectl wait + poll
  WaitForExporter(b)  // sleep 2s + kubectl wait + poll
  WaitForExporter(c)  // sleep 2s + kubectl wait + poll

After (parallel, ~4s minimum for 3 exporters)

WaitForExporters(a, b, c)
  go WaitForExporter(a)  // sleep 2s + kubectl wait + poll ─┐
  go WaitForExporter(b)  // sleep 2s + kubectl wait + poll  ├─ concurrent
  go WaitForExporter(c)  // sleep 2s + kubectl wait + poll ─┘
  wg.Wait()

Expected savings

With 3 exporters per call and 13+ call sites in the core tests: estimated 50-80s reduction in total e2e runtime.

What doesn't change

  • WaitForExporter (singular) — unchanged, keeps its own delay + kubectl wait + polling
  • All test logic — same tests, same assertions
  • WaitForExporterOffline — unchanged
  • No new imports — sync was already imported, GinkgoRecover comes from the Ginkgo v2 dot-import

WaitForExporters was running sequentially despite its comment saying
'in parallel'. For 3 exporters, each call stacked: 2s outer sleep +
3x(2s inner sleep + kubectl wait + polling) = ~8s+ minimum.

Now uses sync.WaitGroup to run all WaitForExporter calls concurrently.
Each WaitForExporter already has its own exporterPostDelay sleep, so
the duplicate outer sleep is removed. GinkgoRecover() ensures assertion
panics from goroutines are properly caught by the test framework.

With 3 exporters in parallel, each call drops from ~8s to ~4s minimum.
Called 13+ times across the core e2e tests, this saves ~50-80s total.
@coderabbitai

coderabbitai Bot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

WaitForExporters now runs exporter readiness checks in parallel, recovers goroutine failures through Ginkgo, and waits for all checks before returning.

Changes

Exporter readiness

Layer / File(s) Summary
Parallel exporter readiness checks
e2e/test/utils.go
WaitForExporters launches concurrent WaitForExporter calls with a sync.WaitGroup and GinkgoRecover(), replacing sequential waits and the previous delay.

Estimated code review effort: 2 (Simple) | ~10 minutes

Poem

I’m a bunny with exporters to check,
Now they hop in parallel—what a trek!
With a wait group in line,
All return just fine,
No sleepy delay by the deck.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly states the main change: parallelizing WaitForExporters to cut test runtime.
Description check ✅ Passed The description accurately explains the exporter wait parallelization and its runtime impact.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch parallelize-wait-for-exporters

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 golangci-lint (2.12.2)

level=error msg="[linters_context] typechecking error: pattern ./...: directory prefix . does not contain main module or its selected dependencies"


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@e2e/test/utils.go`:
- Around line 593-604: Update WaitForExporters to track failures from each
concurrent WaitForExporter call, using an atomic failure indicator alongside
GinkgoRecover. After wg.Wait() completes, abort the main goroutine when any
exporter failed so downstream commands do not run; preserve the current
concurrent waiting behavior for successful exporters.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 2f20fb71-a796-4e62-b095-5a5e51f0d1a0

📥 Commits

Reviewing files that changed from the base of the PR and between 2351e86 and cc0a283.

📒 Files selected for processing (1)
  • e2e/test/utils.go

Comment thread e2e/test/utils.go
@mangelajo
mangelajo requested review from evakhoni and raballew July 17, 2026 11:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant