Skip to content

test: loosen wall-clock tolerances in download handler tests (CI flake) - #7668

Open
AronPerez wants to merge 1 commit into
mainfrom
ao/skyvern-132/download-handler-ci-tolerances
Open

test: loosen wall-clock tolerances in download handler tests (CI flake)#7668
AronPerez wants to merge 1 commit into
mainfrom
ao/skyvern-132/download-handler-ci-tolerances

Conversation

@AronPerez

Copy link
Copy Markdown
Contributor

Summary

  • loosen only the tight wall-clock upper bounds in the download handler tests
  • keep timeout inputs, lower bounds, cleanup checks, and logic assertions unchanged
  • document that the ceilings allow for scheduling jitter on shared CI runners

Context

This addresses two flakes on PRs unrelated to the test file:

  • run 29945960477: test_handle_action_timeout_bounds_browser_download_handler_drain took about 0.214s against a 0.2s ceiling
  • run 29956252756: test_handle_action_download_in_flight_request_does_not_extend_custom_timeout took about 0.181s against a 0.1s ceiling

These tests validate bounded timeout behavior and whether an in-flight request extends the configured timeout; exact sub-second wall-clock scheduling is not part of that contract. The looser upper ceilings preserve those properties while tolerating shared-runner jitter.

Validation

  • ran tests/unit/test_download_file_action_handler.py five times: 77 passed on every run
  • uv run pre-commit run --all-files

@AronPerez

Copy link
Copy Markdown
Contributor Author

Third occurrence of this flake family in ~7h, again on a PR that doesn't touch the file: run 29970786304 (PR #7679, a wording-only change) failed test_handle_action_download_without_explicit_timeout_has_bounded_in_flight_extension with assert 0.2916 < 0.15. That's three distinct tests in this file tripping wall-clock bounds on CI runners (0.2s, 0.1s, 0.15s). Each occurrence costs a sync-queue merge slot + a ~13min retrigger cycle.

@AronPerez

Copy link
Copy Markdown
Contributor Author

Fourth occurrence: run 29972647695 (PR #7679's retry) failed test_handle_action_download_in_flight_request_does_not_extend_custom_timeout again — assert 0.184 < 0.1. The family is now failing most runs, and the sync queue's oldest PR is held on it. This fix unblocks the pipeline.

@AronPerez
AronPerez force-pushed the ao/skyvern-132/download-handler-ci-tolerances branch from c9945b2 to e35f749 Compare July 24, 2026 08:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant