Skip to content

test: fix flaky event handler test#609

Merged
toddbaert merged 1 commit into
mainfrom
fix-flaky-test
Jun 1, 2026
Merged

test: fix flaky event handler test#609
toddbaert merged 1 commit into
mainfrom
fix-flaky-test

Conversation

@gruebel
Copy link
Copy Markdown
Member

@gruebel gruebel commented May 29, 2026

This PR

  • now it guarantees the actual handler execuion

Signed-off-by: gruebel <anton.gruebel@gmail.com>
@gruebel gruebel requested review from a team as code owners May 29, 2026 20:05
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors a test in tests/test_client.py to use separate threading events and wrapper functions for tracking the execution of raising and recording handlers. The reviewer suggests a cleaner approach by leveraging the side_effect parameter of MagicMock to trigger the threading events directly, which eliminates the need for the verbose wrapper functions.

Comment thread tests/test_client.py
@codecov
Copy link
Copy Markdown

codecov Bot commented May 29, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.34%. Comparing base (0a96426) to head (5fa753b).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #609   +/-   ##
=======================================
  Coverage   98.34%   98.34%           
=======================================
  Files          45       45           
  Lines        2477     2483    +6     
=======================================
+ Hits         2436     2442    +6     
  Misses         41       41           
Flag Coverage Δ
unittests 98.34% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@gruebel gruebel requested a review from toddbaert May 29, 2026 20:41
@toddbaert
Copy link
Copy Markdown
Member

Thanks @gruebel

@toddbaert toddbaert merged commit b06d661 into main Jun 1, 2026
15 checks passed
gruebel added a commit to balgaly/python-sdk that referenced this pull request Jun 6, 2026
fix flaky event handler test

Signed-off-by: gruebel <anton.gruebel@gmail.com>
gruebel added a commit that referenced this pull request Jun 7, 2026
* test(e2e): add Behave step definitions for context merging

Add the missing E2E step definitions so the contextMerging.feature
scenarios from the OpenFeature spec run against python-sdk.

Fixes #500

Changes:
- Bump spec submodule to 130df3eb so contextMerging.feature is copied in
  during the `poe e2e` task.
- Add tests/features/environment.py with a before_scenario hook that
  resets provider/hook/API-context/transaction-context state, so
  scenarios cannot leak state between features.
- Add tests/features/steps/context_merging_steps.py:
  - RetrievableContextProvider captures the merged EvaluationContext
    it receives, so assertions can inspect what the SDK merged.
  - Step definitions for all scenarios in contextMerging.feature:
    single-level insert, multi-level insert, and per-key overwrite
    precedence across API / Transaction / Client / Invocation /
    Before Hooks.
  - Client-level context is set via direct attribute assignment on
    OpenFeatureClient.context (no new setter), since merging already
    honors client.context (openfeature/client.py:422-429).

Runs clean: 4 features / 50 scenarios / 233 steps.

Signed-off-by: gruebel <anton.gruebel@gmail.com>

* docs: fix inaccuracies in README code examples (#592)

Signed-off-by: Jonathan Norris <jonathan.norris@dynatrace.com>
Signed-off-by: gruebel <anton.gruebel@gmail.com>

* fix: correctly reset api state on shutdown (#589)

correctly reset api state on shutdown

Signed-off-by: gruebel <anton.gruebel@gmail.com>

* chore(main): release 0.9.0 (#555)

Signed-off-by: OpenFeature Bot <109696520+openfeaturebot@users.noreply.github.com>
Signed-off-by: gruebel <anton.gruebel@gmail.com>

* chore(deps): update pre-commit hook tox-dev/pyproject-fmt to v2.21.2 (#601)

* chore(deps): update pre-commit hook tox-dev/pyproject-fmt to v2.21.2

* upper bound toml-fmt-common till fixed

Signed-off-by: gruebel <anton.gruebel@gmail.com>

---------

Signed-off-by: gruebel <anton.gruebel@gmail.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: gruebel <anton.gruebel@gmail.com>
Signed-off-by: gruebel <anton.gruebel@gmail.com>

* chore(deps): update astral-sh/setup-uv action to v8 (#603)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Signed-off-by: gruebel <anton.gruebel@gmail.com>

* chore(deps): update codecov/codecov-action action to v6 (#604)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Signed-off-by: gruebel <anton.gruebel@gmail.com>

* chore(deps): update googleapis/release-please-action action to v5 (#605)

* chore(deps): update googleapis/release-please-action action to v5

* fix config

Signed-off-by: gruebel <anton.gruebel@gmail.com>

---------

Signed-off-by: gruebel <anton.gruebel@gmail.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: gruebel <anton.gruebel@gmail.com>
Signed-off-by: gruebel <anton.gruebel@gmail.com>

* chore(deps): update pre-commit hook pre-commit/mirrors-mypy to v2 (#606)

* chore(deps): update pre-commit hook pre-commit/mirrors-mypy to v2

* update config

Signed-off-by: gruebel <anton.gruebel@gmail.com>

---------

Signed-off-by: gruebel <anton.gruebel@gmail.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: gruebel <anton.gruebel@gmail.com>
Signed-off-by: gruebel <anton.gruebel@gmail.com>

* chore(deps): update dependency prek to >=0.4.3,<0.5.0 (#607)

* chore(deps): update dependency prek to >=0.4.3,<0.5.0

* adjust CI

Signed-off-by: gruebel <anton.gruebel@gmail.com>

---------

Signed-off-by: gruebel <anton.gruebel@gmail.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: gruebel <anton.gruebel@gmail.com>
Signed-off-by: gruebel <anton.gruebel@gmail.com>

* feat!: make set_provider non-blocking, add set_provider_and_wait (#595)

* feat!: make set_provider non-blocking, add set_provider_and_wait

Signed-off-by: Jonathan Norris <jonathan.norris@dynatrace.com>

* fix: ruff format signature collapse in api.py

Signed-off-by: Jonathan Norris <jonathan.norris@dynatrace.com>

* fix: use threading.Event in error event test to avoid flaky busy-wait

Signed-off-by: Jonathan Norris <jonathan.norris@dynatrace.com>

* fixup: pr feedback and additional checks

Signed-off-by: Todd Baert <todd.baert@dynatrace.com>

* fix: check active registration in stale-init guard, not _provider_status

Signed-off-by: Jonathan Norris <jonathan.norris@dynatrace.com>

* fixup: edge shutdown race

Signed-off-by: Todd Baert <todd.baert@dynatrace.com>

---------

Signed-off-by: Jonathan Norris <jonathan.norris@dynatrace.com>
Signed-off-by: Todd Baert <todd.baert@dynatrace.com>
Co-authored-by: Todd Baert <todd.baert@dynatrace.com>
Signed-off-by: gruebel <anton.gruebel@gmail.com>

* fix: isolate provider event handler dispatch (#599)

* Isolate provider event handlers

Signed-off-by: Lucas-FManager <265058144+Lucas-FManager@users.noreply.github.com>

* Address event handler review feedback

Signed-off-by: Lucas-FManager <265058144+Lucas-FManager@users.noreply.github.com>

* test: cover event dispatch noop path

Signed-off-by: Lucas-FManager <265058144+Lucas-FManager@users.noreply.github.com>

* fixup: drain executor at exit and relax non-blocking test timing margin

Signed-off-by: Todd Baert <todd.baert@dynatrace.com>

---------

Signed-off-by: Lucas-FManager <265058144+Lucas-FManager@users.noreply.github.com>
Signed-off-by: Todd Baert <todd.baert@dynatrace.com>
Co-authored-by: Lucas-FManager <265058144+Lucas-FManager@users.noreply.github.com>
Co-authored-by: Todd Baert <todd.baert@dynatrace.com>
Signed-off-by: gruebel <anton.gruebel@gmail.com>

* test: fix flaky event handler test (#609)

fix flaky event handler test

Signed-off-by: gruebel <anton.gruebel@gmail.com>

* chore(deps): update pre-commit hook astral-sh/ruff-pre-commit to v0.15.15 (#608)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Signed-off-by: gruebel <anton.gruebel@gmail.com>

* chore(main): release 0.10.0 (#602)

* chore(main): release 0.10.0

Signed-off-by: OpenFeature Bot <109696520+openfeaturebot@users.noreply.github.com>

* docs: clarify non-blocking set_provider behavior in changelog

Signed-off-by: Todd Baert <todd.baert@dynatrace.com>

---------

Signed-off-by: OpenFeature Bot <109696520+openfeaturebot@users.noreply.github.com>
Signed-off-by: Todd Baert <todd.baert@dynatrace.com>
Co-authored-by: Todd Baert <todd.baert@dynatrace.com>
Signed-off-by: gruebel <anton.gruebel@gmail.com>

* fix CR comments

Signed-off-by: gruebel <anton.gruebel@gmail.com>

* cleanup

Signed-off-by: gruebel <anton.gruebel@gmail.com>

---------

Signed-off-by: gruebel <anton.gruebel@gmail.com>
Signed-off-by: Jonathan Norris <jonathan.norris@dynatrace.com>
Signed-off-by: OpenFeature Bot <109696520+openfeaturebot@users.noreply.github.com>
Signed-off-by: Todd Baert <todd.baert@dynatrace.com>
Signed-off-by: Lucas-FManager <265058144+Lucas-FManager@users.noreply.github.com>
Co-authored-by: Jonathan Norris <jonathan.norris@dynatrace.com>
Co-authored-by: Anton Grübel <anton.gruebel@gmail.com>
Co-authored-by: OpenFeature Bot <109696520+openfeaturebot@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Todd Baert <todd.baert@dynatrace.com>
Co-authored-by: Nguyen Cat Luong <pkiphone.anhluong@gmail.com>
Co-authored-by: Lucas-FManager <265058144+Lucas-FManager@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants