Skip to content

Fix: Display actual test suite status instead of hardcoded 'PASSED'#5302

Open
SpacePlushy wants to merge 2 commits intoplatformio:developfrom
SpacePlushy:fix/test-status-display
Open

Fix: Display actual test suite status instead of hardcoded 'PASSED'#5302
SpacePlushy wants to merge 2 commits intoplatformio:developfrom
SpacePlushy:fix/test-status-display

Conversation

@SpacePlushy
Copy link
Copy Markdown

Summary

Fixes #5183

The print_suite_footer function was hardcoding 'PASSED' for any non-error test suite status, causing SKIPPED test suites to incorrectly display as PASSED in the test footer.

Changes

Modified platformio/test/cli.py to use the actual test_suite.status.name with appropriate color coding via status.to_ansi_color().

Before:

click.style(test_suite.status.name, fg="red", bold=True)
if is_error
else click.style("PASSED", fg="green", bold=True)  # ← Hardcoded!

After:

status_color = "red" if is_error else test_suite.status.to_ansi_color()
click.style(test_suite.status.name, fg=status_color, bold=True)

Testing

Verified the fix correctly displays all status types:

  • ✅ PASSED (green) - when all tests pass
  • ✅ FAILED (red) - when tests fail
  • ✅ ERRORED (red) - when tests error
  • ✅ SKIPPED (yellow) - when tests are skipped

Reproduction

Created test suites with different statuses and verified footer output matches actual test execution results.

Before fix: Empty test suite showed [PASSED] in footer
After fix: Empty test suite correctly shows [SKIPPED] in footer

Fixes platformio#5237

When using '-U MACRO' (with space) in build_flags, SCons' ParseFlags()
splits it into two separate items in CCFLAGS: ['-U', 'MACRO']. The
previous code only captured '-U' and attempted undef[2:] which resulted
in an empty string.

This fix iterates through CCFLAGS with an index to detect standalone
'-U' flags and pairs them with the following macro name. Both formats
are now supported:
- '-UMACRO' (no space) - already working
- '-U MACRO' (with space) - now fixed

The solution is similar to how SCons handles '-D' flags with spaces.
Fixes platformio#5183

The print_suite_footer function was hardcoding 'PASSED' for any
non-error test suite status. This caused SKIPPED test suites to
incorrectly display as PASSED.

Changed to use the actual test_suite.status.name with appropriate
color coding via status.to_ansi_color(). Now correctly displays:
- PASSED (green) when tests pass
- FAILED (red) when tests fail
- ERRORED (red) when tests error
- SKIPPED (yellow) when tests are skipped

This ensures the footer status matches the actual test execution result.
@ivankravets
Copy link
Copy Markdown
Member

Could you provide a simple project to reproduce this issue?

Copy link
Copy Markdown

@mahdirajaee mahdirajaee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good fix. The original code had a clear bug: the else branch hardcoded click.style("PASSED", fg="green") regardless of the actual test_suite.status, which meant statuses like SKIPPED, WARNED, or any non-error state all displayed as "PASSED". The new code correctly uses test_suite.status.name for the label and delegates to test_suite.status.to_ansi_color() for the color, which properly handles all TestStatus enum variants.

The one thing I'd verify is the behavior of to_ansi_color() for every possible TestStatus value — if any status returns None or an unexpected value, click.style(fg=...) might raise or silently produce uncolored output. A quick check that TestStatus.SKIPPED.to_ansi_color() returns a valid click color string would be good to confirm. The is_error check is still used correctly for the bar styling via the separate is_error=is_error kwarg, so that concern is cleanly separated from the status label. Clean, minimal change that addresses the reported issue.

Note: the diff also includes unrelated changes to piobuild.py (handling -U MACRO flag formats) that seem to belong to a different PR or concern — might want to split those out to keep the commit focused.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unit test runner shows PASSED in summary when there are failing tests

3 participants