Skip to content

Checkup Step 7 gate blocks targeted PRs on out-of-scope/non-blocking criticals when no free-text reason is present #1574

Description

@DianaTao

Summary

In targeted PR mode, pdd checkup's Step 7 gate (_step7_passed_step7_unfixed_critical_blocks_targeted_pr in pdd/agentic_checkup_orchestrator.py) fails a PR on an unfixed critical finding that the verifier itself marked out-of-scope / non-blocking, when the finding does not also carry a free-text *_reason field. The exclusion is coupled to _step7_nonblocking_reason(issue) being non-empty, but the Step 7 verify prompt emits blocking/scope flags without a separate non_blocking_reason / out_of_scope_reason / scope_reason / reason field — so the and reason guard defeats the exclusion.

Impact

A PR that only changes README.md is failed by a pre-existing, repo-wide TypeScript build break (TS18003: frontend/src/ missing) that the PR neither introduced nor can fix. Step 7's own verdict says success: true, issue_aligned: true, "PR-scoped verification: all clear", and tags the finding scope: out-of-scope, blocking: false — yet the gate returns Step 7 reported unfixed critical issues, skips the push, and the Final Gate reports layer1_status: failed. Observed on a real staging checkup run (promptdriven/test_repo_2 #510).

Reproduction (against current main)

from pdd.agentic_checkup_orchestrator import _step7_passed
issue = {"module":"frontend","severity":"critical","fixed":False,
         "scope":"out-of-scope","blocking":False,
         "description":"TS18003: frontend/src/ missing (pre-existing)."}
payload = {"success":True,"issue_aligned":True,"issues":[issue],"changed_files":["README.md"]}
print(_step7_passed("report\n"+__import__("json").dumps(payload),
                    pr_mode=True, has_issue=True, pr_test_scope="targeted"))
# -> (False, 'Step 7 reported unfixed critical issues: frontend: ...')   # BUG: should pass

_step7_nonblocking_reason(issue) returns '', so _step7_unfixed_critical_blocks_targeted_pr returns True despite blocking is False and scope == "out-of-scope".

Root cause

_step7_unfixed_critical_blocks_targeted_pr requires ... and reason on each exclusion branch:

if blocking is False and reason:        return False
if in_scope is False and reason:        return False
if scope in explicit_nonblocking_scopes and reason:  return False

The explicit blocking: false / scope: out-of-scope flags are themselves the authoritative non-blocking signal; requiring an additional free-text reason is too strict and mismatches the prompt's output contract.

Proposed fix

Treat an explicit non-blocking flag/scope as sufficient on its own (keep the structured reason as an additional signal, not a precondition). In targeted PR mode the full-suite truth comes from the GitHub-checks gate, so out-of-scope criticals are informational by design.

Acceptance

  • The reproduction above returns (True, "").
  • A genuinely in-scope/blocking unfixed critical still fails the gate.
  • Regression test added.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

Status
Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions