Skip to content

Add listing-conformance CI (PR check + nightly sweep)#635

Open
tintinthong wants to merge 1 commit into
factory/conformance-harnessfrom
factory/conformance-ci
Open

Add listing-conformance CI (PR check + nightly sweep)#635
tintinthong wants to merge 1 commit into
factory/conformance-harnessfrom
factory/conformance-ci

Conversation

@tintinthong

Copy link
Copy Markdown
Contributor

What

Stacked on #634 (retargets to main when it merges). Adds .github/workflows/conformance.yaml:

  1. PR job — detects top-level listing dirs the PR touches, runs the static harness on each (~60-90s total: node-only, no monorepo checkout, no npm install), posts/updates one summary comment with a per-listing pass/warn/fail table + findings, uploads report artifacts, and fails only on errors (warnings are adjudicated via waiver files, never blocking here).
  2. Nightly sweep (schedule + workflow_dispatch) — runs the harness over every listing dir and maintains a single pinned Catalog conformance sweep issue: updates the table nightly, closes it when the catalog is clean. Informational only — existing listings' findings become fix-up candidates, never PR blockers.

Comment/issue rendering lives in scripts/conformance/ci/render-summary.cjs (tested locally with mocked GitHub APIs).

Verification once merged

  • Open any PR touching a listing dir → conformance check + comment appear in <2 min.
  • workflow_dispatch the sweep → baseline issue created (current state: 9/26 listings failing, dominated by CardDefs that ship no isolated/embedded/fitted templates).

🤖 Generated with Claude Code

@tintinthong tintinthong force-pushed the factory/conformance-ci branch from 7801de2 to c2dc518 Compare July 6, 2026 02:13
PR job runs the static harness on listing dirs the PR touches (~60-90s,
node-only, no monorepo checkout) and posts/updates a summary comment.
Nightly sweep runs it over every listing dir and maintains a single
'Catalog conformance sweep' issue (auto-closes when clean). Complements
ci-lint/ci-test; never replaces them.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@tintinthong tintinthong force-pushed the factory/conformance-ci branch from c2dc518 to 6178bf9 Compare July 6, 2026 08:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant