Skip to content

Harness: Gate the Angular component meta analyzer against Compodoc - #35808

Merged
valentinpalkovic merged 25 commits into
nextfrom
valentin/angular-docgen-harness-acm
Aug 11, 2026
Merged

Harness: Gate the Angular component meta analyzer against Compodoc#35808
valentinpalkovic merged 25 commits into
nextfrom
valentin/angular-docgen-harness-acm

Conversation

@valentinpalkovic

@valentinpalkovic valentinpalkovic commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Closes #

What I did

The analyzer replaces Compodoc as Angular's docgen engine, so "it works on the fixtures I looked at" is not good enough - it has to hold everything Compodoc already recorded, on every fixture, before the swap is safe. This teaches the docgen harness to record a second set of argTypes and snippets from the analyzer next to the existing Compodoc recordings, and compare them field by field.

A run goes red when the analyzer loses ground against either its own previous recording or the legacy Compodoc one. That makes an accidental re-record loud instead of silent, which is the failure mode that matters when a snapshot suite is also the thing you use to approve changes.

The gate

Each of the 11 Angular fixtures runs both engines over the same source file and puts the analyzer's output through two comparisons before it is allowed to become the new recording.

  __testfixtures__/<case>/input.component.ts
              │
      ┌───────┴────────┐
      │                │
  Compodoc         @storybook/angular-cm
      │                │
      ▼                ▼
argtypes.snapshot   extracted argTypes
 (committed)             │
      │                  ├──── expectCurrentOrBetter ────► [lost-description]
      └──── baseline ────┤     legacyBaseline: true        [lost-default]
                         │                                 [lost-enum-member]
 acm-argtypes.snapshot   │
   (committed) ──────────┤
      baseline           ├──── expectCurrentOrBetter ────► + table.type.summary text
                         │     strictTable: true            + required true->false
                         │
                         └──── toMatchFileSnapshot ──────► byte diff vs committed

What a recording looks like

__testfixtures__/signal-io/acm-argtypes.snapshot, produced by the analyzer from an Angular signal input:

{
  "count": {
    "description": undefined,
    "name": "count",
    "table": {
      "category": "inputs",
      "defaultValue": {
        "summary": undefined,
      },
      "type": {
        "required": true,
        "summary": "number",
      },
    },
    "type": {
      "name": "number",
    },
  },

Snippets are recorded as rendered markup, so a template regression is readable rather than an object diff. __testfixtures__/signal-model/acm-snippet-TwoWayBinding.snapshot:

<sb-signal-model [value]="'hello'" [checked]="true" (valueChange)="valueChange($event)" (checkedChange)="checkedChange($event)"></sb-signal-model>

What a bad run looks like

Dropping table.type.summary from a committed acm-argtypes.snapshot trips the self-ratchet:

 FAIL  |@storybook/docgen-harness| src/angular/angular-component-meta-baselines.test.ts > angular component-meta baselines > decorator-io-basics
Error: Snapshot `angular component-meta baselines > decorator-io-basics 1` mismatched

- Expected
+ Received

@@ -26,10 +26,11 @@
        "defaultValue": {
          "summary": undefined,
        },
        "type": {
          "required": false,
+         "summary": "number",
        },
      },

Giving the legacy Compodoc recording a description the analyzer does not produce trips the parity leg, which reports every gap at once rather than the first:

 FAIL  |@storybook/docgen-harness| src/angular/angular-component-meta-baselines.test.ts > angular component-meta baselines > decorator-io-basics
Error: expectCurrentOrBetter found 1 violation(s):
- [lost-description] count: the baseline records a description but the candidate has none
 ❯ expectCurrentOrBetter src/compare/expect-current-or-better.ts:28:11

Commands

Run from the repository root.

command what it does
yarn test --project "@storybook/docgen-harness" the fixture gate, both legs, all engines
yarn workspace @storybook/docgen-harness baselines:sandbox verifies the committed sandbox baselines against a built sandbox

Re-recording is baselines:sandbox --update and vitest's -u, both deliberately explicit.

The perf engine that times the analyzer against Compodoc is not here: it is independent of these
recorders and lives in #35832, stacked on this PR.

Checklist for Contributors

Testing

The changes in this PR are covered in the following automated tests:

  • stories
  • unit tests
  • integration tests
  • end-to-end tests

Manual testing

  1. Check out this branch and run yarn.
  2. Run yarn test --project "@storybook/docgen-harness". Expect 34 files and 450 passing tests, with 25 expected failures. Those are pre-existing test.fails gap markers in angular-legacy-gaps.test.ts and vue3-legacy-gaps.test.ts that record where the legacy pipeline falls short; they are unchanged by this PR.
  3. Confirm the gate actually bites. Open code/lib/docgen-harness/src/angular/__testfixtures__/decorator-io-basics/argtypes.snapshot, put any text inside the empty "description" of the count arg, and re-run step 2. It should fail with [lost-description] count. Revert the file.
  4. Confirm a re-record is not silent. Delete a line from code/lib/docgen-harness/src/angular/__testfixtures__/decorator-io-basics/acm-argtypes.snapshot and re-run step 2. It should fail with a snapshot mismatch rather than rewriting the file. Revert it.
  5. For the sandbox baselines, build a sandbox with yarn task build --template angular-vite/docgen-server-ts --start-from auto, then run yarn workspace @storybook/docgen-harness baselines:sandbox. Expect angular-vite/docgen-server-ts: baselines match. over 37 components.

Documentation

  • Add or update documentation reflecting your changes
  • If you are deprecating/removing a feature, make sure to update
    MIGRATION.MD

code/lib/docgen-harness/README.md and src/sandbox-baselines/README.md describe the two comparison legs and the re-record commands. No user-facing documentation changes: nothing here ships.

Checklist for Maintainers

  • When this PR is ready for testing, make sure to add ci:normal, ci:merged or ci:daily GH label to it to run a specific set of sandboxes. The particular set of sandboxes can be found in code/lib/cli-storybook/src/sandbox-templates.ts

  • Declare whether manual QA will be needed for this PR during the next release, through qa:needed or qa:skip

  • Make sure this PR contains one of the labels below:

    Available labels
    • bug: Internal changes that fixes incorrect behavior.
    • maintenance: User-facing maintenance tasks.
    • dependencies: Upgrading (sometimes downgrading) dependencies.
    • build: Internal-facing build tooling & test updates. Will not show up in release changelog.
    • cleanup: Minor cleanup style change. Will not show up in release changelog.
    • documentation: Documentation only changes. Will not show up in release changelog.
    • feature request: Introducing a new feature.
    • BREAKING CHANGE: Changes that break compatibility in some way with current major version.
    • other: Changes that don't fit in the above categories.

🦋 Canary release

This PR does not have a canary release associated. You can request a canary release of this pull request by mentioning the @storybookjs/core team here.

core team members can create a canary release here or locally with gh workflow run --repo storybookjs/storybook publish.yml --field pr=<PR_NUMBER>

The analyzer has to beat Compodoc on every fixture before it can replace it, so
the harness records a second set of argTypes and snippets from
@storybook/angular-cm next to the existing Compodoc recordings and compares
them field by field. A run is red when the analyzer regresses against either
its own previous recording or the legacy one, which makes an accidental
re-record visible instead of silent.

The comparator changes underneath serve that gate: parsed snapshot elements
carry the raw text they came from, so a field the parser does not understand
yet is reported as pending rather than quietly dropped, and the shared
recording and rendering helpers are pulled out so the legacy and analyzer
recorders cannot drift apart.

The recorded sandbox baselines for angular-vite move over to the analyzer's
output. The perf suite gains an angular-component-meta engine so the two can be
timed against each other on the same project.

Docgen baseline coverage now reads the sandbox template flags directly and
throws for a template whose mainConfig is a function it cannot inspect, instead
of swallowing the failure and dropping that template out of coverage.
@coderabbitai

coderabbitai Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 33 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: f9704372-ef9a-4b1e-8e40-897050c8fd06

📥 Commits

Reviewing files that changed from the base of the PR and between 2e44895 and ccb5ab4.

⛔ Files ignored due to path filters (1)
  • yarn.lock is excluded by !**/yarn.lock, !**/*.lock
📒 Files selected for processing (23)
  • code/lib/cli-storybook/src/sandbox-templates.ts
  • code/lib/docgen-harness/package.json
  • code/lib/docgen-harness/src/angular/angular-baselines.test.ts
  • code/lib/docgen-harness/src/angular/angular-component-meta-baselines.test.ts
  • code/lib/docgen-harness/src/angular/render-helpers.ts
  • code/lib/docgen-harness/src/compare/argtypes.ts
  • code/lib/docgen-harness/src/compare/deep-equal.ts
  • code/lib/docgen-harness/src/compare/parse-element.ts
  • code/lib/docgen-harness/src/compare/parse-snapshot.ts
  • code/lib/docgen-harness/src/compare/snippets-angular.ts
  • code/lib/docgen-harness/src/compare/snippets-vue3.ts
  • code/lib/docgen-harness/src/compare/snippets.test.ts
  • code/lib/docgen-harness/src/compare/snippets.ts
  • code/lib/docgen-harness/src/compare/types.ts
  • code/lib/docgen-harness/src/sandbox-baselines/compare-baselines.test.ts
  • code/lib/docgen-harness/src/sandbox-baselines/compare-baselines.ts
  • code/lib/docgen-harness/src/sandbox-baselines/options.test.ts
  • code/lib/docgen-harness/src/sandbox-baselines/options.ts
  • code/lib/docgen-harness/src/sandbox-baselines/read-static-docgen.ts
  • code/lib/docgen-harness/src/sandbox-baselines/run.ts
  • code/lib/docgen-harness/src/sandbox-baselines/stable-stringify.test.ts
  • code/lib/docgen-harness/src/sandbox-baselines/stable-stringify.ts
  • code/lib/docgen-harness/src/vue3/vue3-component-meta-baselines.test.ts

Walkthrough

This PR strengthens docgen baseline comparison and snapshot integrity, adds shared Angular Component Meta recording, introduces Angular fixtures and baselines, and adds an Angular performance engine. It also validates sandbox template configuration and makes sandbox baseline writes atomic.

Changes

Docgen comparison and recording

Layer / File(s) Summary
Comparator and snapshot validation
code/lib/docgen-harness/src/compare/*
ArgTypes and Angular snippet comparison now detect lost defaults, summaries, requiredness, root elements, and bare attributes. Snapshot parsing and recording now verify ambiguous boundaries, pending content, and round trips.
Angular baseline recording
code/lib/docgen-harness/src/angular/*
Shared helpers attach AOT metadata and record ArgTypes and snippet snapshots with legacy parity and stale-file checks.
Angular fixtures and baselines
code/lib/docgen-harness/src/angular/__testfixtures__/*, code/lib/docgen-harness/src/sandbox-baselines/__baselines__/*
Fixtures and generated baselines cover Angular inputs, outputs, signals, inheritance, snippets, descriptions, requiredness, and metadata errors.
Baseline CLI and documentation
code/lib/docgen-harness/src/sandbox-baselines/*, code/lib/docgen-harness/README.md
Baseline comparisons classify regressions separately from neutral changes. CLI parsing is strict, and writes use staged atomic replacement.

Angular performance integration

Layer / File(s) Summary
Angular Component Meta engine
code/lib/docgen-harness/src/perf/docgen-perf/engines/angular-component-meta.ts
The new engine measures cold extraction, file mutation, invalidation, and affected-component re-extraction.
Engine registry and scenarios
code/lib/docgen-harness/src/perf/docgen-perf/*, code/lib/docgen-harness/src/perf/docgen-shared/engine-ids.ts
Angular scenarios, save counts, control ratios, registry entries, and engine identifiers now support angular-component-meta.

Sequence Diagram(s)

sequenceDiagram
  participant AngularBaselineTest
  participant AngularComponentMetaManager
  participant recordArgTypesSnapshot
  participant recordSnippets
  participant VitestSnapshots
  AngularBaselineTest->>AngularComponentMetaManager: extract fixture metadata
  AngularBaselineTest->>recordArgTypesSnapshot: record candidate ArgTypes
  recordArgTypesSnapshot->>VitestSnapshots: serialize and verify snapshot
  AngularBaselineTest->>recordSnippets: record rendered stories
  recordSnippets->>VitestSnapshots: validate and write snippet snapshots
Loading

Possibly related PRs

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch

Comment @coderabbitai help to get the list of available commands.

…docgen-harness-acm

Records the ancestry link after the base branch gained its own merge commit.
The trees were already identical, so this changes no content.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🧹 Nitpick comments (2)
code/lib/docgen-harness/src/compare/argtypes.ts (1)

58-71: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Remove the investigation provenance from the comment.

The clause "(18 of bitwarden's 54 findings)" records investigation history. It does not help future maintenance. Keep the rule and the rationale only.

The coding guidelines require comments to explain maintenance-relevant rationale, not investigation history or provenance claims.

♻️ Proposed comment edit
       // The invented-default waiver describes the LEGACY side only: a modern candidate records
       // `false`/`null` deliberately, so treating those as absent here manufactured lost-default
-      // findings for genuinely-defaulted args (18 of bitwarden's 54 findings).
+      // findings for genuinely-defaulted args.

As per coding guidelines: "Comments should explain maintenance-relevant rationale, not investigation history, internal ticket or acceptance-criteria codes, provenance claims, or cross-file line references."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@code/lib/docgen-harness/src/compare/argtypes.ts` around lines 58 - 71, In the
comment above the candidate default check in the lost-default comparison, remove
the investigation-history clause mentioning “18 of bitwarden's 54 findings.”
Keep the explanation that modern candidates deliberately record false/null and
that treating them as absent creates false lost-default findings.

Source: Coding guidelines

code/lib/docgen-harness/src/compare/pending-raw-snapshot.ts (1)

12-21: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Remove the repository provenance claim.

Keep the explanation of the separate expect state. Remove “this repo patches that package”. That detail does not define the required behavior.

Proposed change
- * `@vitest/expect` instance (this repo patches that package), the imported `expect` carries a
+ * `@vitest/expect` instance, the imported `expect` can carry a
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@code/lib/docgen-harness/src/compare/pending-raw-snapshot.ts` around lines 12
- 21, Update the comment above expectWithSnapshotState to remove the
repository-specific claim that the repo patches `@vitest/expect`, while preserving
the explanation that a separately resolved `@vitest/expect` instance can have
distinct state and that the global expect instance is authoritative.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@code/lib/docgen-harness/src/angular/__testfixtures__/properties-methods-noise/acm-argtypes.snapshot`:
- Around line 54-89: Correct the extractor’s view-child type mapping so members
such as panel retain their declared ElementRef<HTMLDivElement> type instead of
receiving { name: "other", value: "void" }; preserve the existing mapping for
nextPage and toggled. Re-record
code/lib/docgen-harness/src/angular/__testfixtures__/properties-methods-noise/acm-argtypes.snapshot
(anchor, lines 54-89) and update
code/lib/docgen-harness/src/angular/__testfixtures__/signal-io/acm-argtypes.snapshot
(sibling, lines 70-88) with the resulting snapshots.

In `@code/lib/docgen-harness/src/compare/snippets-angular.ts`:
- Around line 80-91: Update the snippet comparison flow around
parseAngularSnippet and expectCurrentOrBetter to validate candidate childContent
bindings, rejecting unsupported child bindings before toMatchFileSnapshot.
Ensure first-time recordings through recordSnippets perform the same validation
even when no committed snapshot exists, and add coverage for a candidate-only
child binding.

In `@code/lib/docgen-harness/src/perf/docgen-perf/ratios.ts`:
- Line 28: Update the Angular entry in the benchmark pairs used by ratioFor so
compodoc and angular-component-meta are not compared for warm-pass performance;
omit this pair from the warm comparability check while preserving other
framework comparisons.

In `@code/lib/docgen-harness/src/sandbox-baselines/run.ts`:
- Around line 90-101: Update write so the existing baselineDir remains
recoverable during replacement: rename it to a temporary backup before
installing stagingDir, restore the backup if staging installation fails, and
remove the backup only after successful replacement. Keep the existing staging
cleanup in the finally path and avoid any deletion window between removing the
old directory and installing the new one.

---

Nitpick comments:
In `@code/lib/docgen-harness/src/compare/argtypes.ts`:
- Around line 58-71: In the comment above the candidate default check in the
lost-default comparison, remove the investigation-history clause mentioning “18
of bitwarden's 54 findings.” Keep the explanation that modern candidates
deliberately record false/null and that treating them as absent creates false
lost-default findings.

In `@code/lib/docgen-harness/src/compare/pending-raw-snapshot.ts`:
- Around line 12-21: Update the comment above expectWithSnapshotState to remove
the repository-specific claim that the repo patches `@vitest/expect`, while
preserving the explanation that a separately resolved `@vitest/expect` instance
can have distinct state and that the global expect instance is authoritative.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: b30e4a68-a6d5-41e4-b7d6-f9ddd44c8336

📥 Commits

Reviewing files that changed from the base of the PR and between 8f5585f and 2e44895.

⛔ Files ignored due to path filters (1)
  • yarn.lock is excluded by !**/yarn.lock, !**/*.lock
📒 Files selected for processing (110)
  • code/lib/cli-storybook/src/sandbox-templates.ts
  • code/lib/docgen-harness/README.md
  • code/lib/docgen-harness/package.json
  • code/lib/docgen-harness/src/angular/__testfixtures__/complex-selector/acm-argtypes-filtered.snapshot
  • code/lib/docgen-harness/src/angular/__testfixtures__/complex-selector/acm-argtypes.snapshot
  • code/lib/docgen-harness/src/angular/__testfixtures__/complex-selector/acm-snippet-PropsAsWritten.snapshot
  • code/lib/docgen-harness/src/angular/__testfixtures__/cross-file-inheritance/acm-argtypes-filtered.snapshot
  • code/lib/docgen-harness/src/angular/__testfixtures__/cross-file-inheritance/acm-argtypes.snapshot
  • code/lib/docgen-harness/src/angular/__testfixtures__/cross-file-inheritance/acm-snippet-PropsAsWritten.snapshot
  • code/lib/docgen-harness/src/angular/__testfixtures__/decorator-generic/acm-argtypes-filtered.snapshot
  • code/lib/docgen-harness/src/angular/__testfixtures__/decorator-generic/acm-argtypes.snapshot
  • code/lib/docgen-harness/src/angular/__testfixtures__/decorator-generic/acm-snippet-PropsAsWritten.snapshot
  • code/lib/docgen-harness/src/angular/__testfixtures__/decorator-getter-setter/acm-argtypes-filtered.snapshot
  • code/lib/docgen-harness/src/angular/__testfixtures__/decorator-getter-setter/acm-argtypes.snapshot
  • code/lib/docgen-harness/src/angular/__testfixtures__/decorator-getter-setter/acm-snippet-PropsAsWritten.snapshot
  • code/lib/docgen-harness/src/angular/__testfixtures__/decorator-io-basics/acm-argtypes-filtered.snapshot
  • code/lib/docgen-harness/src/angular/__testfixtures__/decorator-io-basics/acm-argtypes.snapshot
  • code/lib/docgen-harness/src/angular/__testfixtures__/decorator-io-basics/acm-snippet-EventHandlerArg.snapshot
  • code/lib/docgen-harness/src/angular/__testfixtures__/decorator-io-basics/acm-snippet-ExplicitUndefinedArg.snapshot
  • code/lib/docgen-harness/src/angular/__testfixtures__/decorator-io-basics/acm-snippet-ObjectAndArrayArgs.snapshot
  • code/lib/docgen-harness/src/angular/__testfixtures__/decorator-io-basics/acm-snippet-PropsAsWritten.snapshot
  • code/lib/docgen-harness/src/angular/__testfixtures__/decorator-union-enum/acm-argtypes-filtered.snapshot
  • code/lib/docgen-harness/src/angular/__testfixtures__/decorator-union-enum/acm-argtypes.snapshot
  • code/lib/docgen-harness/src/angular/__testfixtures__/decorator-union-enum/acm-snippet-PropsAsWritten.snapshot
  • code/lib/docgen-harness/src/angular/__testfixtures__/expression-defaults/acm-argtypes-filtered.snapshot
  • code/lib/docgen-harness/src/angular/__testfixtures__/expression-defaults/acm-argtypes.snapshot
  • code/lib/docgen-harness/src/angular/__testfixtures__/expression-defaults/acm-snippet-PropsAsWritten.snapshot
  • code/lib/docgen-harness/src/angular/__testfixtures__/jsdoc-tags/acm-argtypes-filtered.snapshot
  • code/lib/docgen-harness/src/angular/__testfixtures__/jsdoc-tags/acm-argtypes.snapshot
  • code/lib/docgen-harness/src/angular/__testfixtures__/jsdoc-tags/acm-snippet-PropsAsWritten.snapshot
  • code/lib/docgen-harness/src/angular/__testfixtures__/properties-methods-noise/acm-argtypes-filtered.snapshot
  • code/lib/docgen-harness/src/angular/__testfixtures__/properties-methods-noise/acm-argtypes.snapshot
  • code/lib/docgen-harness/src/angular/__testfixtures__/properties-methods-noise/acm-snippet-PropsAsWritten.snapshot
  • code/lib/docgen-harness/src/angular/__testfixtures__/signal-io/acm-argtypes-filtered.snapshot
  • code/lib/docgen-harness/src/angular/__testfixtures__/signal-io/acm-argtypes.snapshot
  • code/lib/docgen-harness/src/angular/__testfixtures__/signal-io/acm-snippet-EventHandlerArg.snapshot
  • code/lib/docgen-harness/src/angular/__testfixtures__/signal-io/acm-snippet-PropsAsWritten.snapshot
  • code/lib/docgen-harness/src/angular/__testfixtures__/signal-model/acm-argtypes-filtered.snapshot
  • code/lib/docgen-harness/src/angular/__testfixtures__/signal-model/acm-argtypes.snapshot
  • code/lib/docgen-harness/src/angular/__testfixtures__/signal-model/acm-snippet-TwoWayBinding.snapshot
  • code/lib/docgen-harness/src/angular/angular-baselines.test.ts
  • code/lib/docgen-harness/src/angular/angular-component-meta-baselines.test.ts
  • code/lib/docgen-harness/src/angular/angular-provider-seam.test.ts
  • code/lib/docgen-harness/src/angular/render-helpers.ts
  • code/lib/docgen-harness/src/compare/__testfixtures__/pending-raw-snapshot-probe.snapshot
  • code/lib/docgen-harness/src/compare/argtypes.test.ts
  • code/lib/docgen-harness/src/compare/argtypes.ts
  • code/lib/docgen-harness/src/compare/expect-current-or-better.test.ts
  • code/lib/docgen-harness/src/compare/expect-current-or-better.ts
  • code/lib/docgen-harness/src/compare/parse-element.ts
  • code/lib/docgen-harness/src/compare/parse-snapshot.test.ts
  • code/lib/docgen-harness/src/compare/parse-snapshot.ts
  • code/lib/docgen-harness/src/compare/pending-raw-snapshot.test.ts
  • code/lib/docgen-harness/src/compare/pending-raw-snapshot.ts
  • code/lib/docgen-harness/src/compare/record-argtypes-snapshot.ts
  • code/lib/docgen-harness/src/compare/snippets-angular.ts
  • code/lib/docgen-harness/src/compare/snippets.test.ts
  • code/lib/docgen-harness/src/compare/snippets.ts
  • code/lib/docgen-harness/src/compare/types.ts
  • code/lib/docgen-harness/src/index.ts
  • code/lib/docgen-harness/src/perf/docgen-perf/config.test.ts
  • code/lib/docgen-harness/src/perf/docgen-perf/config.ts
  • code/lib/docgen-harness/src/perf/docgen-perf/engines/angular-component-meta.ts
  • code/lib/docgen-harness/src/perf/docgen-perf/engines/compodoc-doc.ts
  • code/lib/docgen-harness/src/perf/docgen-perf/engines/compodoc.ts
  • code/lib/docgen-harness/src/perf/docgen-perf/ratios.ts
  • code/lib/docgen-harness/src/perf/docgen-perf/registry.test.ts
  • code/lib/docgen-harness/src/perf/docgen-perf/registry.ts
  • code/lib/docgen-harness/src/perf/docgen-shared/engine-ids.ts
  • code/lib/docgen-harness/src/sandbox-baselines/README.md
  • code/lib/docgen-harness/src/sandbox-baselines/__baselines__/angular-vite-docgen-server-ts/example-button.json
  • code/lib/docgen-harness/src/sandbox-baselines/__baselines__/angular-vite-docgen-server-ts/example-header.json
  • code/lib/docgen-harness/src/sandbox-baselines/__baselines__/angular-vite-docgen-server-ts/example-page.json
  • code/lib/docgen-harness/src/sandbox-baselines/__baselines__/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-argtypes-doc-button.json
  • code/lib/docgen-harness/src/sandbox-baselines/__baselines__/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-argtypes-doc-directive.json
  • code/lib/docgen-harness/src/sandbox-baselines/__baselines__/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-argtypes-doc-injectable.json
  • code/lib/docgen-harness/src/sandbox-baselines/__baselines__/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-argtypes-doc-pipe.json
  • code/lib/docgen-harness/src/sandbox-baselines/__baselines__/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-basics-angular-forms-customcontrolvalueaccessor-custom-cva-component.json
  • code/lib/docgen-harness/src/sandbox-baselines/__baselines__/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-basics-component-with-complex-selectors-attribute-selectors-component.json
  • code/lib/docgen-harness/src/sandbox-baselines/__baselines__/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-basics-component-with-complex-selectors-class-selector-component.json
  • code/lib/docgen-harness/src/sandbox-baselines/__baselines__/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-basics-component-with-complex-selectors-multiple-class-selector-component.json
  • code/lib/docgen-harness/src/sandbox-baselines/__baselines__/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-basics-component-with-complex-selectors-multiple-selector-component.json
  • code/lib/docgen-harness/src/sandbox-baselines/__baselines__/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-basics-component-with-enums-enums-component.json
  • code/lib/docgen-harness/src/sandbox-baselines/__baselines__/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-basics-component-with-inheritance-base-button.json
  • code/lib/docgen-harness/src/sandbox-baselines/__baselines__/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-basics-component-with-inheritance-icon-button.json
  • code/lib/docgen-harness/src/sandbox-baselines/__baselines__/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-basics-component-with-ng-content-ng-content-about-parent.json
  • code/lib/docgen-harness/src/sandbox-baselines/__baselines__/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-basics-component-with-ng-on-destroy-component-with-on-destroy.json
  • code/lib/docgen-harness/src/sandbox-baselines/__baselines__/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-basics-component-with-on-push-on-push.json
  • code/lib/docgen-harness/src/sandbox-baselines/__baselines__/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-basics-component-with-pipe-custom-pipes.json
  • code/lib/docgen-harness/src/sandbox-baselines/__baselines__/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-basics-component-with-provider-di-component.json
  • code/lib/docgen-harness/src/sandbox-baselines/__baselines__/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-basics-component-with-template-template.json
  • code/lib/docgen-harness/src/sandbox-baselines/__baselines__/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-basics-component-without-selector-without-selector-ng-component-outlet.json
  • code/lib/docgen-harness/src/sandbox-baselines/__baselines__/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-basics-component-without-selector-without-selector.json
  • code/lib/docgen-harness/src/sandbox-baselines/__baselines__/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-basics-ng-module-import-module-chip.json
  • code/lib/docgen-harness/src/sandbox-baselines/__baselines__/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-basics-ng-module-import-module-for-root.json
  • code/lib/docgen-harness/src/sandbox-baselines/__baselines__/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-basics-ng-module-import-module.json
  • code/lib/docgen-harness/src/sandbox-baselines/__baselines__/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-core-decorators-componentwrapperdecorator-decorators.json
  • code/lib/docgen-harness/src/sandbox-baselines/__baselines__/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-core-decorators-theme-decorator-decorators.json
  • code/lib/docgen-harness/src/sandbox-baselines/__baselines__/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-core-modulemetadata-in-export-default.json
  • code/lib/docgen-harness/src/sandbox-baselines/__baselines__/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-core-modulemetadata-in-stories.json
  • code/lib/docgen-harness/src/sandbox-baselines/__baselines__/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-core-modulemetadata-merge-default-and-story.json
  • code/lib/docgen-harness/src/sandbox-baselines/__baselines__/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-core-styles-story-styles.json
  • code/lib/docgen-harness/src/sandbox-baselines/__baselines__/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-others-app-initializer-use-factory.json
  • code/lib/docgen-harness/src/sandbox-baselines/__baselines__/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-others-issues-12009-unknown-component.json
  • code/lib/docgen-harness/src/sandbox-baselines/compare-baselines.test.ts
  • code/lib/docgen-harness/src/sandbox-baselines/compare-baselines.ts
  • code/lib/docgen-harness/src/sandbox-baselines/run.ts
  • code/lib/docgen-harness/src/vue3/vue3-baselines.test.ts
  • code/lib/docgen-harness/src/vue3/vue3-component-meta-baselines.test.ts
  • code/lib/docgen-harness/tsconfig.json
💤 Files with no reviewable changes (15)
  • code/lib/docgen-harness/src/sandbox-baselines/baselines/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-basics-component-with-pipe-custom-pipes.json
  • code/lib/docgen-harness/src/sandbox-baselines/baselines/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-basics-component-without-selector-without-selector.json
  • code/lib/docgen-harness/src/sandbox-baselines/baselines/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-basics-component-with-template-template.json
  • code/lib/docgen-harness/src/sandbox-baselines/baselines/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-basics-component-with-ng-content-ng-content-about-parent.json
  • code/lib/docgen-harness/src/sandbox-baselines/baselines/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-core-decorators-componentwrapperdecorator-decorators.json
  • code/lib/docgen-harness/src/sandbox-baselines/baselines/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-others-app-initializer-use-factory.json
  • code/lib/docgen-harness/src/sandbox-baselines/baselines/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-basics-component-without-selector-without-selector-ng-component-outlet.json
  • code/lib/docgen-harness/src/sandbox-baselines/baselines/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-others-issues-12009-unknown-component.json
  • code/lib/docgen-harness/src/sandbox-baselines/baselines/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-basics-component-with-complex-selectors-multiple-selector-component.json
  • code/lib/docgen-harness/src/sandbox-baselines/baselines/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-basics-component-with-complex-selectors-multiple-class-selector-component.json
  • code/lib/docgen-harness/src/sandbox-baselines/baselines/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-basics-component-with-complex-selectors-attribute-selectors-component.json
  • code/lib/docgen-harness/src/sandbox-baselines/baselines/angular-vite-docgen-server-ts/example-header.json
  • code/lib/docgen-harness/src/sandbox-baselines/baselines/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-basics-component-with-ng-on-destroy-component-with-on-destroy.json
  • code/lib/docgen-harness/src/sandbox-baselines/baselines/angular-vite-docgen-server-ts/example-page.json
  • code/lib/docgen-harness/src/sandbox-baselines/baselines/angular-vite-docgen-server-ts/stories-frameworks-angular-vite-basics-component-with-complex-selectors-class-selector-component.json

Comment thread code/lib/docgen-harness/src/compare/snippets-angular.ts Outdated
Comment thread code/lib/docgen-harness/src/perf/docgen-perf/ratios.ts Outdated
Comment thread code/lib/docgen-harness/src/sandbox-baselines/run.ts
… into worktree-misty-beaming-iverson

# Conflicts:
#	code/lib/docgen-harness/src/sandbox-baselines/compare-baselines.ts
…e three gate holes

The ACM baseline recorder imported an options constant the analyzer refactor
removed, so every fixture re-recorded through the legacy extraction path:
invented NaN defaults, ES-private members, and stubbed function types. It now
makes the same call the docgen worker does.

- Angular snippet comparison rejects ungatable child bindings on the candidate
  side, and on a first-time recording where no comparison runs at all.
- The compodoc/analyzer warm member counts cover different scopes, so that pair
  no longer reports a warm comparability verdict.
- Sandbox baseline replacement moves the committed directory aside instead of
  deleting it, and puts it back if the swap fails.
Docblocks are now reserved for what the package's entry point exports, test
files carry no JSDoc at all, and the remaining comments state the why in one or
two lines instead of restating the code around them.
… into worktree-misty-beaming-iverson

# Conflicts:
#	code/lib/docgen-harness/src/sandbox-baselines/read-static-docgen.ts
The ACM recorder had hand-rolled the gate-then-snapshot flow that
recordArgTypesSnapshot already owns, including the rule that every gate must run
before the snapshot call so `-u` cannot persist a regressed recording. Its
extraGates option existed for this exact parity leg and had no callers.

- recordSnippets takes one `recorder` instead of a prefix plus a parity flag that
  always moved with it, and collects baselines rather than branching per gate.
- The sandbox baseline comparison reuses the shared has-a-default predicate and
  walks both argTypes tables once instead of twice.
- warmComparability derives from the member counts the entry actually reports.
Comment thread code/lib/docgen-harness/src/compare/parse-snapshot.ts Outdated
Comment thread code/lib/docgen-harness/src/compare/parse-snapshot.ts
valentinpalkovic and others added 4 commits August 10, 2026 14:28
Angular needed two violations the represented-names contract could not carry -
the root element identity, because there the root IS the component selector, and
bare attribute survival - so it got an early return and re-implemented the shared
scaffolding around them: the unparsable-baseline throw, the unparsable-candidate
violation, and the lost-representation diff, message strings included.

Frameworks now supply a grammar (parse, represented names, and optional gatability
and structural hooks) and compareSnippet owns the pipeline. Angular contributes
only what is Angular, and the never-exhaustiveness check covers it again: adding a
framework to the union now fails compilation for every framework rather than all
but one.
The sandbox baseline gate had its own loss detection, so "an arg disappeared" and
"an arg stopped recording a default" were defined twice. The two had already
drifted: compareArgTypes waives ES-private `#member`s, which the local copy did
not, so an inaccessible member disappearing failed the sandbox gate as a
regression while the fixture gate waived it.

Losses now come from compareArgTypes filtered to the kinds this gate treats as
blocking. Only the wording and the severity split stay local, and the readable
per-arg summary - the thing the local copy was really added for - is unchanged.
@valentinpalkovic valentinpalkovic self-assigned this Aug 10, 2026
@valentinpalkovic valentinpalkovic added angular feature request ci:normal Run our default set of CI jobs (choose this for most PRs). labels Aug 10, 2026
The sandbox baseline gate compared values by serializing both sides with
stableStringify, while the argTypes comparator already had a structural
deepEqual. Same question, and the serializing one was the weaker answer: it
allocated a full sorted clone plus a pretty-printed string per field per
component only to throw both away, and it collapsed distinctions a byte-exact
gate should keep.

deepEqual moves to its own module and both paths use it. stableStringify follows
the write path it actually serves, where sorted pretty JSON is the point.

Key-order insensitivity was the property the serializing form provided and
nothing pinned it, so a test now does - baselines are written key-sorted while a
build emits its own order, so losing it would report drift on every verify run.

Also names the collision between the gate's UNRESOLVED_STUBS and the perf
engine's RESOLVED_TYPES, where `undefined` means opposite things, and renames the
sandbox recorder's readCommitted, which read a directory of baselines rather than
the snapshot text the identically-named Angular helper reads.
`--template=` parsed as an empty string, which is falsy, so it fell through to
"no template given" and quietly widened the run to every server-docgen template -
the exact outcome the comment above the parser claimed was impossible. Node's
parseArgs only rejects `--template` with the value omitted, not with an empty one.

Parsing moves to its own module and goes through the package's
parseHarnessOptions, so an empty value is a schema failure with the same
`invalid options:` shape the perf harnesses produce. Being importable is what lets
it be tested at all; run.ts calls main() at module scope.

Also asserts stableStringify against a literal instead of against another call of
itself, and moves that test to the module the function now lives in.
The perf engine for the Angular analyzer, its control-pair wiring, and the
supporting utility changes are independent of the baseline recorders this PR is
about, and they read as a separate review. They move to their own PR stacked on
this one.

Nothing here depended on them: the perf tree never referenced the compare or
sandbox-baselines layers, and the two directions the rest of the harness does
reach into it - SANDBOX_DIRECTORY and parseHarnessOptions - are untouched files.
@valentinpalkovic
valentinpalkovic marked this pull request as ready for review August 10, 2026 13:15
@valentinpalkovic valentinpalkovic added the qa:skip Pull Requests that do not need any QA. label Aug 10, 2026
@huang-julien
huang-julien self-requested a review August 10, 2026 20:27
Base automatically changed from valentin/angular-component-meta to next August 11, 2026 14:30
@valentinpalkovic
valentinpalkovic merged commit 8bcc7ae into next Aug 11, 2026
143 checks passed
@valentinpalkovic
valentinpalkovic deleted the valentin/angular-docgen-harness-acm branch August 11, 2026 16:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

angular ci:normal Run our default set of CI jobs (choose this for most PRs). feature request qa:skip Pull Requests that do not need any QA.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants