Skip to content

e2e: cache azure-skills snapshot to skip npm round-trip in global-setup #119

@ryota-murakami

Description

@ryota-murakami

Spun off from #114 Section E2 ("Global-setup robustness"). E1 (offline-skip branch) shipped on feat/e2e-phase4-negative-iron-listener via OfflineError + isOffline() + SnapshotInfo.offline; E2 stayed deferred so this issue tracks it cleanly.

Context

e2e/global-setup.ts runs installAzureSkills() once per test run, which spawns npx skills@1.5.1 add microsoft/azure-skills against the live npm registry. On a typical macOS dev box this adds ~30s of cold-cache install time per CI run, and any registry / DNS hiccup escalates straight to a failed run (E1 mitigates this by classifying "offline" and writing snapshot.offline=true, but every spec that needs azure-* still skips — so the test value is lost).

Goal

Make global-setup hermetic: an empty npm cache and a cut network must still produce a fully-populated snapshot HOME.

Proposed approaches (pick one in design)

  1. Checked-in tarball fixture — commit e2e/.snapshot/azure-skills.tar.zst (the ~/.agents/skills/ tree from a known-good installAzureSkills run, pinned to SKILLS_CLI_VERSION). Global-setup unpacks the tar instead of calling npx. Pro: zero network. Con: ~MB-scale binary in git; requires regeneration script when the CLI version bumps.
  2. CI cache key only — add an actions/cache@v4 step keyed on SKILLS_CLI_VERSION + AZURE_SKILLS_REPO that pulls/pushes ~/.npm (or the snapshot HOME directly). Pro: no binary in git. Con: still needs npm reachable on first miss; no help on dev laptops.
  3. Hybrid — checked-in tarball as the primary path, installAzureSkills only on tarball-miss + version drift (write a regeneration script triggered by pnpm gen:e2e-snapshot).

Acceptance criteria

  • pnpm test:e2e succeeds with --offline npm and a clean ~/.npm.
  • CI runtime drops by ≥20s vs. current feat/e2e-phase4-negative-iron-listener.
  • Cache invalidation is automatic when SKILLS_CLI_VERSION or AZURE_SKILL_NAMES in src/shared/constants.ts changes (no silent stale-fixture risk).
  • Plan covers regeneration UX so a CLI version bump is not a repo-foot-gun.

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requesttestingTest coverage improvement

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions