feat(e2e): add single-chain swap tests (mock + live)#912
Open
feat(e2e): add single-chain swap tests (mock + live)#912
Conversation
Replace the build job in the regression workflow with a lightweight download job that fetches the latest alpha release zip from GitHub Releases, saving CI time and testing the actual shipped artifact. Also enable XP chain send tests. Made-with: Cursor
Replace `yarn build:alpha` with `yarn build:dev` in the smoke workflow so the extension is built with the dev rsbuild config. This ensures smoke tests on PRs include the latest source changes (e.g. data-testid attributes) that may not yet be in an alpha release. - Add `build:dev` script to apps/next and root package.json - Write secrets to `.env.dev` instead of `.env.production` Made-with: Cursor
The dev build uses inline-source-map which embeds full source maps into JS files, making them too large for Chrome to load in CI containers. Add NO_SOURCE_MAPS support to the dev rsbuild config (same pattern as alpha) and use build:dev:no-source-maps in the smoke workflow. Made-with: Cursor
- Add build:dev scripts to every @core/* package so the whole monorepo can build against .env.dev (service-worker, content-script, offscreen were previously forced into prod mode via build:packages). - Add root build:packages:dev and repoint build:dev to it. - Honor NO_SOURCE_MAPS in service-worker, offscreen, content-script and inpage dev rsbuild configs to keep bundles small in CI. - Switch e2e_smoke_tests.yaml to repo-level DEV_* secrets (no more environment: alpha collision with prod secrets).
Dev builds hit core-id-dev's /v2/ext/register which requires X-Api-Key (sendRequest.ts only adds the header when isDevelopment()). Without ID_SERVICE_API_KEY the server returns 401 and App Check never produces a token, cascading to notifications/gasless/profile failures. GLACIER_API_KEY is attached as rltoken to Glacier requests; without it balance/token calls hit the anonymous rate-limit pool and intermittently return empty results.
Regression tests download from s3://.../ext/alpha/ into storage-snapshots/alpha/ and smoke tests from s3://.../ext/dev/ into storage-snapshots/dev/. The snapshot loader picks the folder via the SNAPSHOT_SET env var (defaults to alpha locally).
Explicitly match tags containing `-alpha.` and exclude drafts so the regression workflow cannot pick up a future beta/rc prerelease by accident.
Adds a Playwright-driven TestRail reconciliation script and a non-blocking
GitHub Actions job that surfaces drift in the workflow run summary.
- e2e/scripts/testrail-sync.mjs: discovers tests via `playwright test --list
--reporter=json` so dynamic IDs and parameterized titles are fully resolved
before comparison; supports report-only and `--apply` modes plus optional
`--failOn{Missing,Duplicates,Unannotated}` gates and `--jsonOut` for CI.
- e2e/package.json: adds `testrail:report` and `testrail:sync` scripts.
- e2e/.env.example, e2e/README.md: document required env vars and CLI flags.
- .github/workflows/e2e_regression_tests.yaml: adds a `testrail-drift` job
that runs in parallel with the e2e suite, advisory only
(continue-on-error: true), and writes a Job Summary only when drift exists.
Addresses review feedback on PR #910: - Load `e2e/.env` via dotenv at script startup, mirroring `e2e/config/base.config.ts`. Shell env still wins (`override: false`), so CI behavior is unchanged. - Job summary now lists the actual TestRail IDs missing locally (orphaned cases), not just the count. - Top-of-file header docs include `--jsonOut` to match `--help` output. - Missing TESTRAIL_* env vars now print help and exit 2, matching the behavior for missing `--projectId` (was inconsistently exit 1).
The previous pattern (`.env`, `.env.local`) only covered two specific filenames, which made it easy to accidentally check in copies like `.env.bak`, `.env.dev`, or `.env.production`. Switch to `.env*` with an explicit allowlist for `.env.example` so any new env file variant stays out of git automatically.
Playwright runs invoked from the repo root (e.g. when scripts run `playwright test --list`) drop their output in `./test-results/`, which isn't covered by `e2e/.gitignore`. Add the root-level patterns so these folders never leak into commits.
Two small accuracy fixes to the testrail-sync output: - Summary now distinguishes between annotated specs (count of tests carrying a custom_automation_id) and unique automation IDs (size of the deduplicated set). Previously the single "with custom_automation_id" line printed the unique-ID count, which under-reported the annotated spec count whenever duplicate IDs existed. - The job-summary jq for "Title-match sync candidates" now substitutes "(empty)" for both null and empty-string fromValue. The previous expression used `// "(empty)"`, which only triggers on null/false in jq, so empty strings (the value normalizeId() returns when TestRail has no ID set) leaked through and rendered as empty backticks.
blockqa
force-pushed
the
feat/e2e-single-chain-swap-tests
branch
from
321901d to
bb60a7a
Compare
Two more correctness fixes from PR review: - fetchAllCases() now paginates correctly for both response shapes. The previous loop short-circuited on `Array.isArray(payload)` and collected only the first 250 cases when the API returned the older array shape. We still trust `_links.next` for the modern paginated object response (used by TestRail Cloud), and fall back to "keep going while pages are full" for the array shape. - parseArgs() now validates that value-taking flags (--projectId, --suiteId, --testrailField, --grep, --jsonOut) are followed by a real value. Missing or flag-shaped values now throw MissingConfigError, which the existing main().catch handler renders as a clean "Flag X requires a value" message + help + exit 2 — the same path used by other config errors.
Adds stable `data-testid` attributes to the approval flow, settings
toggles, and reusable form/input components so Playwright e2e tests
can target them without relying on text content or DOM structure:
- TransactionApprovalScreen: approval-screen
- ApprovalScreenTitle: approval-title
- TransactionBalanceChange: approval-balance-change
- CustomApprovalLimit / TokenSpendLimitCard: approval-spend-* family
- FeeSettingsSelector: settings-fee-selector + settings-fee-{value}
- SettingsHomePage: settings-quick-swaps-toggle
- MaxBuySelector: settings-max-buy
Also forwards an optional `data-testid` prop on `TxButton` (falling back
to the existing `tx-submit-button` default) and `TokenAmountInput`
(suffixed with `-amount` on the inner input) so tests can disambiguate
multiple instances on the same page. No behavior changes.
Lays down everything the swap-mock and swap-live test specs need to exist, but introduces no test cases yet (those land in the next commit). Infrastructure: - helpers/mockRpcServer.ts — local HTTP RPC mock used during swap tests to keep mock-mode swaps fully offline. - helpers/swapMocks.ts — Playwright route handlers that intercept the swap quote/transaction APIs and serve fixtures. - helpers/recordSwapFixtures.ts — utility for recording fresh quote/tx fixtures from live API responses (run manually when fixtures go stale). - pages/extension/SwapPage.ts — Page Object Model with navigation, amount entry, quote waits, approval flow handling, success/error detection, and tx hash extraction. - types/swap.ts — SwapToken / SwapPair / SwapTimeouts / SwapChain definitions plus the canonical SWAP_PAIRS table consumed by both the mock and live suites. Config: - config/base.config.ts: register the new global-teardown. - config/global-setup.ts: start the mock RPC server alongside existing bootstrap. - config/global-teardown.ts (new): shut the mock RPC server down cleanly. - fixtures/extension.fixture.ts: small refactor so context teardown cooperates with the mock RPC and swap test lifecycles. Fixtures: - fixtures/swap/*.json — 11 quote/transaction fixture pairs covering AVAX/USDC, ETH/LINK, ETH-Base/USDC, AERO-Base/ETH, BLACK/AVAX, BLACK/USDC, LINK/ETH, LINK/USDC, FARTCOIN/SOL, SOL/FARTCOIN, and USDC→SOL→FARTCOIN.
Adds two new test specs against the infrastructure introduced in the previous commit: - tests/swap-mock.spec.ts (@regression): exercises the full quote and approval flow for 11 single-chain pairs against fixture-backed mock APIs and the local mock RPC server. No on-chain transactions. - tests/swap-live.spec.ts (@swap-live): tagged separately so the suite is opt-in. Runs the same flow end-to-end against live swap APIs and signs real transactions; intended for nightly/manual runs only. Also threads support for the new pairs and CI ergonomics: - helpers/explorerApi.ts: add Base mainnet RPC entries so the explorer helper can resolve Base-Chain swap transactions during live runs. - workflows/e2e_smoke_tests.yaml: expose a `test-run-type` input so a manual dispatch can run @smoke|@regression together for full sweeps. - workflows/e2e_regression_tests.yaml: add an `include-live-swaps` workflow_dispatch toggle that opts into the @swap-live tag in addition to @regression. Defaults to off, so scheduled runs stay unchanged.
blockqa
force-pushed
the
feat/e2e-single-chain-swap-tests
branch
from
7a122cd to
f064bea
Compare
Adds a new live swap test that exercises the gasless approval path (Gas Station relay) for AVAX→USDC, with a graceful fallback to the paid-fee path when the toggle isn't surfaced. New SwapPage helpers (toggleGaslessOn/Off, isGaslessToggleVisible, getGaslessCheckbox, getFeePresetSelector, clickApprove, waitForApprovalDialogClose) mirror the SendPage shape and handle the 5–7s Gas Station finalization window so handleApprovalFlow's double-click race is avoided. Renumbers every swap test's custom_automation_id to the Send-style SWP-NNN sequence (mock = SWP-001..SWP-015, live = SWP-016..SWP-027) and appends "(mock)" / "(live)" suffixes to every leaf title so each TestRail case has a unique, runtime-agnostic identifier and the testrail-sync title-match heuristic can auto-rename the old SWP-MOCK-* / SWP-LIVE-* cases in place. Spec-local lookup tables guarded by `satisfies Record<keyof typeof SWAP_PAIRS, string>` turn a missing ID into a TypeScript error. Verified: 27/27 swap IDs resolve via `playwright test --list --reporter=json`, zero collisions across the suite (CON+NET+ONB+SND+SWP = 77 unique IDs).
…-swap-tests # Conflicts: # e2e/README.md # e2e/scripts/testrail-sync.mjs
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Pull request overview
Adds Playwright end-to-end coverage for single-chain swap flows (mocked + live) across multiple chains, along with supporting e2e infrastructure (page object, swap API mocks, mock RPC server, fixtures) and CI/TestRail workflow wiring.
Changes:
- Introduces swap test config/types, fixtures, and new mock/live swap specs.
- Adds swap-specific e2e utilities (SwapPage POM, swap API route mocks, mock JSON-RPC server, fixture recorder).
- Wires TestRail automation-ID drift reporting + workflow inputs to control which tagged suites run.
Reviewed changes
Copilot reviewed 37 out of 37 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| e2e/types/swap.ts | Defines swap tokens/pairs, timeouts, mock RPC constants, and fixture shape used by swap suites. |
| e2e/tests/swap-mock.spec.ts | Adds regression mock-mode swap flow coverage plus Quick Swaps/settings and edge-case tests. |
| e2e/tests/swap-live.spec.ts | Adds opt-in live swap flows that execute real transactions and verify on-chain via explorers. |
| e2e/pages/extension/SwapPage.ts | New Swap page-object encapsulating navigation, token selection, quoting, approvals, and toast parsing. |
| e2e/helpers/swapMocks.ts | Adds Playwright routing to serve Markr/Jupiter fixtures and proxy EVM RPC calls to the local mock RPC server. |
| e2e/helpers/mockRpcServer.ts | Implements a local HTTP JSON-RPC server for EVM chains to support fully-offline mock swap runs. |
| e2e/helpers/recordSwapFixtures.ts | Utility script to record live Markr/Jupiter responses into reusable JSON fixtures. |
| e2e/helpers/explorerApi.ts | Extends explorer verification support with Base mainnet/testnet RPC endpoints. |
| e2e/scripts/testrail-sync.mjs | Adds a TestRail ↔ Playwright automation-id drift detection/sync script based on playwright --list --reporter=json. |
| e2e/package.json | Adds testrail:report / testrail:sync scripts for the new TestRail sync utility. |
| e2e/config/global-setup.ts | Starts the mock RPC server during Playwright global setup. |
| e2e/config/global-teardown.ts | Shuts down the mock RPC server during Playwright global teardown. |
| e2e/config/base.config.ts | Registers global teardown in the base Playwright config. |
| e2e/fixtures/extension.fixture.ts | Moves screenshot-on-failure capture into the context fixture lifecycle and attaches paths for TestRail. |
| e2e/fixtures/swap/aero-eth-base.json | Adds swap fixture data for Base AERO→ETH mock flow. |
| e2e/fixtures/swap/avax-usdc.json | Adds swap fixture data for Avalanche AVAX→USDC mock flow. |
| e2e/fixtures/swap/black-avax.json | Adds swap fixture data for Avalanche BLACK→AVAX mock flow. |
| e2e/fixtures/swap/black-usdc.json | Adds swap fixture data for Avalanche BLACK→USDC mock flow. |
| e2e/fixtures/swap/eth-base-usdc.json | Adds swap fixture data for Base ETH→USDC mock flow. |
| e2e/fixtures/swap/eth-link.json | Adds swap fixture data for Ethereum ETH→LINK mock flow. |
| e2e/fixtures/swap/fartcoin-sol.json | Adds swap fixture data for Solana Fartcoin→SOL mock flow. |
| e2e/fixtures/swap/link-eth.json | Adds swap fixture data for Ethereum LINK→ETH mock flow. |
| e2e/fixtures/swap/link-usdc.json | Adds swap fixture data for Ethereum LINK→USDC mock flow. |
| e2e/fixtures/swap/sol-fartcoin.json | Adds swap fixture data for Solana SOL→Fartcoin mock flow. |
| e2e/fixtures/swap/usdc-sol-fartcoin.json | Adds swap fixture data for a USDC→SOL→Fartcoin route (currently not represented in SWAP_PAIRS). |
| e2e/README.md | Documents TestRail automation-id sync usage, flags, and CI integration. |
| e2e/.env.example | Adds required TestRail sync env vars (host/email/api key/project/suite). |
| e2e/.gitignore | Updates env-file ignore rules to ignore .env* but allow .env.example. |
| apps/next/src/pages/Settings/components/MaxBuySelector.tsx | Adds data-testid="settings-max-buy" hook for e2e. |
| apps/next/src/pages/Settings/components/Home/Home.tsx | Adds data-testid="settings-quick-swaps-toggle" hook for e2e. |
| apps/next/src/pages/Settings/components/FeeSettingsSelector.tsx | Adds data-testid hooks for fee selector and preset buttons. |
| apps/next/src/pages/Approve/components/ApprovalScreenTitle.tsx | Adds data-testid="approval-title" hook for approval overlay assertions. |
| apps/next/src/pages/Approve/components/ActionDetails/generic/TransactionBalanceChange/TransactionBalanceChange.tsx | Adds data-testid="approval-balance-change" hook for approval assertions. |
| apps/next/src/pages/Approve/components/ActionDetails/evm/EvmTokenApprovals/components/TokenSpendLimitCard.tsx | Adds spend-amount/symbol test ids for approval checks. |
| apps/next/src/pages/Approve/components/ActionDetails/evm/EvmTokenApprovals/components/CustomApprovalLimit.tsx | Adds data-testid="approval-spend-limit-section" hook. |
| apps/next/src/pages/Approve/TransactionApprovalScreen.tsx | Adds data-testid="approval-screen" hook for overlay detection. |
| apps/next/src/components/TxButton.tsx | Allows optional data-testid override (defaults to existing tx-submit-button). |
| apps/next/src/components/TokenAmountInput/TokenAmountInput.tsx | Forwards optional data-testid to container + adds -amount to inner input. |
| .github/workflows/e2e_smoke_tests.yaml | Adds test-run-type input to run smoke-only vs smoke+regression via grep pattern. |
| .github/workflows/e2e_regression_tests.yaml | Adds include-live-swaps toggle, plus new TestRail drift-report job and combined grep support. |
| .gitignore | Ignores root-level Playwright outputs when invoked from repo root. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- #1: drop orphan e2e/fixtures/swap/usdc-sol-fartcoin.json (no entry in SWAP_PAIRS). - #3: fail fast on EADDRINUSE in mockRpcServer instead of returning a dead handle that silently bypasses mocks and lets shutdown requests hit unrelated processes. - #4: rebuild handleApprovalFlow to use approvalDialog hidden as the success signal instead of the unreliable Assets-tab indicator; the 'approved' branch in the union is now actually reachable. #2 (MUI class assertion) deferred to a UI-testid PR that adds aria-pressed to FeeSettingsSelector. #5 (Markr URL coupling) deferred — investigation revealed the dev build hits markr-helium not markr, but existing fixtures don't match its schema; needs fixture refresh, tracked via inline KNOWN ISSUE comment.
- Update MARKR_BASE_URL constant to the URL the build actually hits (markr-helium); the previous `…/markr` value silently bypassed the mock layer and tests ran against the real network. - Re-record all 10 swap fixtures from real markr-helium responses via a new RECORD_FIXTURES=1 passive-observation mode that listens on browser-originated responses (route.fetch() replays are blocked by Cloudflare, so we don't intercept — we observe). - Refresh quote expiredAt to a far-future value at replay time so recorded fixtures stay valid indefinitely. - Route Solana through Markr handlers; production already routes every chain through Markr, so the Jupiter helpers were dead. Removed; restore from git history if a future deployment splits Solana traffic back to Jupiter. - Add installMarkrUrlGuard: any unmocked Markr-shaped URL throws a clear error so future env drift can't silently bypass mocks again.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds Playwright e2e coverage for single-chain swaps across Avalanche, Ethereum, Base, and Solana, plus the supporting page object, mock RPC server, fixtures, and CI wiring. Stacked on top of #911 (TestRail sync) — review that one first; once it merges, GitHub will retarget this PR to
mainautomatically.What changed
1. UI test hooks (
feat(approval): add data-testids…)data-testidattributes to the approval flow, settings toggles, and reusable form/input components (approval-screen,approval-title,approval-balance-change,approval-spend-*,settings-fee-*,settings-quick-swaps-toggle,settings-max-buy).data-testidprop onTxButton(falling back to the existingtx-submit-buttondefault) and onTokenAmountInput(suffixed with-amounton the inner input).2. Test infrastructure (
feat(e2e): add swap test infrastructure and fixtures)e2e/helpers/mockRpcServer.ts— local HTTP RPC mock used during swap tests so mock-mode runs stay fully offline.e2e/helpers/swapMocks.ts— Playwright route handlers that intercept the swap quote/transaction APIs and serve fixtures.e2e/helpers/recordSwapFixtures.ts— utility for recording fresh quote/tx fixtures from live API responses.e2e/pages/extension/SwapPage.ts— Page Object Model with navigation, amount entry, quote waits, approval handling, success/error detection, and tx hash extraction.e2e/types/swap.ts—SwapToken/SwapPair/SwapTimeouts/SwapChaindefinitions plus the canonicalSWAP_PAIRStable consumed by both suites.e2e/config/global-setup.ts/global-teardown.ts— start/stop the mock RPC server alongside existing bootstrap.e2e/fixtures/swap/*.json— 11 quote/transaction fixture pairs (AVAX/USDC, ETH/LINK, ETH-Base/USDC, AERO/ETH-Base, BLACK/AVAX, BLACK/USDC, LINK/ETH, LINK/USDC, FARTCOIN/SOL, SOL/FARTCOIN, USDC→SOL→FARTCOIN).3. Test specs + CI integration (
feat(e2e): add single-chain swap tests + CI integration)e2e/tests/swap-mock.spec.ts(@regression) — full quote + approval flow against fixture-backed mocks for all 11 pairs, no on-chain transactions.e2e/tests/swap-live.spec.ts(@swap-live) — same flow end-to-end against live APIs and signs real transactions; opt-in only.e2e/helpers/explorerApi.ts— adds Base mainnet RPCs for live-run tx resolution..github/workflows/e2e_smoke_tests.yaml— newtest-run-typeinput lets a manual dispatch combine@smoke|@regressioninto one full-sweep run..github/workflows/e2e_regression_tests.yaml— newinclude-live-swapsworkflow_dispatch toggle that opts into the@swap-livetag in addition to@regression. Defaults to off, so scheduled runs are unchanged.