Skip to content

Commit 6efe544

Browse files
authored
fix: unify divergence screen capture with snapshot's full-window scope (#1265)
* fix: unify divergence screen capture with snapshot's full-window scope Route captureDivergenceObservation through captureSnapshotData — the same function the snapshot command itself builds its capture with (Android's snapshot-helper full-window route with its graceful app-scoped fallback, iOS's bounded system-modal probe path, macOS/Linux surface-scoped branches) — instead of a parallel hand-rolled dispatchCommand call. The chrome filter and meaningful-target filter stay layered on top as filters over that full capture, never as a scoping. Amends ADR-0012 decision 4 to state the invariant: an agent must never see a healthier `screen` in a divergence report than a plain `snapshot` would show it, so a separate-window system overlay (volume dialog, quick-settings shade, permission dialog) must survive into `screen.refs` exactly as `snapshot` would present it. Also fixes the synthetic `volume_dialog_slider` id in snapshot-chrome-android-statusbar.test.ts to the real, live-verified `volume_new_ringer_active_icon_container` id and rewords the test comment to read as a filter-logic unit test rather than a live-capture-path claim, and adds unit coverage for the invariant itself. Fixes #1264 * fix: rank divergence screen.refs within the cap so overlays are not buried The #1264 root cause is cap burial, not capture scope: buildReplayDivergenceScreenRefs sliced candidates in document order, so a fully-captured separate-window overlay (volume dialog, QS shade, permission dialog) that enumerates after the app window's ~77 nodes lands past position 20 and is truncated away — the report shows a healthy-looking app under a covering overlay it cannot see (archived evidence: screen.truncated: true, zero volume refs). - Rank within the cap instead of document-order slicing: foreign-window (non-app-bundleId) hittable nodes — the dismiss targets for whatever covers the app — are promoted ahead of app content, otherwise stable (document order preserved within each tier; equal-priority app nodes never reshuffled). The 20-cap is a byte bound, not a first-20-in-tree-order policy. - Occlusion fallback: when a system overlay mass-covers the app (every app node annotated interactionBlocked: 'covered'), surface those covered nodes rather than emitting an empty screen.refs — a report whose capture holds meaningful nodes but whose refs is empty is broken by construction. - repairHint/suggestions consume the full captured node list, not the capped refs slice, so hint routing is unaffected; only screen.refs selection changes. Detection keys off node.bundleId (Android-only, from the a11y package); iOS/macOS leave per-node bundleId undefined, so ranking degrades to document order there (safe — those platforms surface modals via the probe path, not by cap-competing). Guarded on a known appBundleId so a sessionless capture never reorders. Tests: replaces the small-fixture #1264 test (which the overlay fit inside the cap regardless of order, so it did not prove the invariant) with a realistic full fixture (24 app controls + overlay dismiss-target captured LAST) that fails on document-order slicing and passes with ranking; plus occlusion tests (mass-covered app -> overlay surfaces, refs non-empty; bare-scrim fallback -> covered app nodes surfaced, refs non-empty). Both were verified to fail before the fix. ADR-0012 decision 4 amendment reworded to cover ref-selection ranking and the occlusion guarantee, not only capture scope. Refs #1264 * fix: route divergence capture through captureSnapshot wrapper + clean flags policy Completes the #1264 capture unification. The prior round routed captureDivergenceObservation through captureSnapshotData (the inner single-shot capture), but plain `snapshot`'s backend calls the HIGHER captureSnapshot wrapper, which owns Android freshness + post-action retry. A divergence could therefore consume the first stale/app-scoped dump while a plain `snapshot` retries to the fresh full-window tree — a divergence staler/narrower than `snapshot`, violating the invariant. - Route the divergence capture through the same `captureSnapshot` wrapper as plain snapshot, so it inherits freshness/post-action retry parity. No fork: the wrapper's params (device, session, flags, logPath) are all suppliable from the divergence path. - Build the divergence capture's flags from a clean, fixed policy (`divergenceCaptureFlags`: full-window, non-raw, default depth) instead of spreading the failed action's flags — so a failed `snapshot --raw`/scoped/`-d` action can no longer narrow the diagnostic tree. Only the interactive-only policy is carried (extracted as a helper so captureDivergenceObservation stays within complexity budget). Tests: a freshness-retry regression (session carries an active Android freshness marker; capture-1 is a stale near-empty dump that trips sharp-drop, capture-2 holds the overlay — asserts the divergence uses the retried fresh tree and dispatched twice), and a clean-flags regression (a failed raw/scoped/depth action — asserts the snapshot dispatch context drops snapshotRaw/scope/depth while still applying interactive-only). Both verified to fail on the pre-fix code. ADR-0012 decision 4 amendment updated to state the same-wrapper (freshness parity) and clean-flags guarantees. Live overlay acceptance remains a maintainer device step (env down): unit fixtures prove ref SELECTION after nodes are supplied, not that the Android helper returns the separate-window overlay at divergence time. Refs #1264 * test: stub the freshness-retry sleep so the capture-parity test doesn't real-wait The #1264 capture-parity regression exercised the real Android sharp-drop retry, which awaited the real ~250 ms `sleep` delay — repo guidance forbids real-time waits in unit tests. Mock `sleep` (the delay the retry path in snapshot-capture.ts awaits) to a no-op at the module level, so the retry BRANCH still executes (loop runs, retries, re-captures) without a wall-clock wait. The test still proves the branch: two on-device dispatches and use of the retried fresh tree (overlay present). Verified it still fails on the pre-fix single-shot path (1 dispatch) with the delay stubbed, so the stub does not make it vacuous. No production change; the delay stub needs no DI seam since `sleep` is a plain module export. Refs #1264 * fix: reconcile divergence ref selection with #1257 ADR-0014 partial ref frame Rebase reconciliation. #1257 (ADR-0014 session ref-frame lifetime) landed on main and changed captureDivergenceObservation to activate a PARTIAL ref frame (markSessionPartialRefsIssued) authorizing exactly the divergence screen's emitted refs — computing that "digestBodies" set with its own document-order, non-covered-only filter. My #1264 change made buildReplayDivergenceScreenRefs emit a DIFFERENT set (ranked, occlusion-fallback, meaningful-filtered), so the authorized frame would no longer match the shown screen: in the mass-covered fallback the screen surfaces covered refs that #1257's non-covered-only frame filter excluded, leaving the agent a ref the screen advertised but the frame rejects. Extract selectDivergenceScreenRefNodes as the single source of truth for which nodes screen.refs publishes and in what order. Both the rendered digest (buildReplayDivergenceScreenRefs) and the partial-frame authorization (captureDivergenceObservation -> markSessionPartialRefsIssued) derive from it, so the frame authorizes exactly the emitted set — preserving BOTH #1257's ADR-0014 intent and #1264's ranking/occlusion intent. Also refresh the captureDivergenceObservation doc to the partial-frame sequence. Test: assert the partial ref frame scope (session.refFrameScope) equals the emitted screen.refs set in the mass-covered fallback (covered refs included) — verified to fail on #1257's original non-covered-only digestBodies. Refs #1264 #1257
1 parent 54977f3 commit 6efe544

4 files changed

Lines changed: 783 additions & 61 deletions

File tree

docs/adr/0012-interactive-replay.md

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,13 @@ Accepted (2026-07-10); partially implemented (last updated 2026-07-13). See [Mig
1212
`REPLAY_DIVERGENCE` repair-loop fix (#1223).
1313
- Decision 6, the base agent-supervised re-record repair — `replay --save-script` arming, the
1414
post-watermark healed slice, `repairHint`, and the writer's bare-`@ref` fail-loud guard (#1228).
15+
- Decision 4 amendment, `screen`'s capture scope and ref selection — the divergence `screen` capture runs
16+
through the same `captureSnapshot` wrapper as plain `snapshot` (full-window scope + Android freshness /
17+
post-action retry parity) under a clean, fixed capture-flags policy (a failed raw/scoped/`-d` action can
18+
no longer narrow the diagnostic tree), with the chrome and meaningful-target filters layered on top as
19+
filters, not scopings; and `screen.refs` is ranked within the byte cap (foreign-window dismiss targets
20+
ahead of app content; mass-covered app nodes surfaced rather than emptied) instead of sliced in document
21+
order, so a captured overlay is never buried past the cap (#1264).
1522

1623
**Accepted but NOT yet implemented** (this amendment; tracked by #1235 — repair-transaction lifecycle):
1724
the R7 repair-transaction keep-alive and its distinct `resume.repairSessionHeld` signal, the ARMED →
@@ -438,6 +445,42 @@ healthy snapshot digest and the only form that issues actionable refs. `{ state:
438445
hint }` is returned when capture fails or is sparse; it has no refs or generation and must not fall back to
439446
the old session tree. Screen-capture failure never replaces or masks the original replay cause.
440447

448+
> **Amendment (#1264): `screen`'s capture scope and ref selection.** `refs` is a filtered, ranked digest
449+
> of the exact same tree a plain `snapshot` would return at that moment. Two guarantees back the invariant:
450+
>
451+
> 1. **Capture scope.** The capture underneath `screen` runs through the **same `captureSnapshot` wrapper**
452+
> the plain `snapshot` command's backend calls — not a parallel single-shot dispatch — so it inherits
453+
> `snapshot`'s full-window scope (Android: the snapshot-helper route, with the existing graceful
454+
> app-scoped fallback only when the helper is unavailable; iOS: the bounded system-modal probe path;
455+
> macOS/Linux: their surface-scoped branches) AND its Android freshness / post-action retry policy. A
456+
> divergence must not consume a first stale or app-scoped dump while a plain `snapshot` retries to the
457+
> fresh full-window tree — that would make the divergence staler or narrower than `snapshot`, violating
458+
> the invariant. The divergence capture's flags are a **clean, fixed policy** (full-window, non-raw,
459+
> default depth), NOT the failed action's flags: a failed `snapshot --raw`/scoped/`-d` action must never
460+
> narrow or reshape the diagnostic tree below what a plain `snapshot` shows, so `snapshotRaw` /
461+
> `snapshotScope` / `snapshotDepth` are dropped; only the interactive-only policy (full for non-rect
462+
> `get`/`is`/`wait` reads, interactive otherwise) is carried. The chrome filter (#1233/#1256,
463+
> `collectSettleChromeRefs`) and the meaningful-target filter (label/id or `hittable`) are layered ON TOP
464+
> of that full capture as **filters**, not as a separate, narrower scoping — a filter may drop a node the
465+
> full capture contains, but the capture itself must never omit content `snapshot` would show.
466+
> 2. **Ref selection within the cap.** The `screen.refs` cap is a **byte bound**, not a "first N in document
467+
> order" policy. A separate-window overlay enumerates AFTER the app window's nodes, so a document-order
468+
> slice truncates a fully-captured overlay away (its dismiss target sits past the cap) — reporting a
469+
> healthy-looking app even though the capture holds the covering window. So `refs` is RANKED before it is
470+
> capped: foreign-window (non-app-`bundleId`) hittable nodes — the dismiss targets for whatever covers
471+
> the app — are promoted ahead of app content, with document order otherwise preserved (stable within
472+
> each tier). And when a system overlay MASS-COVERS the app so every app node is annotated `covered`,
473+
> those covered nodes are surfaced rather than emitting an empty `refs`: a report whose capture holds
474+
> meaningful nodes but whose `refs` is empty is broken by construction.
475+
>
476+
> This is a hard invariant: **an agent must never see a healthier `screen` in a divergence report than a
477+
> plain `snapshot` would show it.** Concretely, a separate-window system overlay covering the app at the
478+
> moment of capture — a held volume dialog, a persistent quick-settings shade, a permission dialog — must
479+
> appear in `screen.refs` (its actionable/hittable/labeled nodes surviving the filters AND the cap) exactly
480+
> as `snapshot` would present it. `repairHint` (decision 6) and `suggestions` (decision 1) consume the FULL
481+
> captured node list, not the capped `refs` slice, so hint routing is computed over the same full,
482+
> correctly-scoped capture and is never routed as if the app underneath a covering overlay were healthy.
483+
441484
Response levels bound the entire serialized UTF-8 `details.divergence` object, not merely its arrays:
442485
compact (`--level digest`) is at most **8 KiB**, default at most **24 KiB**, and full at most **64 KiB**.
443486
Compact carries at most **8** screen refs and no suggestion entries — it carries `suggestionCount` (the

src/core/__tests__/snapshot-chrome-android-statusbar.test.ts

Lines changed: 17 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -104,14 +104,16 @@ test('Android non-raw capture: status-bar leaves are recognized as chrome once t
104104
}
105105
});
106106

107-
test('Android actionable systemui overlay (volume dialog) still survives with the status-bar leak fix (#1251)', () => {
108-
// Extend the RAW capture with a disjoint systemui run whose leaf ids look
109-
// like an actionable overlay (volume dialog): no real capture of that
110-
// surface was available, but the point of this test is exactly that the
111-
// new leaf-id set must NOT broaden to "any systemui id". Appended to the
112-
// RAW array (not the already-walked output) so the whole tree — status bar
113-
// AND volume dialog — goes through the real walk together, exactly as a
114-
// single device capture would.
107+
test('Android actionable systemui overlay (volume dialog) still survives the chrome filter (#1251)', () => {
108+
// Filter-logic unit test, NOT a live-capture-path claim (#1264 finding 2): this
109+
// exercises `collectSettleChromeRefs` in isolation over a synthetic systemui
110+
// run appended to an unrelated capture fixture. The leaf ids ARE real,
111+
// live-verified volume-dialog ids (`volume_dialog_container`,
112+
// `volume_new_ringer_active_icon_container`), but appending them to this
113+
// fixture does not reproduce how a live capture reaches the filter — that is
114+
// covered separately by the full-capture invariant test below. The point
115+
// here is narrower: the status-bar/nav-bar leaf-id set (#1251) must NOT
116+
// broaden to "any systemui id" and drop an actionable overlay it is handed.
115117
const rawWithVolumeDialog: RawSnapshotNode[] = [
116118
...ANDROID_IME_CAPTURE_RAW_NODES,
117119
{
@@ -125,22 +127,25 @@ test('Android actionable systemui overlay (volume dialog) still survives with th
125127
parentIndex: 9000,
126128
type: 'android.widget.ImageButton',
127129
bundleId: 'com.android.systemui',
128-
identifier: 'com.android.systemui:id/volume_dialog_slider',
129-
label: 'Media volume',
130+
identifier: 'com.android.systemui:id/volume_new_ringer_active_icon_container',
131+
label: 'Ringer volume',
130132
hittable: true,
131133
},
132134
];
133135
const nodes = attachRefs(walkNonRawAndroidFixture(rawWithVolumeDialog));
134136
const chromeRefs = collectSettleChromeRefs(nodes, 'com.callstack.agentdevicelab');
135137

136138
// The container is non-hittable with a generic id, but its hittable
137-
// `volume_dialog_slider` child keeps it in the walked tree (descendantHittable).
139+
// `volume_new_ringer_active_icon_container` child keeps it in the walked
140+
// tree (descendantHittable).
138141
assert.equal(
139142
chromeRefs.has(refForIdentifier(nodes, 'com.android.systemui:id/volume_dialog_container')),
140143
false,
141144
);
142145
assert.equal(
143-
chromeRefs.has(refForIdentifier(nodes, 'com.android.systemui:id/volume_dialog_slider')),
146+
chromeRefs.has(
147+
refForIdentifier(nodes, 'com.android.systemui:id/volume_new_ringer_active_icon_container'),
148+
),
144149
false,
145150
);
146151
// The status-bar leak fix stays active in the very same tree.

0 commit comments

Comments
 (0)