chore: stabilize Maestro CI (Maestro 2.5.1, emulator headroom)#7311
chore: stabilize Maestro CI (Maestro 2.5.1, emulator headroom)#7311diegolmello wants to merge 11 commits into
Conversation
Maestro 2.5.0 switched gRPC to dadb direct ADB socket — removes the broken-pipe install class on the Android emulator. Drop the monkey warmup that was stressing system_server right before driver install. Broaden iOS retry to fire on any error, not just timeout.
Bump emulator RAM 4G→6G to keep system_server out of the meminfo-dump timeout window, free ~6 GB pre-launch via jlumbroso/free-disk-space, and split the default-profile path into a cache-miss snapshot warmup plus a cache-hit replay (-no-snapshot-save) so we stop cold-booting the AVD on every shard.
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughThis PR upgrades Maestro to 2.5.1 (Android/iOS), changes Android AVD caching and emulator RAM/options, replaces the Android monkey pre-test step with ChangesMaestro Test Infrastructure Upgrade
Sidebar View Layout Change
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Suggested labels
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Warning Review ran into problems🔥 ProblemsErrors were encountered while retrieving linked issues. Errors (4)
Tip 💬 Introducing Slack Agent: The best way for teams to turn conversations into code.Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.
Built for teams:
One agent for your entire SDLC. Right inside Slack. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
.github/workflows/maestro-android.yml (1)
41-48:⚠️ Potential issue | 🟠 Major | ⚡ Quick winShard 14 can poison the shared AVD cache, defeating the snapshot optimization for all other shards
The
Cache Android AVDstep has noifcondition, so shard 14 participates in both cache restore and save under the same key (avd-${{ runner.os }}-api34).On a cold run, all 14 shards race to save to that key (GitHub Actions is first-writer-wins). If shard 14 wins, the cache stores only
Pixel_API_34_Keyboard.avd+Pixel_API_34_Keyboard.ini. On subsequent runs:
- Non-keyboard shards see
steps.avd-cache.outputs.cache-hit == 'true'- The "Generate AVD snapshot" step is skipped (its condition checks
cache-hit != 'true')- The test step runs with
force-avd-creation: false— but the expected default-profile AVD is absent from the restored cacheThe behavior of
reactivecircus/android-emulator-runnerwhenforce-avd-creation: falseand the named AVD doesn't exist is unspecified; it may silently cold-boot (losing the optimization) or fail outright. Either way, the caching strategy is silently broken.🐛 Proposed fix — exclude shard 14 from AVD caching
- name: Cache Android AVD + if: ${{ inputs.shard != '14' }} id: avd-cache uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0 with: path: | ~/.android/avd/* ~/.android/adb* key: avd-${{ runner.os }}-api34🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/maestro-android.yml around lines 41 - 48, The "Cache Android AVD" step (id: avd-cache, name: Cache Android AVD, key: avd-${{ runner.os }}-api34) must be excluded from shard 14 so it doesn't poison the shared AVD cache; update the step to run only when the shard is not 14 (e.g. add an if condition referencing your matrix/shard variable such as if: matrix.shard != 14) so shard 14 neither restores nor saves to the common key, leaving other shards to generate and cache the full AVD snapshot correctly.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/maestro-android.yml:
- Around line 24-33: The workflow currently uses the jlumbroso/free-disk-space
action with swap-storage: true while configuring the emulator's ram-size: 6144M,
which disables swap and leaves under ~800MB host memory — change either the
swap-storage or ram-size to avoid OOMs: set swap-storage: false to retain the
runner swapfile, or reduce ram-size to 5120M or 4608M so host+emulator fit in
physical RAM; update the keys swap-storage and ram-size in the action usage
(uses: jlumbroso/free-disk-space@...) and apply the same change to the other
occurrences noted (the other blocks referenced) so all workflow entries are
consistent.
In @.github/workflows/maestro-ios.yml:
- Around line 14-15: The comment incorrectly mentions the Android-specific
"dadb"/ADB change; update the comment text that currently references "dadb" and
ADB to an iOS-appropriate note explaining that iOS Maestro uses
idb/libimobiledevice (not ADB) and that the Android-specific CLI 2.5.0 change
does not apply to the iOS workflow; either remove the ADB-specific sentence or
replace it with a short iOS-focused explanation and keep or adjust the release
link as needed so the comment accurately reflects iOS tooling.
---
Outside diff comments:
In @.github/workflows/maestro-android.yml:
- Around line 41-48: The "Cache Android AVD" step (id: avd-cache, name: Cache
Android AVD, key: avd-${{ runner.os }}-api34) must be excluded from shard 14 so
it doesn't poison the shared AVD cache; update the step to run only when the
shard is not 14 (e.g. add an if condition referencing your matrix/shard variable
such as if: matrix.shard != 14) so shard 14 neither restores nor saves to the
common key, leaving other shards to generate and cache the full AVD snapshot
correctly.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: ee299d9c-ce31-45fd-89b1-673983985558
📒 Files selected for processing (3)
.github/scripts/run-maestro.sh.github/workflows/maestro-android.yml.github/workflows/maestro-ios.yml
💤 Files with no reviewable changes (1)
- .github/scripts/run-maestro.sh
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
- GitHub Check: E2E Build Android / android-build
- GitHub Check: ESLint and Test / run-eslint-and-test
|
Android Build Available Rocket.Chat 4.72.0.108814 |
|
Android Build Available Rocket.Chat 4.72.0.108814 Internal App Sharing: https://play.google.com/apps/test/RQQ8k09hlnQ/ahAO29uNRwjAsJvaiquGjKRmucVItIwkqzqmtF75VX3MLDxOhuznxMA2GxetWxp_dAfRbh33wL0-dHZMTeQVHqchi7 |
|
Android Build Available Rocket.Chat Experimental 4.72.0.108813 |
|
Android Build Available Rocket.Chat Experimental 4.72.0.108813 Internal App Sharing: https://play.google.com/apps/test/RQVpXLytHNc/ahAO29uNTowuRjjHlDYUgXqroMl8epJWmBJOAe-7N_6W5JjAcJyBHu-MfbZDIilMfFnPJkwzs7kKMzme1NVFTUUxnE |
|
iOS Build Available Rocket.Chat Experimental 4.72.0.108816 |
|
Android Build Available Rocket.Chat Experimental 4.72.0.108821 Internal App Sharing: https://play.google.com/apps/test/RQVpXLytHNc/ahAO29uNQLTMEQOTK7Yu4n98ufM0ImDLGPoEI7tdqqt4_8PqA3XXsk_0JhIS1tJIiRPRZJU0Eh3mtvcPjDUgnnZ0rq |
|
Android Build Available Rocket.Chat 4.72.0.108820 Internal App Sharing: https://play.google.com/apps/test/RQQ8k09hlnQ/ahAO29uNQrTqib34jYoZve6A-78lxC3zI5hjF5NuHRtFdGLOxp7tcQ3xwQyvjeTqlI5Lq2c3nCQqQiv5zaR2Rc0VBd |
|
iOS Build Available Rocket.Chat Experimental 4.72.0.108822 |
The 60s→120s bump in c14d8d0 didn't address the actual flake. Investigation showed that maestro's gesture injection causes the room view to exit to the rooms list mid-scroll, regardless of timeout, hideKeyboard, or constrained swipe coords — manual `adb shell input swipe` at identical coordinates works fine. Pre-existing shard 12 behavior stays covered by maestro's internal rerun loop, as it has since PR #6680 landed. Restores the file to its develop form.
Maestro 2.5.1 on iOS shard 6 (DeepLink → create-account on alternate server) failed because the email TextInput sat under the software keyboard when Maestro's tapOn fired. The element's hit-test centre lay below the keyboard top edge, so the tap routed to the QWERTY 't' key and deposited 't' into the already-focused email field before inputText typed the rest, yielding 'tmobile+…@rocket.chat'. FormContainer wrapped a plain ScrollView in KeyboardAvoidingView (behavior='padding'), which only padded the bottom inset on keyboard appearance — focused inputs were never scrolled above the keyboard. Swap for KeyboardAwareScrollView from react-native-keyboard-controller (already a dependency). pressKey: enter on each field still drives setFocus(next) via onSubmitEditing, which KAS now turns into an auto-scroll bringing the next field above the keyboard before Maestro taps it. Verified locally: 3/3 successful create-account reproductions on iPhone 16 Pro iOS 26 with CI sim defaults (UIAnimationDragCoefficient=10), down from ~40% (2/5) failure rate on the unfixed branch.
|
Android Build Available Rocket.Chat Experimental 4.73.0.108867 Internal App Sharing: https://play.google.com/apps/test/RQVpXLytHNc/ahAO29uNTnzfDm_AHURT9Pw2SncEbPOaRNvLiEXkhsX2mzyk3VI0uTIlNGSEvB0mfJlKEpMmMk1MBQ45cY2PHUWiYs |
|
iOS Build Available Rocket.Chat Experimental 4.73.0.108869 |
Conditional `tapOn: header-back` only fires when room-view is still visible — covers both paths after `hideKeyboard` (which on Android issues KEYCODE_BACK whether the IME is up or not, so it sometimes already pops nav on its own).
The Create Team and Delete flow was the only Maestro test tapping the room header via accessibility-label text (regex `private team <name> .`). Under Maestro 2.5.1, when the room subtitle shows "Waiting for network..." (slower WebSocket reconnect after team creation on CI), the wrapping View's accessibilityLabel extends past the regex and the tap target is not found. Same class as the sidebar-view testID fix earlier in this PR. Switch to `id: 'room-header'`, matching every other flow that opens RoomActionsView from the room header.
Proposed changes
CI stabilization (original scope)
Broken pipeinstall class that has been the dominant Android Maestro shard failure mode.monkeywarmup (and its trailingsleep 6+am force-stop) inrun-maestro.sh— the warmup was stressingsystem_serverimmediately before Maestro's driver install attempt.retry_on: timeouttoretry_on: error.reactivecircus/android-emulator-runnerinvocation (ubuntu-latesthas 16 GB total — leaves comfortable headroom).jlumbroso/free-disk-space@v1.3.1(Android SDK and tool cache preserved; ~6 GB freed).force-avd-creation: trueto seed the snapshot, cache hit → replay withforce-avd-creation: falseand-no-snapshot-saveto prevent corruption. Keyboard AVD path (shard 14) untouched — has its own manual recreation each run.The bespoke per-failed-flow JUnit retry loop in
run-maestro.shis intentionally retained — it reruns only failed flows, which is strictly better than any native Maestro mechanism (no--retryCLI flag in 2.5.1; YAMLretry:block restarts whole flow).Android job-level retry wrapper deferred —
nick-fields/retry@v3only wraps shell commands, notuses:actions; will revisit in a follow-up.Sidebar testID fix (folded in)
PR #6918 (Voice support, cd2faa0) collapsed the
SidebarViewwrapper —testID='sidebar-view'moved from aSafeAreaViewonto theScrollView. On Android, React Native'sScrollViewdoes not propagatetestIDto the native accessibility tree, so Maestro can no longer find the element. TheSafeAreaView(a regularView) did. Wrapping theScrollViewin a plainViewthat carries the testID restores accessibility-tree visibility without changing the layout (both usestyles.container→flex: 1).This unblocks shard 8 (Android) and 12 other Maestro flows that wait on
sidebar-view:profile.yaml,status.yaml,setting.yaml,user-preferences.yaml,change-avatar.yaml,i18n.yaml,e2ee/utils/navigate-to-e2ee-security.yaml,assorted/utils/check-server.yaml,assorted/utils/nav-to-language.yaml, etc.Profile test rate-limit fix (folded in)
Shard 8 (
profile.yaml) was failing on Android even after the sidebar testID fix: the test made three consecutiveusers.updateOwnBasicInfocalls (name+username, then nickname+bio, then email) within Maestro 2.5.1's faster pacing. The endpoint is rate-limited per-user, so the second or third call returned HTTP 429 and the error alert disrupted the password-confirm sheet that follows the email change.Collapse the three submits into one combined submit that edits all five fields (name, username, nickname, bio, email) before tapping submit. The email change still triggers the
ConfirmEmailChangeActionSheet, so the password sheet → save path is preserved. Result: 3 rate-limited calls → 1 call. The downstream password-change step (ChangePasswordView) is unchanged.Form keyboard scroll fix — Maestro iOS shard 6 (folded in)
iOS shard 6 (
assorted/deeplink.yaml→helpers/create-account.yamlon the alternate server) was failing with the email field corrupted astmobile+…@rocket.chator similar leading-character mutations. Maestro logs showed it had typed the correct text; the field state did not match.Root cause is the keyboard covering the focused field at tap time.
FormContainerwraps a plainScrollViewinKeyboardAvoidingView(behavior='padding'), which only pads the bottom inset on keyboard appearance — focused inputs are never scrolled above the keyboard. With each field'sonSubmitEditing → setFocus(next)advancing focus through the form, the email input's hit-test centre ended up below the keyboard top edge. Maestro 2.5.1'stapOnthen routed to the QWERTYtkey, depositingtinto the (already-focused) email field beforeinputTexttyped the rest. Maestro 2.2.0 didn't expose this because its input mechanism predated keystroke-based typing.Swap
KeyboardAvoidingView + ScrollViewforKeyboardAwareScrollViewfromreact-native-keyboard-controller(already a dep). KAS auto-scrolls the focusedTextInputabove the keyboard on focus change, so every subsequenttapOnlands on a visible target. Verified locally on iPhone 16 Pro iOS 26 with CI sim defaults (UIAnimationDragCoefficient=10): 3/3 reproductions, down from 2/5 failure rate (40%) on the unfixed branch.Known status — shard 12 (
room-last-message-thread-50-plus.yaml) deferredTracked as NATIVE-1128. Earlier commits on this branch (
fecac89bd,c14d8d0ff) attempted to stabilize shard 12'sscrollUntilVisiblestep; both have been reverted (file matchesdevelopexactly). Investigation against a local emulator showed the failure isn't a timeout, keyboard, or coordinate-bounds issue:adb shell input swipe 540 840 540 1800 400repeatedly load older messages and keep the room view intact.swipe: start: 50%,35% end: 50%,75%(orscrollUntilVisible direction:UP) cause the room view to exit to the rooms list mid-scroll, then time out searching formessage-content-message 1in a screen where it can't possibly appear.hideKeyboard(keyboard re-opens mid-scroll on CI) and gating the scroll onnotVisible: "Waiting for network..."did not change the outcome.Root cause is in maestro's Android gesture injection vs RN's gesture handling, not in the YAML or the app. Tracked for follow-up. CI's per-flow internal rerun continues to recover this shard the same way it has since #6680 landed (see run 25804460186 — Android shard 12 failed first attempt at 2m 51s, passed on rerun in 44s).
Known status — shard 11 (
jump-to-message.yaml) deferredShard 11 remains chronically red on both Android and iOS, and is not fixed by this PR. Treated as a pre-existing baseline failure per the team's "gate on e2e build success, treat shard pass/fail as informational" policy.
Investigation during this PR surfaced two product regressions in the room view that likely contribute to the Maestro flow failing (the flow exercises both the composer and the quote-attachment tap path):
Root cause (both regressions and the shard 11 flow itself) is not yet confirmed and is tracked as follow-up work. Shard 11 will be re-enabled once those bugs land their fixes.
Issue(s)
NATIVE-1123
Related (deferred, will not be fixed in this PR):
How to test or reproduce
Broken pipe,meminfo DUMP TIMEOUT,Monkey aborted, andFailed to install apk.maestro --device emulator-5554 hierarchy | grep sidebar-viewreturns 0 matches on develop and returns the testID on this branch. Shard 8 (profile.yaml) progresses past the sidebar step.ChangePasswordViewand finish.rooms-list-viewafterhelpers/create-account.yamlwithout the email field being corrupted (no leadingt/tmobilesubstitution).developruns (informational here; full 10-run acceptance window is a separate follow-up).jump-to-message.yaml) is expected to remain red — see deferred-status note above.Screenshots
n/a
Types of changes
Checklist
Further comments
Acceptance is measured empirically over a 10-run window on
developafter this lands; that measurement is tracked separately.The sidebar testID fix was originally opened as #7319 and folded in here (#7319 closed) so the CI stabilization and the test-visibility fix it depends on ship together.
Summary by CodeRabbit
Chores
Tests
Style