test: improve changed validation routing

Author: Pluze

.agents/skills/labkit-test-planner/SKILL.md

@@ -25,17 +25,24 @@ another skill already read shared AGENTS context, do not reread it.
 
 Use the smallest source-aligned validation set that covers the touched
 boundary. `docs/testing.md` owns the stable build-task names, CI scope, and
-command examples. Build tasks are broad discovery-driven entry points; use
+command examples. Keep the public build-task set small: improve changed-file
+planner routing, representative selectors, printed plan reasons, or focused
+runner selectors before adding a new public task.
+
+Use the fast changed-file build task for tight local iteration when git state
+is available, and the conservative changed-file build task before handoff.
+These tasks inspect the current diff and print why each selected scope is
+being run. Use
 `runLabKitTests("Suites", ...)` for component, app-family, or focused GUI
-iteration. For local GUI edits that only touch one app, prefer the app-level
-GUI folder, for example a `Suites` value such as
+diagnosis after a scope is known. For local GUI edits that only touch one app,
+prefer the app-level GUI folder, for example a `Suites` value such as
 `gui/apps/image_measurement/batch_crop` with `IncludeGui=true` and
 `GuiMode="hidden"`.
 
 For long focused suites, `runLabKitTests` can split a selected suite with
 zero-based `ShardCount` and `ShardIndex`. Use shards only when every shard is
 run with a distinct `RunName` and the combined shards cover the same selected
-suite; keep `buildtool changed` or the relevant broad build task as the
+suite; keep the conservative changed-file or relevant broad build task as the
 handoff gate for substantive changes.
 
 For a dirty worktree, route through the changed-file validation planner before
@@ -49,9 +56,9 @@ instead of rerunning narrower tests for ceremony.
 After a planned run fails, do not rerun the planner just to discover the same
 scope again. Fix the root cause and rerun the narrowest failed suite or test
 directly, for example `runLabKitTests("Suites", "project")` after a project
-guardrail failure. Escalate back to `buildtool changed`, `headless`, or `gui`
-only when the fix touches additional areas, changes validation routing, or the
-user explicitly asks for a broader/release gate.
+guardrail failure. Escalate back to the changed-file, headless, or GUI build
+task only when the fix touches additional areas, changes validation routing,
+or the user explicitly asks for a broader/release gate.
 
 ```text
 project                    startup, architecture, package surface, sample-data hygiene

docs/README.md

@@ -14,8 +14,8 @@ the app catalog; maintainer references are grouped separately.
 
 - [apps.md](apps.md): app file shape, app ownership, and new-app guidance.
 - [architecture.md](architecture.md): package boundaries and extraction rules.
-- [testing.md](testing.md): supported build tasks, CI scope, fixtures, and GUI
-  validation limits.
+- [testing.md](testing.md): compact build-task entry points, automatic
+  changed-file routing, CI scope, fixtures, and GUI validation limits.
 - [release.md](release.md): version selection, tag naming, and GitHub release
   note format.
 - [workflow-assets.md](workflow-assets.md): command-line generation of real app

docs/testing.md

@@ -1,11 +1,13 @@
 # Testing
 
-Use this page to choose the smallest supported build task that proves the
-change you made.
+Use this page to choose the smallest supported validation entry point. The
+public build-task set is intentionally small; changed-file tasks inspect the
+current git diff and print why each selected scope is being run.
 
 ## Default Check
 
-Run the default non-GUI build task for broad local validation:
+Run the default broad non-GUI check when you are not sure which focused scope
+applies:
 
 ```bash
 buildtool headless
@@ -39,10 +41,10 @@ buildtool listTasks
 
 | Task | Use it for |
 | --- | --- |
-| `changed` | Fast local validation selected from changed and untracked files. |
-| `changedFast` | Faster changed-file validation for local iteration; uses representative GUI workflow/layout coverage when broad app GUI coverage would otherwise run. |
+| `changedFast` | Tight local iteration from the current diff; substitutes representative GUI coverage for expensive broad GUI scopes. |
+| `changed` | Conservative pre-handoff validation from the current diff. |
 | `headless` | Full non-GUI validation. |
-| `gui` | Noninteractive GUI launch, layout, callback, workflow, and gesture checks. GUI windows are hidden by default. |
+| `gui` | Full automated GUI validation with hidden figures. |
 
 Report and discovery tasks:
 
@@ -53,34 +55,28 @@ Report and discovery tasks:
 
 ## Choosing A Task
 
-Use build tasks directly for local iteration. The `changed` task is the
-default focused choice before committing: it inspects changed and untracked
-files and runs a conservative serial validation plan inside one MATLAB
-process. It requires git and a git checkout. Use `headless` in exported
-source trees, packaged copies, or environments without git state.
-Use `changedFast` during tight edit cycles when a shared UI or broad app-GUI
-change would otherwise run the full downstream app GUI suite. It keeps
-reusable UI coverage and representative downstream app checks, but it is
-not a substitute for the conservative `changed` task before handoff.
-Fast validation plans may combine suite routing with test-name selectors for
-expensive GUI coverage. Keep those selectors small and contract-oriented:
-choose tests that cover startup, declarative shell behavior, debug trace
-plumbing, hidden synthetic workflows, and representative downstream app
-layouts.
+Prefer automatic routing before hand-picking suites:
 
 Common choices:
 
 | Change area | Build task |
 | --- | --- |
-| Changed source, tests, or docs before commit | `buildtool changed` |
-| Local iteration after a broad UI or GUI-adjacent edit | `buildtool changedFast` |
+| Local iteration while files are changing | `buildtool changedFast` |
+| Before commit, PR, or handoff | `buildtool changed` |
 | Full broad non-GUI validation | `buildtool headless` |
-| Any GUI launch, layout, callback, or gesture change | `buildtool gui` |
-| Architecture, docs, package surface, hygiene | `buildtool headless` |
+| Full automated GUI validation | `buildtool gui` |
+| Coverage report | `buildtool coverage` |
 
-Focused runner suite targets include `labkit/image` for the reusable image
-facade, `labkit/thermal` for thermal parsing and rendering, and
-`apps/image_measurement` for downstream image apps.
+`changedFast` and `changed` require a git checkout. In exported source trees,
+packaged copies, or environments without git state, use `headless` or an
+explicit `runLabKitTests` suite selection instead.
+
+The changed-file planner routes by source ownership. For example, a single app
+change maps to that app family plus its GUI folder when one exists; reusable
+UI changes map to reusable UI coverage plus downstream GUI coverage; docs,
+AGENTS, tools, and runner support files map to project guardrails unless their
+own tests require broader coverage. The printed plan includes the selected
+suites, test-name selectors, GUI mode, and reason for each step.
 
 ## CI Scope
 
@@ -137,43 +133,29 @@ is an implementation detail used by the buildfile.
 
 ## Focused And Parallel Iteration
 
-For tight local diagnosis, run the smallest suite directly through the lower-level
-runner after adding `tests` to the MATLAB path:
+When diagnosing one failure, rerun the smallest matching suite directly after
+adding `tests` to the MATLAB path:
 
 ```matlab
 addpath("tests")
 runLabKitTests("Suites", "gui/apps/image_measurement/batch_crop", ...
     "IncludeGui", true, "GuiMode", "hidden", "HtmlReport", false)
 ```
 
-For long GUI or broad component suites, the runner can split a selected suite
-into deterministic zero-based shards. Start each shard in a separate MATLAB
-process and give each shard a distinct run name:
+For long GUI or broad component suites, split the same selected suite into
+deterministic zero-based shards. Start each shard in a separate MATLAB process
+and give each shard a distinct run name:
 
 ```matlab
 runLabKitTests("Suites", "gui", "IncludeGui", true, "GuiMode", "hidden", ...
     "ShardCount", 3, "ShardIndex", 0, "RunName", "gui_shard_0", ...
     "HtmlReport", false)
 ```
 
-Repeat with `ShardIndex` 1 and 2 for the remaining shards. Sharding is useful
-for local agent or maintainer iteration when MATLAB licenses and host resources
-allow multiple processes. Use the same suite, test-name filters, tags, and GUI
-settings for every shard; vary only `ShardIndex` and `RunName`. Keep final
-handoff validation on the official build task unless the sharded runs together
-cover the same selected suite and all shards passed.
-
-App GUI tests live at:
-
-```text
-tests/cases/gui/apps/<family>/<app_slug>/
-```
-
-When a change affects one app, `buildtool changed` maps the touched app
-folder to the matching GUI test folder when one exists. Shared UI, launcher,
-runner, or broad documentation changes map to broader build-task selections.
-Maintainer diagnostics under `tools/` map to project guardrails rather than the
-full non-GUI suite unless their tests or shared runner code also changed.
+Repeat with `ShardIndex` 1 and 2 for the remaining shards. Use the same suite,
+test-name filters, tags, and GUI settings for every shard; vary only
+`ShardIndex` and `RunName`. A sharded result is equivalent only when all shards
+passed and together cover the same selected suite.
 
 ## GUI Validation
 

tests/AGENTS.md

@@ -25,6 +25,9 @@ Tests mirror source ownership. Do not create a parallel runner framework unless
 - Do not add a separate custom runner or direct pass/fail test tree. Build
   tasks are the human and CI entry points; `tests/runLabKitTests.m` is the
   lower-level implementation behind those tasks.
+- Do not add public build tasks for every timing strategy. Keep the public
+  build-task set compact and improve changed-file planner routing, printed
+  plan reasons, representative selectors, or sharding support instead.
 - CI may shard non-GUI validation across multiple GitHub Actions jobs for
   wall-clock speed. Keep those shards as CI-only buildfile tasks invoked with
   `matlab-actions/run-build`; workflow YAML must not call
@@ -112,15 +115,17 @@ Tests mirror source ownership. Do not create a parallel runner framework unless
 
 ## Validation Scope Discipline
 
-- Use the changed-file planner once to choose the initial affected scope for a
-  dirty worktree. If that planned run fails, fix the specific failure and
-  rerun the narrowest failing scope or failing suite directly; do not rerun
-  `buildtool changed` just to rediscover the same plan.
-- Use the fast changed-file task for tight local iteration when shared UI or
-  broad GUI-adjacent edits would trigger full downstream app GUI coverage. It
-  is an iteration gate, not the final handoff gate; run the conservative changed
-  task or the relevant broad task before pushing substantive
-  validation-routing changes.
+- Use the fast changed-file build task for local iteration and the
+  conservative changed-file build task for pre-handoff validation when git
+  state is available. These tasks should route from the current diff and print
+  why each selected scope is being run.
+- If a changed-file plan fails, fix the specific failure and rerun the
+  narrowest failing scope or suite directly; do not rerun the changed-file
+  planner just to rediscover the same plan.
+- For new timing strategies, first improve planner routing or representative
+  selectors. Add a new public task only when the workflow cannot be expressed
+  through the compact task set listed in `docs/testing.md` or a focused
+  `runLabKitTests` invocation.
 - Prefer `runLabKitTests("Suites", "...")` for rerunning a failed suite such
   as `project`, `labkit/ui`, `labkit/image`, `labkit/thermal`, or
   `apps/image_measurement`. Rerun broader

tests/cases/contract/project/build/BuildTaskEfficiencyGuardrailTest.m

@@ -29,19 +29,28 @@ function changedFastValidationPlanUsesRepresentativeAppGuiCoverage(testCase)
                 "+labkit/+ui/+app/runBusy.m", "Mode", "fast");
             signatures = validationStepSignatures(steps);
             tests = validationStepTestSelectors(steps);
+            reasons = validationStepReasons(steps);
 
             testCase.verifyTrue(any(signatures == "labkit/ui|false"), ...
                 "Fast changed UI validation should keep reusable UI non-GUI coverage.");
             testCase.verifyTrue(any(signatures == "gui/labkit/ui|true"), ...
                 "Fast changed UI validation should keep reusable UI GUI coverage.");
+            testCase.verifyTrue(any(signatures == "gui/gesture/labkit/ui|true"), ...
+                "Fast changed UI validation should keep representative UI gesture coverage.");
             testCase.verifyFalse(any(signatures == "gui/apps|true"), ...
                 "Fast changed UI validation should avoid the full downstream app GUI suite.");
             testCase.verifyTrue(any(signatures == ...
                 "gui/apps/image_measurement/image_enhance,gui/apps/image_measurement/batch_crop|true"), ...
                 "Fast changed UI validation should run representative downstream app GUI coverage.");
             testCase.verifyTrue(any(contains(tests, "test_gui_layout_ui_declarative_app") & ...
+                contains(tests, "test_gui_layout_ui_scale_bar_tool") & ...
                 contains(tests, "test_gui_layout_ui_debug_trace")), ...
                 "Fast changed UI validation should use reusable UI test-name selectors.");
+            testCase.verifyTrue(any(contains(tests, ...
+                "referenceEditAndPlacementEmitStructuredTrace")), ...
+                "Fast changed UI validation should include representative scale-bar gesture coverage.");
+            testCase.verifyTrue(all(strlength(reasons) > 0), ...
+                "Changed validation plan steps should explain why each scope was selected.");
 
             appGuiSelectors = [ ...
                 "image_enhance_workflow_applies_tool_and_exports", ...
@@ -52,6 +61,14 @@ function changedFastValidationPlanUsesRepresentativeAppGuiCoverage(testCase)
             verifySelectorsMatchTests(testCase, appGuiSelectors, ...
                 ["gui/apps/image_measurement/image_enhance", ...
                 "gui/apps/image_measurement/batch_crop"]);
+            verifySelectorsMatchTests(testCase, [ ...
+                "test_gui_layout_ui_declarative_app", ...
+                "test_gui_layout_ui_scale_bar_tool", ...
+                "test_gui_layout_ui_debug_trace"], ...
+                "gui/labkit/ui");
+            verifySelectorsMatchTests(testCase, ...
+                "referenceEditAndPlacementEmitStructuredTrace", ...
+                "gui/gesture/labkit/ui");
         end
     end
 end
@@ -73,6 +90,15 @@ function changedFastValidationPlanUsesRepresentativeAppGuiCoverage(testCase)
     end
 end
 
+function reasons = validationStepReasons(steps)
+    reasons = strings(1, numel(steps));
+    for k = 1:numel(steps)
+        if isfield(steps(k), "Reason")
+            reasons(k) = string(steps(k).Reason);
+        end
+    end
+end
+
 function verifySelectorsMatchTests(testCase, selectors, suites)
     output = listLabKitTestsQuietly( ...
         "Suites", suites, ...

tests/runLabKitTests.m

@@ -196,9 +196,10 @@ function ensurePlanHasNoExplicitSelectors(opts)
 
 function printValidationPlanSteps(steps)
     for k = 1:numel(steps)
-        fprintf("  %d. %s includeGui=%d suites=%s tests=%s\n", ...
+        fprintf("  %d. %s includeGui=%d suites=%s tests=%s reason=%s\n", ...
             k, steps(k).RunNameSuffix, steps(k).IncludeGui, ...
-            stepSuitesLabel(steps(k)), stepTestsLabel(steps(k)));
+            stepSuitesLabel(steps(k)), stepTestsLabel(steps(k)), ...
+            stepReasonLabel(steps(k)));
     end
 end
 
@@ -217,6 +218,14 @@ function printValidationPlanSteps(steps)
     end
 end
 
+function label = stepReasonLabel(step)
+    if ~isfield(step, "Reason") || strlength(string(step.Reason)) == 0
+        label = "<unspecified>";
+    else
+        label = char(step.Reason);
+    end
+end
+
 function steps = namedValidationPlan(planName)
     switch planName
         case "ui"