-
Notifications
You must be signed in to change notification settings - Fork 669
Add dynamo-build-tests and dynamo-run-tests agent skills #17044
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
jasonstratton
wants to merge
3
commits into
DynamoDS:master
Choose a base branch
from
jasonstratton:master
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+223
−0
Open
Changes from all commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,90 @@ | ||
| --- | ||
| name: dynamo-build-tests | ||
| description: Build Dynamo test projects so test DLLs reflect the latest source changes. Use this skill when you need to compile test projects before running tests, when dotnet build fails on satellite assembly projects, or when you see "discovered 0 of 0 NUnit test cases" because the DLL is stale. | ||
| --- | ||
|
|
||
| # Dynamo Build Tests | ||
|
|
||
| ## When to use | ||
| - Compiling test projects before running tests | ||
| - Rebuilding a specific test DLL after editing test or production code | ||
| - Troubleshooting build failures in the test pipeline (e.g. `MSB4803`, missing LibG) | ||
|
|
||
| ## When not to use | ||
| - Running tests -- use `dynamo-run-tests` instead | ||
| - Writing test code -- use `dynamo-unit-testing` instead | ||
|
|
||
| ## Inputs expected | ||
| The name of the test project or area being tested (e.g. "DynamoCoreWpf3Tests", "DynamoCoreTests"). | ||
|
|
||
| ## Output format | ||
| Confirmation that the build succeeded with 0 errors in the target project, or a diagnosis of any failures. | ||
|
|
||
| --- | ||
|
|
||
| ## Workflow | ||
|
|
||
| ### 1. Resolve tool paths | ||
|
|
||
| Do not hardcode tool paths. Obtain `<MSBuildPath>` and `<DotNetPath>` from your environment before running any commands: | ||
|
|
||
| - **Claude Code users**: read `reference_build_tools.md` from the personal memory file (`~/.claude/projects/<project>/memory/reference_build_tools.md` — not part of the repo). Update it if a path is stale. | ||
| - **All users**: discover MSBuild via `vswhere`: | ||
|
|
||
| ```bash | ||
|
jasonstratton marked this conversation as resolved.
|
||
| "/c/Program Files (x86)/Microsoft Visual Studio/Installer/vswhere.exe" -latest -requires Microsoft.Component.MSBuild -find "MSBuild\**\Bin\MSBuild.exe" | ||
| ``` | ||
|
|
||
| Use the resolved paths as `<MSBuildPath>` and `<DotNetPath>` in the commands below. | ||
|
|
||
| ### 2. Build the test project | ||
|
|
||
| **Preferred: MSBuild** (this repo builds with MSBuild; it handles all project types correctly): | ||
|
|
||
| ```bash | ||
| "<MSBuildPath>" "test/<TestProject>/<TestProject>.csproj" "-p:Configuration=Release" "-v:q" "-nologo" | ||
| ``` | ||
|
|
||
| > **Important**: When calling MSBuild from bash, quote all flag arguments (e.g. `"-p:Configuration=Release"`) to prevent the shell from interpreting `/p:` as a path. | ||
|
|
||
| **Fallback: `dotnet build --no-dependencies`** if MSBuild is unavailable. This compiles only the specified project against pre-built dependency DLLs already in the output folder, avoiding `MSB4803` errors from satellite assembly tasks: | ||
|
|
||
| ```bash | ||
| "<DotNetPath>" build test/<TestProject>/<TestProject>.csproj -c Release --no-restore --no-dependencies -v q | ||
| ``` | ||
|
|
||
| **If `dotnet build --no-dependencies` fails** because dependency DLLs are missing from the output folder, they need to be built first. Rebuild the production project, then the test project — both with `--no-dependencies`: | ||
|
|
||
| ```bash | ||
| "<DotNetPath>" build src/<ProductionProject>/<ProductionProject>.csproj -c Release --no-restore --no-dependencies -v q | ||
| "<DotNetPath>" build test/<TestProject>/<TestProject>.csproj -c Release --no-restore --no-dependencies -v q | ||
| ``` | ||
|
|
||
| ### 3. Verify success | ||
|
|
||
| Check the build output for `0 Error(s)` in the target project. Filter out known pre-existing errors from unrelated projects: | ||
| - `MSB4803` from `DynamoUnits` / `DesignScriptBuiltin` (satellite assembly task not supported on .NET Core MSBuild) | ||
| - `MSB3030` from `DynamoCore` (missing LibG.Interface.dll — build environment issue) | ||
|
|
||
| These are infrastructure issues, not code errors. | ||
|
|
||
| ## Finding test projects | ||
|
|
||
| Discover available test projects from the repo: | ||
|
|
||
| ```bash | ||
| find test -name "*.csproj" | sort | ||
| ``` | ||
|
|
||
| > **Exception**: The visualization test assembly is `WpfVisualizationTests`, but its directory is `test/VisualizationTests/` — directory and project name differ. All other projects follow the `test/<Name>/<Name>.csproj` pattern. | ||
|
|
||
| Test DLLs are output to `bin/AnyCPU/Release/<TestProject>.dll`. | ||
|
|
||
| ## Boundaries | ||
|
|
||
| - ✅ **Always**: build test and production projects, report errors | ||
| - ⚠️ **Ask first**: modify `.csproj` files, change build configuration | ||
| - 🚫 **Never**: delete build output folders, modify `Directory.Build.props` | ||
|
|
||
| ## Related skills | ||
| `dynamo-run-tests` • `dynamo-unit-testing` | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,123 @@ | ||
| --- | ||
| name: dynamo-run-tests | ||
| description: Run NUnit tests in the Dynamo repo using dotnet test. Use this skill when executing tests by name, filter, or project — including verifying a fix, running regression suites, or checking specific test results. | ||
| --- | ||
|
|
||
| # Dynamo Run Tests | ||
|
|
||
| ## When to use | ||
| - Running specific tests by name or filter after a code change | ||
| - Running all tests in a project to check for regressions | ||
| - Verifying that new tests pass | ||
| - Debugging test failures (re-running with verbose output) | ||
|
|
||
| ## When not to use | ||
| - Building test projects -- use `dynamo-build-tests` instead | ||
| - Writing test code -- use `dynamo-unit-testing` instead | ||
|
|
||
| ## Inputs expected | ||
| The test project, and optionally a filter (test name, class name, or category). | ||
|
|
||
| ## Output format | ||
| Pass/fail summary with counts. For failures: the failing test name, assertion message, and relevant stack trace. | ||
|
|
||
| --- | ||
|
|
||
| ## Workflow | ||
|
|
||
| ### 1. Resolve tool path | ||
|
|
||
| Do not hardcode the `dotnet` path. Obtain `<DotNetPath>` from your environment: | ||
|
|
||
| - **Claude Code users**: read `reference_build_tools.md` from the personal memory file (`~/.claude/projects/<project>/memory/reference_build_tools.md` — not part of the repo). | ||
| - **All users**: run `where dotnet` (Windows) or `which dotnet` (bash) to locate it. | ||
|
|
||
| ### 2. Find the test project | ||
|
|
||
| Discover available test projects: | ||
|
|
||
| ```bash | ||
| find test -name "*.csproj" | sort | ||
| ``` | ||
|
|
||
| > **Exception**: The visualization test assembly is `WpfVisualizationTests`, but its directory is `test/VisualizationTests/` — directory and project name differ. All other projects follow the `test/<Name>/<Name>.csproj` pattern. | ||
|
|
||
| ### 3. Ensure the DLL is current | ||
|
|
||
| If you edited test or production code since the last build, rebuild first using `dynamo-build-tests`. A stale DLL will either miss new tests ("discovered 0 of 0") or run old code. | ||
|
|
||
| ### 4. Run tests | ||
|
|
||
| Use `dotnet test` with `--no-build` to skip recompilation (since you built in step 3): | ||
|
|
||
| ```bash | ||
| # Run specific test(s) by name substring | ||
| "<DotNetPath>" test test/<TestProject>/<TestProject>.csproj -c Release --no-restore --no-build --filter "FullyQualifiedName~<TestNameSubstring>" -v n | ||
|
|
||
| # Run all tests in a test class | ||
| "<DotNetPath>" test test/<TestProject>/<TestProject>.csproj -c Release --no-restore --no-build --filter "FullyQualifiedName~<ClassName>" -v n | ||
|
|
||
| # Run all tests in a project | ||
| "<DotNetPath>" test test/<TestProject>/<TestProject>.csproj -c Release --no-restore --no-build -v n | ||
| ``` | ||
|
|
||
| > **Important**: Always use `--no-restore --no-build` to avoid triggering the `MSB4803` satellite assembly errors from `DynamoUnits`/`DesignScriptBuiltin`. The DLL must already be built. | ||
|
|
||
| ### 5. Interpret results | ||
|
|
||
| **Successful output** ends with: | ||
| ``` | ||
| Test Run Successful. | ||
| Total tests: N | ||
| Passed: N | ||
| ``` | ||
|
|
||
| **Failed output** includes assertion details: | ||
| ``` | ||
| Failed TestName [Xs] | ||
| Error Message: | ||
| Expected: ... | ||
| But was: ... | ||
| ``` | ||
|
|
||
| ### Filter syntax | ||
|
|
||
| The `--filter` flag uses `dotnet test` filter expressions: | ||
|
|
||
| | Goal | Filter | | ||
| |---|---| | ||
| | Test name contains substring | `"FullyQualifiedName~UnpinGroupedNote"` | | ||
| | Exact test name | `"FullyQualifiedName=DynamoCoreWpfTests.NoteViewTests.UnpinGroupedNote_NoteRemainsInGroup"` | | ||
| | All tests in a class | `"FullyQualifiedName~NoteViewTests"` | | ||
|
|
||
|
jasonstratton marked this conversation as resolved.
|
||
| Multiple tests (OR) — use a literal `|` inside the quoted filter string: | ||
|
|
||
| ```bash | ||
| --filter "FullyQualifiedName~TestA|FullyQualifiedName~TestB" | ||
| ``` | ||
|
|
||
| ### Verbosity levels | ||
|
|
||
| | Flag | Shows | | ||
| |---|---| | ||
| | `-v q` | Quiet — pass/fail summary only | | ||
| | `-v n` | Normal — test names and timing (recommended) | | ||
| | `-v d` | Detailed — full output including stdout from tests | | ||
|
|
||
| ## Troubleshooting | ||
|
|
||
| | Symptom | Cause | Fix | | ||
| |---|---|---| | ||
| | "discovered 0 of 0 NUnit test cases" | DLL is stale — new tests not compiled | Rebuild with `dynamo-build-tests` (step 3) | | ||
| | `MSB4803` errors | `--no-build` was omitted or `dotnet test` triggered a rebuild | Add `--no-restore --no-build` flags | | ||
| | Test passes but behavior is wrong | Production DLL is stale | Rebuild the production project too | | ||
| | `No test matches the given testcase filter` | Filter typo or wrong project | Check spelling; try broader filter like class name | | ||
|
|
||
| ## Boundaries | ||
|
|
||
| - ✅ **Always**: run tests, report results, rerun with different verbosity | ||
| - ⚠️ **Ask first**: run the entire solution's test suite (can take a long time) | ||
| - 🚫 **Never**: add `[Ignore]` or `[Explicit]` to make tests pass; weaken assertions | ||
|
|
||
| ## Related skills | ||
| `dynamo-build-tests` • `dynamo-unit-testing` | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Loaded directly ...is repeated twice?