Quality, skill system, adoption workflow, and pyproject best-practices overhaul

A broad improvement pass across the toolkit, its dogfooded config, the bundled
skills, and the new-project/adoption flow. Tree is green (pyclawd check) and the
dogfood golden baseline is unchanged.

Quality tooling & config model
- BuildConfig: group compile/dist/clean knobs into Project.build (BuildConfig | None);
  compile/dist/clean exit 2 when unset. Scaffold, dogfood, reference example, CHANGELOG
  updated (breaking config change).
- doctor: detect deps by distribution name via importlib.metadata (fixes the false WARN
  for pytest-xdist/pytest-cov); remove the core->commands layering by extracting skill
  install/drift logic into a new core module, skills_install.py.
- check: replace the positional 4-tuple with a StepResult dataclass (JSON unchanged).
- test: clean "unknown category" error; argparse for the timings flags.
- golden: pyclawd now dogfoods golden (GoldenConfig + a dedicated golden tier) with a
  seeded baseline; fix golden_dir to an absolute repo-root path so the CLI and plugin
  agree regardless of pytest rootdir.
- backfill tests: junit/reporting, GoldenStore IO, BuildConfig.

Skills: 7 -> 4 -> 5 (progressive disclosure)
- Collapse the umbrella + tests/quality/docs skills into one `pyclawd` router skill
  (lean SKILL.md + references/{mental-model,tests,quality,docs,packaging}.md, on-demand).
- Keep golden/doctor/upgrade standalone; add a new `pyclawd-adopt` skill (the
  field-tested red->green adoption playbook, self-contained).
- Add `pyclawd skills prune` + `install --prune` + a doctor orphan WARN so skills
  dropped from the bundle don't linger in ~/.claude/skills.
- Consistency/completeness pass: router now routes to adopt; cross-link tables on every
  standalone skill; document skills-prune, coverage, and the timings flags.

Adoption (existing repo -> pyclawd)
- `pyclawd new` adopt mode detects flat-vs-src layout, infers root_markers, prints a
  Phase-0 readiness report, and offers opt-in `--scaffold-pyproject` (appends only
  missing [tool.*] sections, never clobbers).
- Route the "not a project" errors to `pyclawd new` + the pyclawd-adopt skill.
- Golden adoption doctrine: existing tests are the primary oracle for mechanical
  changes; golden is the targeted supplement for outputs tests don't lock.

pyproject best practices (researched, .claude/docs/pyproject-toml.md)
- PEP 639: drop the deprecated `License ::` classifier; add `license-files`.
- PEP 561: ship `py.typed` and include it in the wheel (pyclawd is a typed lib).
- mypy: scope `ignore_missing_imports` to jupyter_cache (no global); fix `mypy_path`
  to `src` (correct for src layout). Stays non-strict by choice.
- pin build backend `hatchling>=1.27` (PEP 639); ruff `select` adds `W`.
- scaffold: line-length 100, `--import-mode=importlib`, `xfail_strict`, `exclude_also`,
  and drop `-n auto` from addopts (pyclawd injects parallelism) so bare pytest / the
  scaffolded CI no longer require pytest-xdist; CI installs the PEP 735 dev group.
- Verified by building the wheel: ships py.typed, METADATA carries License-Expression.

Author: blankjul

.claude/CLAUDE.md

@@ -32,7 +32,7 @@ all of these and reports OK/WARN/FAIL.
 | `pyclawd test …` | **pytest** | `test fast` also wants **pytest-xdist** (`-n auto`) |
 | `pyclawd lint` / `format` | **ruff** | argv set in `QualityConfig` |
 | `pyclawd typecheck` | **mypy** | argv set in `QualityConfig` |
-| `pyclawd compile` / `dist` | the project's build backend | only if `compile_cmd` / `dist_cmd` are set |
+| `pyclawd compile` / `dist` | the project's build backend | only if `project.build` (a `BuildConfig`) sets `compile_cmd` / `dist_cmd`; `build=None` → exit 2 |
 | `pyclawd docs build/run/render/exec` | a **`./docs` runner** (sphinx + nbsphinx, via uvx or installed) | delegated to `DocsConfig.runner`; **`render` also needs the `pandoc` system binary** |
 | `pyclawd docs failures` / `timings` | a **jupyter-cache** backend at `cache_db`; `failures` imports `jupyter_cache` + `nbformat` **in this env** | see the backend note on `DocsConfig` |
 
@@ -79,7 +79,11 @@ toolchain is the dogfooded one: ruff + mypy + pytest, all configured in
 - **Commands degrade, never crash.** An unconfigured group (e.g. `docs`, `quality`)
   self-reports and exits `2`; an undefined test tier applies no `-m` filter. Keep
   that contract — prefer `dict.get(...)` / clean `typer.Exit` over raw `KeyError`s.
-- **Exit-code contract:** `0` ok · `2` not-configured · otherwise the tool's own code.
+- **Exit-code contract:** `0` ok · `2` = not-configured **or** a CLI usage error
+  (Typer's own convention) · otherwise the underlying tool's own code. Note that
+  `pyclawd docs` preflight uses distinct codes — `3` for a missing `pandoc` binary,
+  `1` for "no built docs yet" — so don't assume every nonzero from docs is the
+  runner's code.
 - **Keep the dogfood honest.** If you add a command or config field, update
   `.pyclawd/config.py`, `AGENTS.md`, `README.md`, and the scaffold templates so a
   freshly-`pyclawd new`-ed project still matches reality.

.claude/docs/README.md

@@ -7,6 +7,7 @@ with concrete examples and cited sources. These findings are used to shape the p
 
 - [BEST_PRACTICES.md](BEST_PRACTICES.md) — Opinionated consolidated rules for agent-driven development
 - [PYCLAWD_ROADMAP.md](PYCLAWD_ROADMAP.md) — Concrete change/add recommendations for pyclawd, grouped by priority
+- [pyproject-toml.md](pyproject-toml.md) — Best-practice `pyproject.toml` defaults (build/metadata/license/deps/ruff/mypy/pytest/coverage), with the line-length verdict (June 2026)
 
 ## Source Research
 

.claude/docs/migration.md

@@ -212,6 +212,6 @@ they still run.
 | Need | Reference |
 |---|---|
 | Prove behavior unchanged (the oracle) | `pyclawd-golden` skill · `examples/golden_demo/` |
-| Lint / format / typecheck / `check` gate | `pyclawd-quality` skill |
-| Tiered tests, the fix-loop | `pyclawd-tests` skill |
-| Mental model + full doctrine | `pyclawd` (umbrella) · `AGENTS.md` |
+| Lint / format / typecheck / `check` gate | the `pyclawd` skill's `references/quality.md` |
+| Tiered tests, the fix-loop | the `pyclawd` skill's `references/tests.md` |
+| Mental model + full doctrine | `pyclawd` (router) · `AGENTS.md` |

.claude/docs/pyproject-toml.md

@@ -6,20 +6,25 @@
 pure-Python project with no compile step and no docs.
 """
 
-from pyclawd import CoverageConfig, DocsConfig, DoctorConfig, Project, QualityConfig, TestConfig
+from pyclawd import (
+    BuildConfig,
+    CoverageConfig,
+    DocsConfig,
+    DoctorConfig,
+    GoldenConfig,
+    Project,
+    QualityConfig,
+    TestConfig,
+)
 
 project = Project(
     name="pyclawd",
     conda_env=None,  # env-agnostic — runs in whatever env it's installed into
     root_markers=["pyproject.toml", "src/pyclawd/__init__.py"],
     # The pyclawd this config targets; `pyclawd doctor` WARNs on a minor mismatch.
     pyclawd_version="0.1.0",
-    # Pure-python: no compile or dist step, nothing project-specific to clean.
-    compile_cmd=[],
-    dist_cmd=[],
-    clean_targets=["build", "dist"],
-    clean_ext_dir="",
-    clean_ext_globs=[],
+    # Pure-python: no compile or dist step, just the build/dist dirs to clean.
+    build=BuildConfig(clean_targets=["build", "dist"]),
     # Default directory `pyclawd ls` lists (the code/source root) — src-layout repo.
     src_dir="src",
     # Dogfood the docs pipeline on pyclawd itself: the worked ./docs runner builds
@@ -38,10 +43,16 @@
         tests_dir="tests/",
         classname_prefix="tests.",
         integration_files=[],
-        # No slow/integration tests here, so every tier collects nearly the same suite.
-        markers={"fast": "not slow and not integration", "default": "not slow", "all": ""},
+        # golden is its own gate (`pyclawd golden`), excluded from the unit tiers.
+        markers={
+            "fast": "not slow and not integration and not golden",
+            "default": "not slow and not golden",
+            "all": "not golden",
+        },
     ),
     coverage=CoverageConfig(source=["src/pyclawd"]),
+    # Dogfood the behavior oracle: `pyclawd golden` gates `@pytest.mark.golden` tests.
+    golden=GoldenConfig(),
     doctor=DoctorConfig(
         core_deps=["typer", "rich"],
         dev_deps=["pytest", "pytest-xdist", "pytest-cov"],

.pyclawd/config.py

@@ -89,7 +89,7 @@ walks up from cwd to find `.pyclawd/config.py`.
 
 Mark slow tests `@pytest.mark.slow`, tests needing live services
 `@pytest.mark.integration`. Unmarked tests run in every tier — never mark a test
-`fast`. The fix-loop and failure taxonomy live in the `pyclawd-tests` skill.
+`fast`. The fix-loop and failure taxonomy live in the `pyclawd` skill's `references/tests.md`.
 
 ## Behavior oracle (golden)
 
@@ -124,8 +124,10 @@ in one file:
   root.
 - The `Project` model groups config: `QualityConfig` (lint/format/typecheck/check
   argv), `TestConfig` (tests dir + tier markers), `DocsConfig` (docs toolchain, or
-  `None`), `DoctorConfig` (deps/binaries/tool-files to probe), `CoverageConfig`,
-  `DescriptionConfig`. Unset optional groups disable their commands cleanly.
+  `None`), `DoctorConfig` (deps/binaries/tool-files to probe), `BuildConfig`
+  (compile/dist/clean argv, via `project.build`, or `None`), `CoverageConfig`,
+  `DescriptionConfig`, and `GoldenConfig` (the behavior-oracle wrapper, or `None`).
+  Unset optional groups disable their commands cleanly.
 - To inspect a project's setup, read its `.pyclawd/config.py` — it is the single
   source of truth for env, paths, markers, and checks.
 
@@ -170,18 +172,33 @@ the convention when adopting an existing repo.
 
 ## How you know you're done
 
-- `pyclawd check` is green (format-check, lint, typecheck, and tests all ✓).
+- `pyclawd check` is green (format-check, lint, typecheck, descriptions, and tests
+  all ✓). The `descriptions` step is the enforced gate — it passes when every
+  `DescriptionConfig`-included file (default `.py`/`.pyx`) has a one-line
+  description.
 - `pyclawd doctor` exits 0 — no FAILs.
-- `pyclawd ls --missing` is empty — every file has a one-line description.
 - Behavior is verified by tests, not just by inspection.
 
+`pyclawd ls --missing` is the broader **exploratory** view: it lists *every* repo
+file lacking a one-liner, including templates and Markdown the `descriptions` gate
+deliberately ignores. It may legitimately be non-empty even when `pyclawd check` is
+green — the gate is `descriptions`, not `ls --missing`.
+
 ---
 
 **Going deeper.** This file is the contract. For the doctrine behind it, invoke the
-Claude Code skills: **`pyclawd`** (mental model + best practices) is the umbrella;
-**`pyclawd-tests`**, **`pyclawd-quality`**, **`pyclawd-doctor`**, **`pyclawd-docs`**,
-and **`pyclawd-golden`** are the focused deep-dives; **`pyclawd-upgrade`** migrates a
-project after pyclawd itself is updated (when `pyclawd version` shows config drift).
-They are generic (not specific to any one repo) and update centrally when pyclawd is
-upgraded — which is exactly why the deep doctrine lives there and is not duplicated
-into this file.
+Claude Code skills: **`pyclawd`** is the umbrella router — a lean overview plus
+on-demand reference docs (`references/mental-model.md`, `references/tests.md`,
+`references/quality.md`, `references/docs.md`, `references/packaging.md`) that carry
+the testing, quality, docs, and packaging doctrine. Four focused standalone skills
+sit alongside it: **`pyclawd-adopt`** (adopt pyclawd into an existing repo —
+red-to-green with zero behavior regression; `pyclawd new` adopt mode now detects the
+flat-vs-`src` layout, infers `root_markers`, and prints a Phase-0 readiness report of
+what is missing, with `--scaffold-pyproject` to drop a starter ruff/mypy/pytest config),
+**`pyclawd-golden`** (the behavior
+oracle), **`pyclawd-doctor`** (diagnose a broken env), and **`pyclawd-upgrade`**
+(migrate a project *after* a pyclawd version bump, when `pyclawd version` shows config
+drift — the upgrade counterpart to `pyclawd-adopt`'s first-time onboarding). They are
+generic (not
+specific to any one repo) and update centrally when pyclawd is upgraded — which is
+exactly why the deep doctrine lives there and is not duplicated into this file.

AGENTS.md

@@ -50,6 +50,19 @@ the running pyclawd has a different `major.minor`, pointing here for what change
   an `(editable)` marker) so you can tell which pyclawd is driving a project.
 
 ### Changed
+- **Build fields grouped into a new `BuildConfig` (breaking config change).** The
+  five loose top-level build knobs — `compile_cmd`, `dist_cmd`, `clean_targets`,
+  `clean_ext_dir`, `clean_ext_globs` — have moved off `Project` into a new frozen
+  `BuildConfig` dataclass, reached via `Project.build` (a `BuildConfig | None`,
+  default `None`). Migrate `Project(compile_cmd=…, clean_targets=…, …)` to
+  `Project(build=BuildConfig(compile_cmd=…, clean_targets=…, …))`; `BuildConfig` is
+  exported from `pyclawd` alongside the other `*Config`s. When `Project.build` is
+  `None` the `compile` / `dist` / `clean` commands self-report and exit 2 (the same
+  0/2 contract as before). This is a clean break — there is no compatibility shim.
+- **Scaffolded `check_sequence` now includes the `descriptions` step** — `pyclawd
+  new` writes `["format-check", "lint", "typecheck", "descriptions", "test"]`, so a
+  freshly scaffolded project enforces the file-description code-map doctrine out of
+  the box, matching pyclawd's own dogfood config.
 - **Test taxonomy aligned to two marker axes** — *speed* (`slow`) and *scope*
   (`integration`) — with a canonical tier ladder (`fast = not slow and not
   integration`, `default = not slow`, `all =` everything). The `long` marker is

CHANGELOG.md

@@ -41,7 +41,9 @@ pyclawd new myproj      # scaffold a fresh best-practice project
 cd myproj
 
 pyclawd new             # …or, inside an existing repo, ADOPT pyclawd
-                        # (writes ./.pyclawd/config.py, inferring sensible defaults)
+                        # (detects flat-vs-src layout, infers root_markers, writes
+                        #  ./.pyclawd/config.py, and prints a Phase-0 readiness report)
+pyclawd new --scaffold-pyproject   # also drop a starter ruff/mypy/pytest config
 
 pyclawd doctor          # health-check the dev env
 pyclawd test fast       # <30s smoke tier
@@ -62,7 +64,7 @@ pyclawd check           # the full quality gate
 | Aggregate quality gate | `pyclawd check` |
 | Build / dist / clean | `pyclawd compile` · `pyclawd dist` · `pyclawd clean [--ext]` |
 | Docs (if configured) | `pyclawd docs build\|run\|render\|serve\|status\|failures\|exec <page>` |
-| Scaffold / adopt | `pyclawd new <name>` · `pyclawd new` |
+| Scaffold / adopt | `pyclawd new <name>` · `pyclawd new` (adopt: detects layout + Phase-0 readiness) · `pyclawd new --scaffold-pyproject` |
 | Code map (file → description) | `pyclawd ls [DIR]` · `pyclawd ls --missing` · `pyclawd ls --py` |
 | Repo root / version | `pyclawd root` · `pyclawd version` |
 
@@ -103,7 +105,7 @@ project = Project(
         format_cmd=["ruff", "format"],
         format_check_cmd=["ruff", "format", "--check"],
         typecheck_cmd=["mypy", "src"],
-        check_sequence=["format-check", "lint", "typecheck", "test"],
+        check_sequence=["format-check", "lint", "typecheck", "descriptions", "test"],
     ),
     test=TestConfig(                      # tests dir + tier marker expressions
         tests_dir="tests/",
@@ -121,7 +123,7 @@ project = Project(
 )
 ```
 
-The nested configs (`QualityConfig`, `TestConfig`, `DocsConfig`, `DoctorConfig`) each gate their command group: leave one out and those commands self-report as unconfigured instead of crashing.
+The nested configs (`QualityConfig`, `TestConfig`, `DocsConfig`, `DoctorConfig`, plus the optional `BuildConfig` via `project.build` and `GoldenConfig`) each gate their command group: leave one out and those commands self-report as unconfigured instead of crashing (e.g. `build=None` → `compile`/`dist`/`clean` exit 2).
 
 Two worked examples bracket the range:
 
@@ -132,7 +134,7 @@ Copy whichever is closer and delete what you don't use.
 
 ## Skills
 
-pyclawd ships agent-facing slash-command skills under [`skills/`](skills/): the start-here umbrella `pyclawd` skill plus the focused `pyclawd-doctor`, `pyclawd-tests`, `pyclawd-quality`, `pyclawd-golden`, `pyclawd-docs`, and `pyclawd-upgrade`. They are thin wrappers over the CLI. Copy or symlink the `pyclawd-*` directories into a project's `.claude/skills/` — see [`skills/README.md`](skills/README.md). Agent doctrine for any pyclawd project lives in [`AGENTS.md`](AGENTS.md).
+pyclawd ships agent-facing slash-command skills under [`skills/`](skills/): the start-here umbrella `pyclawd` router skill (a lean overview plus on-demand `references/{mental-model,tests,quality,docs,packaging}.md` carrying the testing/quality/docs/packaging doctrine) plus the four focused standalone skills `pyclawd-adopt` (adopt pyclawd into an existing repo — red-to-green with zero behavior regression), `pyclawd-golden`, `pyclawd-doctor`, and `pyclawd-upgrade` (migrate *after* a pyclawd version bump — the upgrade counterpart to first-time adoption). They are thin wrappers over the CLI. `pyclawd skills install` copies (or symlinks) the `pyclawd-*` directories into your **user-scope** `~/.claude/skills/` by default — they are generic, so they are shared across every project rather than committed per-repo — see [`skills/README.md`](skills/README.md). Agent doctrine for any pyclawd project lives in [`AGENTS.md`](AGENTS.md).
 
 ## Status
 

README.md

@@ -20,6 +20,7 @@
     FAIL,
     OK,
     WARN,
+    BuildConfig,
     Check,
     DocsConfig,
     DoctorConfig,
@@ -79,15 +80,19 @@ def acme_doctor_checks() -> list[Check]:
     #   ["conda", "run", "-n", "acme-dev", "python"]  -> one pyclawd, many conda envs
     #   ["uv", "run", "python"]                  -> uv-managed env
     python_cmd=[],
-    # --- build / dist / clean (omit any for a pure-Python project) --------- #
-    # Args handed to the dev Python: `pyclawd compile` → `python setup.py build_ext ...`.
-    compile_cmd=["setup.py", "build_ext", "--inplace"],
-    dist_cmd=["setup.py", "sdist"],
-    # `pyclawd clean` removes these root-relative paths (guarded to stay in-repo).
-    clean_targets=["build", "dist", "acme.egg-info"],
-    # `pyclawd clean --ext` removes these globs under this dir (forces a recompile).
-    clean_ext_dir="acme/functions/compiled",
-    clean_ext_globs=["*.c", "*.cpp", "*.so", "*.html"],
+    # --- build / dist / clean (omit `build` entirely for a pure-Python project) #
+    # All build knobs live in one nested BuildConfig, reached via `project.build`.
+    # When `build` is None the compile/dist/clean commands self-report and exit 2.
+    build=BuildConfig(
+        # Args handed to the dev Python: `pyclawd compile` → `python setup.py build_ext ...`.
+        compile_cmd=["setup.py", "build_ext", "--inplace"],
+        dist_cmd=["setup.py", "sdist"],
+        # `pyclawd clean` removes these root-relative paths (guarded to stay in-repo).
+        clean_targets=["build", "dist", "acme.egg-info"],
+        # `pyclawd clean --ext` removes these globs under this dir (forces a recompile).
+        clean_ext_dir="acme/functions/compiled",
+        clean_ext_globs=["*.c", "*.cpp", "*.so", "*.html"],
+    ),
     # Default directory `pyclawd ls` lists (the code/source root), relative to root.
     src_dir="acme",
     # --- code quality (lint / format / typecheck / check) ----------------- #

examples/config.reference.py

@@ -1,5 +1,5 @@
 [build-system]
-requires = ["hatchling"]
+requires = ["hatchling>=1.27"]  # >=1.27 for PEP 639 license-expression support
 build-backend = "hatchling.build"
 
 [project]
@@ -11,12 +11,12 @@ description = "A project-generic, config-driven dev-task runner for Python proje
 authors = [{ name = "Julian Blank", email = "blankjul@outlook.com" }]
 readme = "README.md"
 license = "Apache-2.0"
+license-files = ["LICENSE*"]
 requires-python = ">=3.10"
 keywords = ["developer-tools", "task-runner", "cli", "testing", "documentation"]
 classifiers = [
     "Development Status :: 4 - Beta",
     "Intended Audience :: Developers",
-    "License :: OSI Approved :: Apache Software License",
     "Programming Language :: Python :: 3.10",
     "Programming Language :: Python :: 3.11",
     "Programming Language :: Python :: 3.12",
@@ -61,6 +61,7 @@ path = "src/pyclawd/__init__.py"
 
 [tool.hatch.build.targets.wheel]
 packages = ["src/pyclawd"]
+include = ["src/pyclawd/py.typed"]  # PEP 561 — ship the typed marker (we're a typed lib)
 
 # Ship the CHANGELOG inside the wheel so `pyclawd changelog` works from any
 # pip-installed pyclawd (PyPI included) — the agent-driven upgrade flow reads it to
@@ -88,7 +89,7 @@ extend-exclude = ["docs"]
 # under ruff's unstable `preview`, so selecting them is a silent no-op (and emits a
 # warning). The Google docstring *convention* below + the `D` rules are the gate;
 # the no-types / Args-Returns style is upheld by review and the `pyclawd ls` map.
-select = ["E", "F", "I", "B", "UP", "SIM", "C4", "RUF", "PGH", "D"]
+select = ["E", "W", "F", "I", "B", "UP", "SIM", "C4", "RUF", "PGH", "D"]
 # `compile` and `format` shadow builtins by design — they are intentional
 # command names (`pyclawd compile`, `pyclawd format`).
 
@@ -109,13 +110,12 @@ extend-immutable-calls = ["typer.Option", "typer.Argument"]
 # Non-strict to start: wire up type-checking without drowning in noise.
 python_version = "3.10"
 files = ["src"]
-ignore_missing_imports = true
 warn_unused_configs = true
 check_untyped_defs = true
 # Needed for `pyclawd typecheck/check <file>` — prevents "Duplicate module"
 # errors when mypy resolves a single source file by path instead of by name.
 explicit_package_bases = true
-mypy_path = "."
+mypy_path = "src"
 
 # numpy is an optional dep used only via `Any` (lazy import in pyclawd.golden).
 # Don't follow into its stubs: with python_version pinned to 3.10, mypy rejects the
@@ -125,3 +125,10 @@ mypy_path = "."
 module = ["numpy", "numpy.*"]
 follow_imports = "skip"
 follow_imports_for_stubs = true
+
+# jupyter-cache ships no type stubs / py.typed; docs.py imports it lazily for the
+# `pyclawd docs failures` path. Scope the missing-import allowance to it (rather
+# than a global ignore_missing_imports) so every other unstubbed import still errors.
+[[tool.mypy.overrides]]
+module = ["jupyter_cache.*"]
+ignore_missing_imports = true