Skip to content

Commit d0af4bc

Browse files
authored
Document provider foundation completion (#126)
1 parent 6e521ec commit d0af4bc

2 files changed

Lines changed: 93 additions & 20 deletions

File tree

docs/src/provider-platform-roadmap.md

Lines changed: 39 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -348,6 +348,25 @@ explicitly marks them safe for local-only use.
348348

349349
## Workstream A: Provider Platform Foundation
350350

351+
Track status: complete for [#47](https://github.com/AbdelStark/worldforge/issues/47).
352+
353+
Completion signals:
354+
355+
- Provider promotion rules cover `scaffold`, `experimental`, `beta`, and `stable` in the
356+
[Provider Authoring Guide](./provider-authoring-guide.md), including profile metadata,
357+
generated catalog updates, validation commands, and the current provider classification table.
358+
- Runtime manifests are packaged under `src/worldforge/providers/runtime_manifests/`, validated by
359+
tests, linked from provider docs, and used by health/configuration summaries without installing
360+
optional runtimes.
361+
- Capability conformance helpers exist for `predict`, `generate`, `transfer`, `reason`, `embed`,
362+
`score`, and `policy`, plus provider-event redaction checks for safe fixture and injected-runtime
363+
coverage.
364+
- Optional live-smoke commands can write sanitized `run_manifest.json` evidence with command,
365+
package, provider profile, capability, runtime manifest, input digest, event count, result digest,
366+
and artifact-path summaries.
367+
- Runtime pytest profiles keep default checkout validation deterministic while prepared hosts can
368+
opt into `live`, `network`, `credentialed`, `gpu`, `robotics`, and provider-specific test runs.
369+
351370
### WF-PROV-001: Provider Promotion Gate
352371

353372
Type: provider platform
@@ -368,10 +387,10 @@ Scope:
368387

369388
Acceptance criteria:
370389

371-
- [ ] Promotion rules cover all current statuses: `scaffold`, `experimental`, `beta`, `stable`.
372-
- [ ] Rules explain when to change provider profile metadata and generated catalog docs.
373-
- [ ] Rules include exact local validation commands.
374-
- [ ] Existing providers are classified against the new checklist without changing behavior.
390+
- [x] Promotion rules cover all current statuses: `scaffold`, `experimental`, `beta`, `stable`.
391+
- [x] Rules explain when to change provider profile metadata and generated catalog docs.
392+
- [x] Rules include exact local validation commands.
393+
- [x] Existing providers are classified against the new checklist without changing behavior.
375394

376395
Validation:
377396

@@ -400,10 +419,10 @@ Scope:
400419

401420
Acceptance criteria:
402421

403-
- [ ] Manifest schema is documented and validated by tests.
404-
- [ ] Manifests exist for `leworldmodel`, `lerobot`, `gr00t`, `cosmos`, and `runway`.
405-
- [ ] Missing optional dependencies produce actionable health messages using manifest data.
406-
- [ ] Docs link from provider pages to the relevant manifest.
422+
- [x] Manifest schema is documented and validated by tests.
423+
- [x] Manifests exist for `leworldmodel`, `lerobot`, `gr00t`, `cosmos`, and `runway`.
424+
- [x] Missing optional dependencies produce actionable health messages using manifest data.
425+
- [x] Docs link from provider pages to the relevant manifest.
407426

408427
Validation:
409428

@@ -431,10 +450,10 @@ Scope:
431450

432451
Acceptance criteria:
433452

434-
- [ ] Each capability has a reusable conformance helper.
435-
- [ ] Current provider tests call the helpers where applicable.
436-
- [ ] The helpers can run against injected runtimes without credentials.
437-
- [ ] The helpers do not use bare Python `assert` statements.
453+
- [x] Each capability has a reusable conformance helper.
454+
- [x] Current provider tests call the helpers where applicable.
455+
- [x] The helpers can run against injected runtimes without credentials.
456+
- [x] The helpers do not use bare Python `assert` statements.
438457

439458
Validation:
440459

@@ -462,10 +481,10 @@ Scope:
462481

463482
Acceptance criteria:
464483

465-
- [ ] Live smoke commands can emit `run_manifest.json`.
466-
- [ ] Manifest validation rejects secret-like metadata.
467-
- [ ] Robotics showcase manifest links policy, score, replay, and report artifacts.
468-
- [ ] Docs explain which artifacts are safe to attach to GitHub issues.
484+
- [x] Live smoke commands can emit `run_manifest.json`.
485+
- [x] Manifest validation rejects secret-like metadata.
486+
- [x] Robotics showcase manifest links policy, score, replay, and report artifacts.
487+
- [x] Docs explain which artifacts are safe to attach to GitHub issues.
469488

470489
Validation:
471490

@@ -493,11 +512,11 @@ Scope:
493512

494513
Acceptance criteria:
495514

496-
- [ ] Live tests skip with clear reasons when runtime/env is missing.
497-
- [ ] Prepared-host commands are documented for each real provider.
498-
- [ ] Default CI remains deterministic and does not require credentials, network calls, GPUs, or
515+
- [x] Live tests skip with clear reasons when runtime/env is missing.
516+
- [x] Prepared-host commands are documented for each real provider.
517+
- [x] Default CI remains deterministic and does not require credentials, network calls, GPUs, or
499518
downloaded checkpoints.
500-
- [ ] Live-smoke evidence can be attached to release notes or provider issues.
519+
- [x] Live-smoke evidence can be attached to release notes or provider issues.
501520

502521
Validation:
503522

tests/test_docs_site.py

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -156,6 +156,60 @@ def test_observability_roadmap_tracker_records_completion() -> None:
156156
assert "request_id" in service_host
157157

158158

159+
def test_provider_platform_foundation_roadmap_tracker_records_completion() -> None:
160+
roadmap = (ROOT / "docs/src/provider-platform-roadmap.md").read_text(encoding="utf-8")
161+
authoring = (ROOT / "docs/src/provider-authoring-guide.md").read_text(encoding="utf-8")
162+
provider_index = (ROOT / "docs/src/providers/README.md").read_text(encoding="utf-8")
163+
operations = (ROOT / "docs/src/operations.md").read_text(encoding="utf-8")
164+
playbooks = (ROOT / "docs/src/playbooks.md").read_text(encoding="utf-8")
165+
166+
runtime_manifests = {
167+
path.stem for path in (ROOT / "src/worldforge/providers/runtime_manifests").glob("*.json")
168+
}
169+
170+
assert "Track status: complete for [#47]" in roadmap
171+
for child in ("WF-PROV-001", "WF-PROV-002", "WF-PROV-003", "WF-PROV-004", "WF-PROV-005"):
172+
assert child in roadmap
173+
174+
for completed_criterion in (
175+
"Promotion rules cover all current statuses: `scaffold`, `experimental`, `beta`, `stable`.",
176+
"Rules explain when to change provider profile metadata and generated catalog docs.",
177+
"Manifest schema is documented and validated by tests.",
178+
"Manifests exist for `leworldmodel`, `lerobot`, `gr00t`, `cosmos`, and `runway`.",
179+
"Each capability has a reusable conformance helper.",
180+
"The helpers do not use bare Python `assert` statements.",
181+
"Live smoke commands can emit `run_manifest.json`.",
182+
"Manifest validation rejects secret-like metadata.",
183+
"Live tests skip with clear reasons when runtime/env is missing.",
184+
"Prepared-host commands are documented for each real provider.",
185+
"Live-smoke evidence can be attached to release notes or provider issues.",
186+
):
187+
assert f"- [x] {completed_criterion}" in roadmap
188+
189+
for status in ("`scaffold`", "`experimental`", "`beta`", "`stable`"):
190+
assert status in authoring
191+
192+
for helper in (
193+
"assert_predict_conformance",
194+
"assert_generate_conformance",
195+
"assert_transfer_conformance",
196+
"assert_reason_conformance",
197+
"assert_embed_conformance",
198+
"assert_score_conformance",
199+
"assert_policy_conformance",
200+
"assert_provider_events_conform",
201+
):
202+
assert helper in authoring
203+
204+
assert {"leworldmodel", "lerobot", "gr00t", "cosmos", "runway"} <= runtime_manifests
205+
assert "`src/worldforge/providers/runtime_manifests/`" in provider_index
206+
assert "Optional live smoke entrypoints accept `--run-manifest <path>`" in provider_index
207+
208+
for marker in ("live", "network", "credentialed", "gpu", "robotics", "provider_profile"):
209+
assert marker in operations
210+
assert marker in playbooks
211+
212+
159213
def test_reference_host_roadmap_tracker_records_completion() -> None:
160214
roadmap = (ROOT / "docs/src/provider-platform-roadmap.md").read_text(encoding="utf-8")
161215
examples = (ROOT / "docs/src/examples.md").read_text(encoding="utf-8")

0 commit comments

Comments
 (0)