-
Notifications
You must be signed in to change notification settings - Fork 9.4k
generate integrations reference from catalog #2563
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
DyanGalih
wants to merge
35
commits into
github:main
Choose a base branch
from
DyanGalih:002-generate-integrations-docs
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
35 commits
Select commit
Hold shift + click to select a range
848cb96
docs: generate integrations reference from catalog
DyanGalih 2a72b53
refactor: integrate table rendering into specify integration search -…
DyanGalih df40ef8
fix: address Copilot review feedback on catalog_docs and integration_…
DyanGalih 73602ca
fix: add sync test, INTEGRATIONS_REFERENCE_PATH constant, and fix naming
DyanGalih c621732
revert: restore docs/reference/integrations.md to upstream/main; remo…
DyanGalih b4bd56f
fix: remove dead INTEGRATIONS_REFERENCE_PATH, drop URL-length padding…
DyanGalih 7caace8
fix: send --markdown warnings/errors to stderr, rename test for clarity
DyanGalih 01be38f
fix: detect stale doc-map keys, test _render_cell escaping, strengthe…
DyanGalih 70afa5c
refactor: promote _render_cell to public render_cell function
DyanGalih 1c5af18
test: mock registry and doc maps to avoid brittle live registry coupling
DyanGalih b64cc3b
refactor: flatten patches, remove unused imports, fix trailing whites…
DyanGalih c351766
refactor: make validation non-fatal, fix context manager syntax, add …
DyanGalih e9c4bc4
fix: improve docstring clarity, test robustness, and exception handling
DyanGalih 7d1a401
fix: improve test assertions, disable warnings by default, enhance ex…
DyanGalih dd32eb1
fix: make CLI tests deterministic and improve config access resilience
DyanGalih d9bda8a
fix: remove extra blank line, add stale keys validation, add regressi…
DyanGalih f11bca9
Fix 5 remaining feedback items:
DyanGalih 4dab389
address all outstanding copilot review feedback on PR 2563
DyanGalih be4b7a6
Address Copilot feedback: escape URLs in markdown links, deduplicate …
DyanGalih 28e68d6
Address 3 new Copilot feedback: add URL escaping test, fix parse_firs…
DyanGalih 2b27eed
Address 3 new Copilot feedback: escape id field, remove unused alias,…
DyanGalih 295dcb6
Address 3 new Copilot feedback: fix comment name, include all integra…
DyanGalih 826fbf5
Fix architectural issue: escape raw fields before composing Markdown …
DyanGalih 77c7327
Deduplicate _escape_url_for_markdown_link and add URL escaping test
DyanGalih 34fff7c
Address 4 new Copilot feedback: add trailing newline, fix test helper…
DyanGalih 60fc926
Address 4 new Copilot feedback: make escape function public, fix erro…
DyanGalih e275568
Update error message in test_missing_catalog_file for clarity
DyanGalih 47b3982
Remove obsolete integrations sync test
DyanGalih 6ecc914
keep integrations docs in sync
DyanGalih c1f078b
Potential fix for pull request finding
DyanGalih 538fd21
Potential fix for pull request finding
DyanGalih 8141018
Potential fix for pull request finding
DyanGalih c85a00b
Potential fix for pull request finding
DyanGalih d16c26f
fix: address remaining integrations docs feedback
DyanGalih 37fee9a
fix: address remaining PR 2563 feedback
DyanGalih File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,210 @@ | ||
| """Helpers for rendering the built-in integrations reference table.""" | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| from typing import Any | ||
|
|
||
| from ._assets import _repo_root | ||
|
|
||
|
|
||
| ROOT_DIR = _repo_root() | ||
| INTEGRATIONS_REFERENCE_PATH = ROOT_DIR / "docs" / "reference" / "integrations.md" | ||
|
|
||
|
|
||
| INTEGRATION_DOC_URLS: dict[str, str | None] = { | ||
| "amp": "https://ampcode.com/", | ||
| "agy": "https://antigravity.google/", | ||
| "auggie": "https://docs.augmentcode.com/cli/overview", | ||
| "bob": "https://www.ibm.com/products/bob", | ||
| "claude": "https://www.anthropic.com/claude-code", | ||
| "codebuddy": "https://www.codebuddy.ai/cli", | ||
| "codex": "https://github.com/openai/codex", | ||
| "copilot": "https://code.visualstudio.com/", | ||
| "cursor-agent": "https://cursor.sh/", | ||
| "devin": "https://cli.devin.ai/docs", | ||
| "forge": "https://forgecode.dev/", | ||
| "gemini": "https://github.com/google-gemini/gemini-cli", | ||
| "generic": None, | ||
| "goose": "https://block.github.io/goose/", | ||
| "iflow": "https://docs.iflow.cn/en/cli/quickstart", | ||
| "junie": "https://junie.jetbrains.com/", | ||
| "kilocode": "https://github.com/Kilo-Org/kilocode", | ||
| "kimi": "https://code.kimi.com/", | ||
| "kiro-cli": "https://kiro.dev/docs/cli/", | ||
| "lingma": "https://lingma.aliyun.com/", | ||
| "opencode": "https://opencode.ai/", | ||
| "pi": "https://pi.dev", | ||
| "qodercli": "https://qoder.com/cli", | ||
| "qwen": "https://github.com/QwenLM/qwen-code", | ||
| "roo": "https://roocode.com/", | ||
| "shai": "https://github.com/ovh/shai", | ||
| "tabnine": "https://docs.tabnine.com/main/getting-started/tabnine-cli", | ||
| "trae": "https://www.trae.ai/", | ||
| "vibe": "https://github.com/mistralai/mistral-vibe", | ||
| "windsurf": "https://windsurf.com/", | ||
| } | ||
|
|
||
| INTEGRATION_LABEL_OVERRIDES: dict[str, str] = { | ||
| "agy": "Antigravity (agy)", | ||
| "codebuddy": "CodeBuddy CLI", | ||
| "generic": "Generic", | ||
| "shai": "SHAI (OVHcloud)", | ||
| } | ||
|
|
||
| INTEGRATION_NOTES: dict[str, str] = { | ||
| "agy": "Skills-based integration; skills are installed automatically", | ||
| "claude": "Skills-based integration; installs skills in `.claude/skills`", | ||
| "codex": ( | ||
| "Skills-based integration; installs skills into `.agents/skills` " | ||
| "and invokes them as `$speckit-<command>`" | ||
| ), | ||
| "bob": "IDE-based agent", | ||
| "devin": ( | ||
| "Skills-based integration; installs skills into `.devin/skills/` " | ||
| "and invokes them as `/speckit-<command>`" | ||
| ), | ||
| "goose": "Uses YAML recipe format in `.goose/recipes/`", | ||
| "kimi": ( | ||
| "Skills-based integration; supports `--migrate-legacy` " | ||
| "for dotted→hyphenated directory migration" | ||
| ), | ||
| "kiro-cli": ( | ||
| "Kiro CLI does not substitute `$ARGUMENTS` in file-based prompts, " | ||
| "so Spec Kit ships a prose fallback at render time " | ||
| "(see [Manage prompts](https://kiro.dev/docs/cli/chat/manage-prompts/) " | ||
| "and issue [#1926](https://github.com/github/spec-kit/issues/1926)). " | ||
| "Alias: `--integration kiro`" | ||
| ), | ||
| "lingma": "Skills-based integration; skills are installed automatically", | ||
| "pi": ( | ||
| "Pi doesn't have MCP support out of the box, so `taskstoissues` " | ||
| "won't work as intended. MCP support can be added via " | ||
| "[extensions](https://github.com/badlogic/pi-mono/tree/main/" | ||
| "packages/coding-agent#extensions)" | ||
| ), | ||
| "generic": ( | ||
| "Bring your own agent — use `--integration generic " | ||
| "--integration-options=\"--commands-dir <path>\"` " | ||
| "for AI coding agents not listed above" | ||
| ), | ||
| "trae": "Skills-based integration; skills are installed automatically", | ||
| } | ||
|
|
||
|
|
||
| def render_cell(value: str) -> str: | ||
| r"""Escape markdown special characters (pipes) and normalize newlines to spaces. | ||
|
|
||
| This ensures table cells remain valid markdown even if they contain | ||
| pipes (escaped as \|) or carriage returns (normalized to spaces). | ||
| """ | ||
| value = value.replace("\r\n", " ").replace("\r", " ").replace("\n", " ") | ||
| return value.replace("|", "\\|") | ||
|
|
||
|
|
||
| def escape_url_for_markdown_link(url: str) -> str: | ||
| """Escape characters that can break Markdown link syntax. | ||
|
|
||
| Escapes `)` and `|` which can terminate or corrupt the link destination. | ||
| """ | ||
| return url.replace(")", "\\)").replace("|", "\\|") | ||
|
|
||
|
|
||
| def escape_markdown_link_text(text: str) -> str: | ||
| """Escape characters that can break Markdown link text.""" | ||
| return text.replace("[", "\\[").replace("]", "\\]") | ||
|
|
||
|
|
||
| def _get_integration_registry() -> dict[str, Any]: | ||
| from specify_cli.integrations import INTEGRATION_REGISTRY | ||
|
|
||
| return INTEGRATION_REGISTRY | ||
|
|
||
|
|
||
| def list_integrations_for_docs( | ||
| warn_on_missing: bool = False, | ||
| warn_on_extra: bool = False, | ||
| ) -> list[tuple[str, str, str | None, str]]: | ||
| """List all integrations with their documentation URLs and notes. | ||
|
|
||
| Returns all integrations in the registry. Missing entries in INTEGRATION_DOC_URLS | ||
| default to None; if `warn_on_missing` is True, emits a warning for these. | ||
| If `warn_on_extra` is True, emits a warning for stale keys in the doc maps that | ||
| are no longer in the registry. Missing notes entries default to empty string. | ||
| """ | ||
| import warnings | ||
|
|
||
| registry = _get_integration_registry() | ||
|
mnriem marked this conversation as resolved.
|
||
| registry_keys = set(registry) | ||
|
|
||
| # Warn if there are integrations missing from INTEGRATION_DOC_URLS (when enabled) | ||
| missing = sorted(registry_keys - set(INTEGRATION_DOC_URLS)) | ||
| if missing and warn_on_missing: | ||
| warnings.warn( | ||
| f"Integration(s) missing from INTEGRATION_DOC_URLS: " | ||
| f"{', '.join(missing)}. They will be included in the docs table " | ||
| "without documentation links. Add them to INTEGRATION_DOC_URLS in " | ||
| "catalog_docs.py if a link should be available.", | ||
| stacklevel=2 | ||
| ) | ||
|
|
||
| # Warn if there are stale keys in doc maps not in the registry (when enabled) | ||
| if warn_on_extra: | ||
| extra_in_urls = sorted(set(INTEGRATION_DOC_URLS) - registry_keys) | ||
| extra_in_labels = sorted( | ||
| set(INTEGRATION_LABEL_OVERRIDES) - registry_keys | ||
| ) | ||
| extra_in_notes = sorted(set(INTEGRATION_NOTES) - registry_keys) | ||
| extra_keys = extra_in_urls or extra_in_labels or extra_in_notes | ||
| if extra_keys: | ||
| stale_keys = sorted( | ||
| set(extra_in_urls + extra_in_labels + extra_in_notes) | ||
| ) | ||
| warnings.warn( | ||
| f"Stale key(s) found in doc maps (no longer in registry): " | ||
| f"{', '.join(stale_keys)}. Consider removing them from " | ||
| "INTEGRATION_DOC_URLS, INTEGRATION_LABEL_OVERRIDES, and " | ||
| "INTEGRATION_NOTES.", | ||
| stacklevel=2 | ||
| ) | ||
|
DyanGalih marked this conversation as resolved.
DyanGalih marked this conversation as resolved.
|
||
|
|
||
| rows: list[tuple[str, str, str | None, str]] = [] | ||
|
|
||
| for key, integration in registry.items(): | ||
| config = getattr(integration, "config", {}) | ||
| if not isinstance(config, dict): | ||
| config = {} | ||
| label = INTEGRATION_LABEL_OVERRIDES.get(key, str(config.get("name") or key)) | ||
| url = INTEGRATION_DOC_URLS.get(key) # None if not in map | ||
| notes = INTEGRATION_NOTES.get(key, "") | ||
| rows.append((key, label, url, notes)) | ||
|
|
||
|
mnriem marked this conversation as resolved.
|
||
| return sorted(rows, key=lambda r: r[0]) | ||
|
|
||
|
|
||
| def render_integrations_table() -> str: | ||
| """Render the built-in integrations reference table as markdown.""" | ||
| table_rows: list[list[str]] = [] | ||
|
|
||
| for key, label, url, notes in list_integrations_for_docs(): | ||
| # Escape raw field values *before* composing Markdown syntax so that | ||
| # a pipe inside a label or notes doesn't break a link target. | ||
| safe_label = escape_markdown_link_text(render_cell(label)) | ||
| safe_notes = render_cell(notes) | ||
| safe_url = escape_url_for_markdown_link(url) if url else None | ||
| agent = ( | ||
| f"[{safe_label}]({safe_url})" | ||
| if safe_url | ||
| else safe_label | ||
| ) | ||
| table_rows.append([agent, f"`{render_cell(key)}`", safe_notes]) | ||
|
|
||
| headers = ("Agent", "Key", "Notes") | ||
|
|
||
| def render_row(values: list[str]) -> str: | ||
| # Values are already escaped; do not re-apply render_cell here. | ||
| return "| " + " | ".join(values) + " |" | ||
|
|
||
| separator = "| " + " | ".join("---" for _ in headers) + " |" | ||
| lines = [render_row(list(headers)), separator] | ||
| lines.extend(render_row(row) for row in table_rows) | ||
| return "\n".join(lines) + "\n" | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,114 @@ | ||
| """Helpers for rendering the community extensions reference table.""" | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
|
DyanGalih marked this conversation as resolved.
|
||
| import json | ||
| from pathlib import Path | ||
| from typing import Any | ||
|
|
||
| from ._assets import _repo_root | ||
| from .catalog_docs import ( | ||
| escape_markdown_link_text, | ||
| escape_url_for_markdown_link, | ||
| render_cell, | ||
| ) | ||
|
|
||
|
|
||
| ROOT_DIR = _repo_root() | ||
| COMMUNITY_CATALOG_PATH = ROOT_DIR / "extensions" / "catalog.community.json" | ||
|
|
||
|
DyanGalih marked this conversation as resolved.
|
||
|
|
||
| def _format_tags(tags: Any) -> str: | ||
| if not isinstance(tags, list) or not tags: | ||
| return "—" | ||
| # Clean first, then filter: a tag of " | " would pass str(tag).strip() but produce | ||
| # an empty backtick span after pipe removal, so filter on the cleaned value. | ||
| cleaned = [ | ||
| f"`{c}`" | ||
| for tag in tags | ||
| if (c := str(tag).replace("|", "").replace("\r\n", " ").replace("\r", " ").replace("\n", " ").strip()) | ||
| ] | ||
| return ", ".join(cleaned) if cleaned else "—" | ||
|
|
||
|
|
||
| def list_community_extensions( | ||
| path: Path = COMMUNITY_CATALOG_PATH, | ||
| ) -> list[dict[str, Any]]: | ||
| """Return community extensions sorted alphabetically by name then ID.""" | ||
| if not path.exists(): | ||
| if path == COMMUNITY_CATALOG_PATH: | ||
| message = ( | ||
| f"Community catalog not found at {path}. " | ||
| "Ensure the repository checkout includes the extensions/ directory." | ||
| ) | ||
| else: | ||
| message = ( | ||
| f"Community catalog not found at {path}. " | ||
| "Provide path= to a valid community catalog JSON file." | ||
| ) | ||
| raise FileNotFoundError(message) | ||
| data = json.loads(path.read_text(encoding="utf-8")) | ||
| if not isinstance(data, dict): | ||
| raise ValueError(f"Expected {path} to contain a JSON object") | ||
| extensions = data.get("extensions") | ||
| if not isinstance(extensions, dict): | ||
| raise ValueError(f"Expected {path} to contain an 'extensions' object") | ||
|
|
||
| rows: list[dict[str, Any]] = [] | ||
| for ext_id, ext in extensions.items(): | ||
| if not isinstance(ext, dict): | ||
| raise ValueError(f"Community extension {ext_id!r} must be a mapping") | ||
| rows.append( | ||
| { | ||
| "name": str(ext.get("name") or ext_id), | ||
| "id": str(ext.get("id") or ext_id), | ||
| "description": str(ext.get("description") or ""), | ||
| "tags": ext.get("tags") or [], | ||
| "verified": "Yes" if bool(ext.get("verified")) else "No", | ||
| "repository": str(ext.get("repository") or "").strip(), | ||
| } | ||
| ) | ||
|
|
||
| return sorted( | ||
| rows, | ||
| key=lambda row: (row["name"].casefold(), row["id"].casefold()), | ||
| ) | ||
|
|
||
|
|
||
| def render_community_extensions_table(path: Path = COMMUNITY_CATALOG_PATH) -> str: | ||
| """Render the community extensions table from catalog.community.json.""" | ||
| rows = list_community_extensions(path=path) | ||
| if not rows: | ||
| raise ValueError("Community catalog has no extensions") | ||
|
|
||
| table_rows: list[list[str]] = [] | ||
| for row in rows: | ||
| # Escape raw field values *before* composing Markdown syntax so that | ||
| # a pipe inside a name or description doesn't break a link target. | ||
| safe_name = escape_markdown_link_text(render_cell(row["name"])) | ||
| repository = row["repository"] | ||
| if repository: | ||
| safe_repo = escape_url_for_markdown_link(repository) | ||
| link = f"[{safe_name}]({safe_repo})" | ||
| else: | ||
| link = safe_name | ||
| table_rows.append( | ||
| [ | ||
| link, | ||
| f"`{render_cell(row['id'])}`", | ||
| render_cell(row["description"]), | ||
| _format_tags(row["tags"]), | ||
| row["verified"], | ||
| ] | ||
| ) | ||
|
DyanGalih marked this conversation as resolved.
DyanGalih marked this conversation as resolved.
|
||
|
|
||
| headers = ("Extension", "ID", "Description", "Tags", "Verified") | ||
|
|
||
| def render_row(values: list[str]) -> str: | ||
| # Values are already escaped; do not re-apply render_cell here. | ||
| return "| " + " | ".join(values) + " |" | ||
|
|
||
| separator = "| " + " | ".join("---" for _ in headers) + " |" | ||
| lines = [render_row(list(headers)), separator] | ||
| lines.extend(render_row(row) for row in table_rows) | ||
| return "\n".join(lines) + "\n" | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.