Skip to content

--skip-fts / GITNEXUS_SKIP_FTS=1 to skip FTS index creation for consumers that bring their own retrieval #3091

Description

@BecauseY

Area

gitnexus (CLI / core / indexing / MCP server)

Problem or opportunity

gitnexus analyze always loads the FTS extension and builds the BM25 search indexes (buildSearchIndexesOrDegrade in gitnexus/src/core/run-analyze.ts). The only way to not pay for it is for the extension to be unavailable, which is logged as a degradation. Consumers that read the graph through Cypher / the MCP tools and run their own keyword or semantic retrieval (we keep vectors in a separate store) rebuild an index nobody queries, on every analyze.

Measured on a ~6.3k-file Java-dominant monorepo: the FTS phase costs ~24 s of a 63 s warm analyze (63.4 s → 87.8 s when enabled) and inflates the LadybugDB file from 153.6 MB to 440.7 MB; on the full graph of the same repo the database grows from ~352 MiB to ~676 MiB. For short-lived per-review indexes that is the single largest avoidable cost after the parse caches.

Proposed solution

An explicit opt-out, default unchanged:

  • gitnexus analyze --skip-fts and the equivalent GITNEXUS_SKIP_FTS=1 (env for callers that cannot change the command line), documented in the README environment-variable table.
  • Skips only the FTS extension load and index creation in the "Creating search indexes" phase. Graph load, communities, processes, embeddings, meta, parse caches and registry are untouched.
  • Meta records capabilities.fts.status = "unavailable" with skipReason = "disabled-by-flag" (or disabled-by-env), so the query side degrades exactly as it does when the extension is missing, and status/search report "disabled" rather than a failure.
  • A later analyze without the flag rebuilds the indexes as usual.

Alternatives considered

  • Relying on the extension-unavailable degrade path: requires breaking the environment and produces warnings on every run.
  • Dropping the indexes after analyze: the time is already spent.
  • --index-only: addresses agent-guide writes, not search indexes.

Acceptance criteria

  • With the flag: no FTS extension load, no FTS index tables, meta shows the explicit skip reason, search reports FTS disabled.
  • Without the flag: byte-identical behavior to today.
  • README env table row for GITNEXUS_SKIP_FTS.

Constraints

  • Must not be confused with a failed index build in diagnostics (ftsFailureIsFatal paths unaffected).
  • Interaction with serve/MCP: a database indexed with the flag should answer search with a clear "FTS disabled for this index" message.

Contribution

  • I am willing to open a PR for this (may need design discussion first).

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions