feat: add registry dual-map and engine factory for VectorStore

[ochanism]

Overview

Third PR for VectorStore multi-store architecture (Phase 1). Extends the RetrieveEngineRegistry with a dual-map (byEngineType + byStoreID) so DB-stored VectorStore records can be loaded into the registry alongside env-configured engines. CRUD operations dynamically register/unregister engines without requiring an app restart.

Part of #921

Why

• byEngineType (existing) handles env stores via RETRIEVE_DRIVER — unchanged, 100% backward compatible
• byStoreID (new) enables multiple instances of the same engine type (e.g., two Elasticsearch clusters)
• StoreRegistry interface avoids modifying the existing RetrieveEngineRegistry interface used by 6 services (17 call sites)
• EngineFactory function type breaks circular import between container and service packages via DI

Changes

Interfaces (internal/types/interfaces/vectorstore.go)

• Add StoreRegistry interface (RegisterWithStoreID, GetByStoreID, UnregisterByStoreID)
• Add EngineFactory function type for DI-injected engine creation

Registry (internal/application/service/retriever/registry.go)

• Rename repositories → byEngineType, add byStoreID map
• Implement StoreRegistry methods with upsert semantics and idempotent unregister
• All existing interface methods unchanged — only search byEngineType

Engine Factory (internal/container/engine_factory.go)

• Per-driver SDK client construction from ConnectionConfig: Postgres, SQLite, Elasticsearch (v7/v8 auto-select), Qdrant, Milvus, Weaviate
• ES version-based SDK selection via ConnectionConfig.Version

Container (internal/container/container.go)

• loadDBStoresIntoRegistry: loads VectorStore records from DB at startup (non-fatal on per-store failure)
• DI registration for StoreRegistry and EngineFactory

Service (internal/application/service/vectorstore.go)

• CreateStore: registers engine in registry after DB persist (best-effort, self-healing on restart)
• DeleteStore: unregisters engine from registry after DB delete
• 10-second context timeout on factory calls to prevent blocking on unreachable hosts

Core

Key files to review:

• internal/application/service/retriever/registry.go — dual-map implementation
• internal/container/engine_factory.go — per-driver factory functions
• internal/application/service/vectorstore.go — dynamic registry updates

Test Plan

• Registry unit tests: dual-map isolation, upsert, idempotent unregister, concurrent access (go test -race)
• Service unit tests: Create registers in registry, Delete unregisters, factory failure doesn't roll back DB, nil registry graceful degradation
• E2E: full CRUD regression (all 26 test cases pass)
• E2E: app restart stability with 6 existing DB stores — no crash, all stores preserved

Notes

• DB stores are registered in the registry but not yet consumed — all KBs still use env stores. Phase 2 (KB ↔ VectorStore binding) will activate the GetByStoreID path.
• engine_factory_test.go omitted due to protobuf namespace conflict between Milvus and Qdrant SDKs in the container package — covered by E2E tests.
• gRPC client connections are not closed on UnregisterByStoreID — known Phase 1 limitation, tracked for Phase 2 Close() interface addition.

Related

• Phase 1 PR 1: feat: add VectorStore entity, repository, and migrations #922 (entity + repository + migrations) — merged
• Phase 1 PR 2: feat: add VectorStore service layer with types and connection testing #933 (service layer + connection testing) — merged
• Phase 1 PR 2: feat: add VectorStore CRUD API endpoints and documentation #934 (CRUD API + documentation) — merged
• Issue: Phase 1: VectorStore Entity + Registry Refactoring #921