[P1] Benchmarks vs FAISS/Qdrant for vector search (recall, QPS)

[varun29ankuS]

Context

shodh-redb includes IVF-PQ and fractal vector indexes. To validate these for real AI/ML workloads, we need recall and throughput benchmarks against established vector search engines.

Scope

• Benchmark datasets:
  • SIFT1M (128d, 1M vectors)
  • GloVe (100d/200d, 1.2M vectors)
  • Deep1M (96d, 1M vectors) if feasible
• Metrics:
  • Recall@1, Recall@10, Recall@100
  • Queries per second (QPS) at each recall level
  • Index build time
  • Memory usage during search
  • Index size on disk
• Compare against:
  • FAISS (IVF-PQ with equivalent parameters)
  • Qdrant (if feasible, via HTTP API)
  • hnswlib (as HNSW baseline)
• Test both IVF-PQ and fractal index implementations
• Vary parameters: nprobe, nlist, m (sub-quantizers), nbits

Acceptance Criteria

• Benchmark harness for vector search workloads
• SIFT1M results for shodh-redb vs FAISS at minimum
• Recall-vs-QPS tradeoff curves plotted
• Identified parameter sweet spots for shodh-redb indexes
• Clear documentation of where shodh-redb is competitive and where it's not