Multi-Model BYOK Support (i.e. OpenAI, Anthropic, DeepSeek) for RAG Engine

[samarthsaxena2004]

🚀 Feature Description

Currently, AI Knowledge Hub is tightly coupled with Google Gemini 2.5 Flash for its RAG and summary capabilities. While Gemini is fast and cost-effective, different users prefer different models for specific tasks (e.g., Claude 3.5 Sonnet for reasoning, GPT-4o for creative writing, or DeepSeek for budget efficiency).

I propose expanding the Bring Your Own Key (BYOK) architecture to support multiple Model Providers via the frontend UI and backend LangChain integration.

🧠 Motivation & Use Case

• User Choice: Users may already have paid subscriptions/API credits for OpenAI or Anthropic.
• Performance: Some complex PDFs (e.g., legal or medical) might yield better results with larger models like GPT-4o or Claude 3.5 Sonnet.
• Resilience: If the Google Gemini API experiences downtime or rate limits, users can switch providers.

🛠 Proposed Implementation Details

1. Frontend (frontend/)

• UI Update: Modify the Settings/Upload modal to include a "Select Model Provider" dropdown (Shadcn/UI Select component).
• Input Logic: Dynamically change the API Key input label based on the selection (e.g., "Enter OpenAI Key" vs "Enter Gemini Key").
• State Management: Pass the selected provider and model_name along with the api_key in the API request headers or body.

Proposed Options:

• Google: Gemini 1.5 Flash (Default), Gemini 1.5 Pro
• OpenAI: GPT-4o, GPT-3.5-Turbo
• Anthropic: Claude 3.5 Sonnet
• DeepSeek: DeepSeek-V3 (via OpenAI-compatible endpoint)

2. Backend (backend/)

• Dependencies: Update requirements.txt to include provider-specific LangChain packages (e.g., langchain-openai, langchain-anthropic).
• Refactor rag.py:
  • Create a factory function/switch case to initialize the LLM based on the user's request.
  • Example logic:

  def get_llm(provider, api_key):
      if provider == "openai":
          return ChatOpenAI(api_key=api_key, model="gpt-4o")
      elif provider == "anthropic":
          return ChatAnthropic(api_key=api_key, model="claude-3-5-sonnet")
      # ... default to Gemini

• Endpoint Update: Update main.py to accept provider string in the request schema.

✅ Acceptance Criteria

• Users can select a provider (Google, OpenAI, Anthropic) from the UI.
• Users can input the specific API key for that provider.
• Backend successfully initializes the correct LangChain LLM wrapper based on the selection.
• RAG chat and Summarization work correctly with the new models.
• Error handling added for invalid keys for specific providers.

ℹ️ Additional Context

Since the project uses LangChain, switching models should be straightforward using the unified Chat Model interface. The Vector Store (ChromaDB) and Embeddings can likely remain consistent (using Google Embeddings or switching to text-embedding-3-small if the user selects OpenAI, though keeping embeddings consistent is easier for existing databases).