Agentic RAG Platform

A production-grade, containerized RAG (Retrieval-Augmented Generation) system optimized for document searching, retrieval, query paraphrasing, and LLM-driven tool calling.

The platform provides a modular structure with support for local embedding models, cross-encoder rerankers, keyword retrieval (BM25), SQLite chat storage, and observability integrations.

---

Architecture Overview

project-root/
├── backend/
│   ├── data/                 # SQLite persistence databases, cached text chunk files & ChromaDB collections
│   ├── models/               # Locally downloaded Hugging Face embedding & cross-encoder models
│   ├── agent.py              # Orchestration loop for the assistant LLM and function calls
│   ├── chat_store.py         # Thread-safe SQLite backend persistence layer
│   ├── chunker.py            # Recursive document parser and smart text segmenter
│   ├── download_models.py    # Local downloader/cacher for sentence-transformers models
│   ├── indexer.py            # Sentence embeddings pipeline and ChromaDB indexing manager
│   ├── mcp_manager.py        # Model Context Protocol (MCP) server connector
│   ├── rag_engine.py         # LLM prompt configuration, context assembling, and generation pipeline
│   ├── retriever.py          # Hybrid search orchestrator (Paraphraser + BM25 + Vector DB + RRF + Cross-Encoder)
│   ├── server.py             # Uvicorn/FastAPI backend HTTP endpoint servers
│   ├── tool_registry.py      # Core agent execution tools registration
│   └── tracing.py            # Langfuse observability and trace orchestration module
├── frontend/                 # React SPA frontend (Vite)
├── docker-compose.yml        # Multi-container service configuration
└── Dockerfile                # Multi-stage optimized builder image

---

Features

• Hybrid Retrieval Pipeline: Combine vector similarity search (ChromaDB) with keyword search (BM25) over reciprocal rank fusion (RRF).
• Reranking: Cross-encoder model (ms-marco-MiniLM-L-6-v2) evaluates semantic relevance to bubble up the best matching text chunks.
• Query Paraphrasing: Automatically converts colloquial user queries into formal terminology variants using an LLM.
• Model Context Protocol (MCP): Connects to external data sources and custom local/remote servers.
• Observability: Native trace tracking for LLM actions, token costs, and API calls using Langfuse.
• Containerized Stack: Single-command local environment execution using Docker Compose.

---

Quick Start

1. Prerequisites

Ensure you have Docker and Docker Compose installed.

2. Configure Environment

Create a .env file at the root of the project:

# LLM API details
LLM_BASE_URL=http://localhost:20128/v1
LLM_API_KEY=your-api-key
LLM_MODEL=ag/gemini-3.5-flash-extra-low

# Observability traces (Optional)
LANGFUSE_PUBLIC_KEY=pk-lf-...
LANGFUSE_SECRET_KEY=sk-lf-...
LANGFUSE_BASE_URL=http://localhost:3000

3. Spin Up Services

Run the following command to build the frontend and launch the backend application container:

docker compose up --build

Once started, the RAG API and static dashboard will be available at http://localhost:8000.

---

Development Setup

To run the backend locally without Docker:

1. Create a virtual environment:

   python -m venv .venv
   source .venv/bin/activate  # On Windows: .venv\Scripts\activate

2. Install Python dependencies:

   pip install -r backend/requirements.docker.txt

3. Load the model files:

   python backend/download_models.py

4. Run the API server:

   python backend/server.py