Skip to content

matteo-brandolino/notes-rag

Repository files navigation

Notes RAG - AI-Powered Personal Knowledge Base

A modern full-stack RAG (Retrieval-Augmented Generation) application that combines personal note-taking with AI-powered semantic search. Built with Next.js, OpenAI, and PostgreSQL with vector search capabilities.

Tech Stack TypeScript PostgreSQL Vitest

πŸŽ₯ Demo

Application Demo

A quick walkthrough of the application showing note management, AI chat, and RAG visualization in action.

🎯 What is this?

This project demonstrates a production-ready RAG system where you can:

  • πŸ“ Manage Notes: Create, edit, organize notes with tags, colors, and pinning
  • πŸ€– AI Chat: Ask questions and get answers based on your notes and knowledge base
  • πŸ” Semantic Search: Find relevant information using OpenAI embeddings and vector similarity
  • πŸ“Š Visual Results: See search results visualized in an interactive node graph

Perfect for building a "second brain" or demonstrating RAG architecture to potential employers.

πŸ—οΈ Architecture

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚   Next.js   β”‚ ← Server Components + API Routes
β”‚  Frontend   β”‚
β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”˜
       β”‚
β”Œβ”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚  Vercel AI SDK + OpenAI GPT-4       β”‚ ← Chat & Embeddings
β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
       β”‚
β”Œβ”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚  PostgreSQL + pgvector              β”‚ ← Vector Similarity Search
β”‚  (Notes, Resources, Embeddings)     β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Key Components

  • Frontend: React 18 with Server/Client Components, shadcn-ui, TailwindCSS
  • Backend: Next.js App Router API routes, Server Actions
  • AI: OpenAI GPT-4o for chat, text-embedding-ada-002 for embeddings
  • Database: PostgreSQL with pgvector extension for vector operations
  • ORM: Drizzle ORM with type-safe queries
  • Testing: Vitest with comprehensive unit and integration tests

πŸš€ Quick Start

Prerequisites

  • Node.js 20+
  • PostgreSQL 14+ with pgvector extension
  • OpenAI API key
  • pnpm (recommended) or npm

1. Clone and Install

git clone https://github.com/yourusername/notes-rag.git
cd notes-rag
pnpm install

2. Database Setup

# Install pgvector extension in PostgreSQL
# Connect to your database and run:
CREATE EXTENSION vector;

3. Environment Variables

Create a .env file in the root:

DATABASE_URL="postgresql://user:password@localhost:5432/notes_rag"
OPENAI_API_KEY="sk-..."

See .env.example for reference.

4. Database Migration

# Generate migration files
pnpm db:generate

# Run migrations
pnpm db:migrate

# (Optional) Open Drizzle Studio to view your database
pnpm db:studio

5. Run Development Server

pnpm dev

Open http://localhost:3000

πŸ“ Project Structure

notes-rag/
β”œβ”€β”€ app/
β”‚   β”œβ”€β”€ api/chat/          # AI chat endpoint
β”‚   β”œβ”€β”€ layout.tsx         # Root layout
β”‚   └── page.tsx           # Home page
β”œβ”€β”€ components/
β”‚   β”œβ”€β”€ notes/             # Notes management UI
β”‚   β”œβ”€β”€ chat/              # Chat interface + RAG visualization
β”‚   └── ui/                # Reusable UI components (shadcn)
β”œβ”€β”€ lib/
β”‚   β”œβ”€β”€ ai/
β”‚   β”‚   β”œβ”€β”€ provider.ts    # OpenAI configuration
β”‚   β”‚   └── embedding.ts   # Embedding generation & search
β”‚   β”œβ”€β”€ actions/
β”‚   β”‚   β”œβ”€β”€ notes.ts       # Note CRUD operations
β”‚   β”‚   └── resources.ts   # Knowledge base operations
β”‚   └── db/
β”‚       β”œβ”€β”€ schema/        # Database schema (Drizzle)
β”‚       └── migrate.ts     # Migration runner
β”œβ”€β”€ test/                  # Test configuration
└── vitest.config.ts       # Testing setup

πŸ§ͺ Testing

Run the comprehensive test suite:

# Watch mode (interactive)
pnpm test

# Run once (CI/CD)
pnpm test:run

# Visual UI dashboard
pnpm test:ui

# Coverage report
pnpm test:coverage

Test Coverage:

  • Unit tests for embedding logic (text chunking, vector generation)
  • Unit tests for note CRUD operations
  • Integration tests for chat API endpoint
  • All external dependencies mocked (OpenAI API, database)

πŸ”‘ Key Features Explained

1. Automatic Embeddings

When you create or update a note, the system automatically:

  1. Splits the text into semantic chunks (sentences)
  2. Generates embeddings using OpenAI's text-embedding-ada-002
  3. Stores embeddings in PostgreSQL with pgvector
  4. Indexes them for fast similarity search

2. Semantic Search

The AI chat uses vector similarity search:

  • User query β†’ embedding
  • Compare with all stored embeddings using cosine similarity
  • Return top 8 results with similarity > 0.5
  • AI uses these results to answer questions

3. RAG Visualization

Interactive React Flow diagram shows:

  • Central query node
  • Connected result nodes (notes/resources)
  • Color-coded similarity scores
  • Draggable, zoomable interface

4. AI Tools

The chat assistant has two tools:

  • Search: Find relevant content in notes/resources
  • Add Resource: Save new knowledge base items

The AI automatically decides when to use each tool based on the conversation.

πŸ“Š Database Schema

Notes

  • Stores user notes with title, content, tags, color
  • Automatic timestamp tracking
  • Pin functionality for important notes

Resources

  • General knowledge base items
  • Added via chat or API

Embeddings

  • Links to notes or resources
  • Contains text chunks and 1536-dimensional vectors
  • HNSW index for fast similarity search

πŸ› οΈ Available Scripts

# Development
pnpm dev              # Start dev server
pnpm build            # Build for production
pnpm start            # Start production server
pnpm lint             # Run ESLint

# Testing
pnpm test             # Run tests (watch mode)
pnpm test:run         # Run tests once
pnpm test:ui          # Open Vitest UI
pnpm test:coverage    # Generate coverage report

# Database
pnpm db:generate      # Generate migrations
pnpm db:migrate       # Run migrations
pnpm db:push          # Push schema to DB (dev only)
pnpm db:studio        # Open Drizzle Studio
pnpm db:drop          # Drop all tables

🎨 Tech Stack Deep Dive

Category Technology Why?
Framework Next.js 16 Server Components, API routes, excellent DX
Language TypeScript Type safety, better tooling
AI OpenAI GPT-4 + Embeddings Industry-standard, reliable
AI SDK Vercel AI SDK Streaming, tool calling, React hooks
Database PostgreSQL Robust, supports vector extension
Vector Search pgvector Native PostgreSQL extension, simpler than separate vector DB
ORM Drizzle Type-safe, performant, great DX
UI shadcn-ui + Tailwind Beautiful, accessible, customizable
Visualization React Flow Interactive node graphs
Testing Vitest Fast, modern, Vite-native

πŸ” Environment Variables

Variable Description Required
DATABASE_URL PostgreSQL connection string βœ…
OPENAI_API_KEY OpenAI API key βœ…
NODE_ENV Environment (development/production/test) ❌

🀝 Contributing

This is a portfolio/demo project, but suggestions are welcome!

  1. Fork the repository
  2. Create a feature branch
  3. Make your changes
  4. Run tests: pnpm test:run
  5. Submit a pull request

πŸ“ License

MIT License - feel free to use this project for learning or as a portfolio piece.

πŸ™ Acknowledgments

πŸ“§ Contact

For questions or feedback, open an issue or reach out via GitHub.


⭐ If this project helped you learn RAG , consider starring it!

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors