Skip to content

Fix: Resolve chat export truncation (Issue #1299)#1314

Open
SyncWithRaj wants to merge 2 commits intokhoj-ai:masterfrom
SyncWithRaj:fix-chat-export-pagination
Open

Fix: Resolve chat export truncation (Issue #1299)#1314
SyncWithRaj wants to merge 2 commits intokhoj-ai:masterfrom
SyncWithRaj:fix-chat-export-pagination

Conversation

@SyncWithRaj
Copy link
Copy Markdown

Description

This PR fixes the data truncation issue reported in #1299, where the "Export Chats" feature would download an incomplete conversations.json file, missing recent conversations and getting stuck in a loop of duplicate records.

Root Cause

  1. Broken Slicing: The database adapter get_all_conversations_for_export used the page argument directly in the Django queryset slice ([page : page + 10]). This meant page=0 fetched [0:10], page=1 fetched [1:11], causing massive overlaps and preventing it from ever reaching all records.
  2. Missing Order: The queryset lacked an explicit .order_by(), which could lead to inconsistent pagination results across different databases.
  3. Frontend Mismatch: The frontend was passing a sequential page index, while the backend expected an explicit offset value.

Changes Made

  • Backend (adapters/__init__.py): Updated get_all_conversations_for_export to accept offset and limit instead of page. Added .order_by("-updated_at") to ensure a consistent, predictable order. Fixed the slice to [offset : offset + limit].
  • Backend (api_chat.py): Updated the /api/chat/export route signature to accept offset and limit query parameters.
  • Frontend (settings/page.tsx): Updated the exportChats loop to explicitly pass ?offset=${page * 10}&limit=10 to the backend.

How to Test

  1. Create more than 10 test conversations in the Khoj app.
  2. Navigate to Settings and click "Export Chats".
  3. Check the downloaded conversations.json file to verify that the total count of unique conversations matches the database count exactly, with zero duplicates.

Before:

image

After:

image

Fixes #1299

- Changed ConversationAdapters.get_all_conversations_for_export to use offset/limit pagination and added ordering by updated_at.
- Updated /api/chat/export route to accept explicit offset and limit parameters instead of page index.
- Updated frontend exportChats function to pass offset and limit to the backend.
- Changed ConversationAdapters.get_all_conversations_for_export to use offset/limit pagination and added ordering by updated_at.
- Updated /api/chat/export route to accept explicit offset and limit parameters instead of page index.
- Updated frontend exportChats function to pass offset and limit to the backend.
@SyncWithRaj
Copy link
Copy Markdown
Author

Hi! This is my first PR here. I fixed the pagination issue for the chat export. Let me know if you need me to make any changes!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

I can't export chats from after the end of 2025

1 participant