Fix: Resolve chat export truncation (Issue #1299)#1314
Open
SyncWithRaj wants to merge 2 commits intokhoj-ai:masterfrom
Open
Fix: Resolve chat export truncation (Issue #1299)#1314SyncWithRaj wants to merge 2 commits intokhoj-ai:masterfrom
SyncWithRaj wants to merge 2 commits intokhoj-ai:masterfrom
Conversation
- Changed ConversationAdapters.get_all_conversations_for_export to use offset/limit pagination and added ordering by updated_at. - Updated /api/chat/export route to accept explicit offset and limit parameters instead of page index. - Updated frontend exportChats function to pass offset and limit to the backend.
- Changed ConversationAdapters.get_all_conversations_for_export to use offset/limit pagination and added ordering by updated_at. - Updated /api/chat/export route to accept explicit offset and limit parameters instead of page index. - Updated frontend exportChats function to pass offset and limit to the backend.
Author
|
Hi! This is my first PR here. I fixed the pagination issue for the chat export. Let me know if you need me to make any changes! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR fixes the data truncation issue reported in #1299, where the "Export Chats" feature would download an incomplete
conversations.jsonfile, missing recent conversations and getting stuck in a loop of duplicate records.Root Cause
get_all_conversations_for_exportused thepageargument directly in the Django queryset slice ([page : page + 10]). This meantpage=0fetched[0:10],page=1fetched[1:11], causing massive overlaps and preventing it from ever reaching all records..order_by(), which could lead to inconsistent pagination results across different databases.pageindex, while the backend expected an explicit offset value.Changes Made
adapters/__init__.py): Updatedget_all_conversations_for_exportto acceptoffsetandlimitinstead ofpage. Added.order_by("-updated_at")to ensure a consistent, predictable order. Fixed the slice to[offset : offset + limit].api_chat.py): Updated the/api/chat/exportroute signature to acceptoffsetandlimitquery parameters.settings/page.tsx): Updated theexportChatsloop to explicitly pass?offset=${page * 10}&limit=10to the backend.How to Test
conversations.jsonfile to verify that the total count of unique conversations matches the database count exactly, with zero duplicates.Before:
After:
Fixes #1299