fix: SQLite/Lite mode compatibility for FAQ import and chunk operations#968
Merged
lyingbug merged 1 commit intoTencent:mainfrom Apr 15, 2026
Merged
fix: SQLite/Lite mode compatibility for FAQ import and chunk operations#968lyingbug merged 1 commit intoTencent:mainfrom
lyingbug merged 1 commit intoTencent:mainfrom
Conversation
Multiple fixes for WeKnora Lite mode (SQLite, no Redis):
1. Redis nil pointer panic on FAQ import (knowledge.go)
- Add in-memory fallback (sync.Map) for FAQ import progress tracking
- Add nil guards for running task locks and progress queries
2. seq_id not auto-incrementing in SQLite (chunk.go, tag.go)
- GORM autoIncrement on non-PK columns doesn't work in SQLite
- Add AssignChunkSeqIDs() to pre-assign seq_ids before batch insert
- Add BeforeCreate hook for KnowledgeTag
- Use Unscoped() to include soft-deleted records in MAX(seq_id) query
3. NOW() function not available in SQLite (chunk.go)
- Replace raw SQL NOW() with time.Now() or datetime('now')
- Use dialector check for raw SQL that must differ between PG and SQLite
4. FAQ KB should reject file uploads (knowledge.go)
- Add kb.Type == "faq" check in CreateKnowledgeFromFile
5. Linux/macOS build compatibility (Makefile)
- Conditionally apply macOS-only linker flag based on uname
6. Add SQLite integration tests (chunk_sqlite_test.go)
- TestCreateChunks_SQLite_SeqIDAutoAssigned
- TestCreateChunks_SQLite_SeqIDContinuesFromExisting
- TestCreateChunks_SQLite_SeqIDUniqueAcrossKBs
- TestCreateChunks_SQLite_SeqIDAfterSoftDelete
- TestKnowledgeTag_SQLite_SeqIDAutoAssigned
- TestUpdateChunk_SQLite_NoNOWError
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request
描述 (Description)
Fix multiple SQLite/Lite mode compatibility issues that cause FAQ import to crash or produce incorrect results. These bugs affect all users running WeKnora in Lite mode (SQLite + no Redis).
变更类型 (Type of Change)
影响范围 (Scope)
测试 (Testing)
测试步骤 (Test Steps)
SKIP_FRONTEND=1 make build-lite.env.liteconfig, create a FAQ knowledge basego test -tags sqlite_fts5 -v ./internal/application/repository/ -run SQLite检查清单 (Checklist)
相关 Issue
Fixes #963
数据库迁移 (Database Migration)
部署说明 (Deployment Notes)
No special deployment steps. Changes are backward-compatible with PostgreSQL (Standard mode). The
AssignChunkSeqIDsfunction is a no-op when seq_ids are already set by PostgreSQL's autoIncrement.其他信息 (Additional Information)
Bugs Fixed
1. Redis nil pointer panic on FAQ import
UpsertFAQEntriescallsredisClient.Set()/Get()without nil checks. In Lite mode Redis is disabled (redisClient == nil), causing panic.Fix: Add in-memory
sync.Mapfallback for progress tracking and task locks.2. seq_id always NULL in SQLite
GORM's
autoIncrementtag on non-primary-key columns (SeqID) doesn't work in SQLite. All chunks and tags getseq_id = NULL, breaking FAQ entry listing and tag deletion.Fix: Add
AssignChunkSeqIDs()pre-assignment before batch insert; addBeforeCreatehook forKnowledgeTag.3. UNIQUE constraint violation after soft-delete + re-import
AssignChunkSeqIDsuses GORM's default query which filters out soft-deleted records (deleted_at IS NULL). New seq_ids collide with soft-deleted ones.Fix: Use
Unscoped()when queryingMAX(seq_id).4.
no such function: NOWin SQLiteRaw SQL uses PostgreSQL's
NOW()function which doesn't exist in SQLite.Fix: Replace with
time.Now()(GORM updates) ordatetime('now')(raw SQL), with dialector check where needed.5. FAQ knowledge base accepts file uploads
CreateKnowledgeFromFiledoesn't checkkb.Type, allowing file uploads to FAQ knowledge bases which creates duplicate document-type chunks alongside FAQ entries.Fix: Add
kb.Type == "faq"guard returning descriptive error.6. Linux build failure
Makefile uses macOS-only linker flag
-Wl,-no_warn_duplicate_librariesunconditionally.Fix: Conditionally apply based on
uname.New Tests (6 cases)
TestCreateChunks_SQLite_SeqIDAutoAssignedTestCreateChunks_SQLite_SeqIDContinuesFromExistingTestCreateChunks_SQLite_SeqIDUniqueAcrossKBsTestCreateChunks_SQLite_SeqIDAfterSoftDeleteTestKnowledgeTag_SQLite_SeqIDAutoAssignedTestUpdateChunk_SQLite_NoNOWError