Self Checks
Is your feature request related to a problem?
I currently use an external automation tool to insert papers from Zotero into RagFlow. It would be great if it could be added as an official data source.
Describe the feature you'd like
The system currently supports many data sources, and I suggest adding Zotero (and its official/WebDAV storage options) as another. The basic feature includes supplying a user ID and API key to fetch from the Zotero official API and selecting the attachment storage location. Attachments can come from the web library or a configured WebDAV source. Later, additional features may include filtering by tags, sub-library, etc.
Describe implementation you've considered
Zotero bills the library and attachment storage separately. Users get unlimited library entries and cloud syncing, and they have a web API to fetch library entries. After fetching, you can filter the items to ones with PDF attachments only. You can extract the attachment ID from the response body. Zotero has 2 storage options for attachments: either in the SaaS web library, where you can download from the web API using the attachment ID, or from WebDAV storage. In this case, the easiest way is to get the ZIP directly using the item ID, then unpack it and ingest the PDF file inside. The main web library API also exposes filtering by tags, etc. I can submit a PR to implement this feature if the feature request is accepted.
Documentation, adoption, use case
A scientific paper is a valuable source of knowledge for agents, and indexing it in RagFlow offers strong integration: RagFlow handles retrieval and indexing; a human or agent adds the paper to Zotero; and downstream clients access the knowledge. It also fits naturally with the knowledge structure in current RagFlow versions.
One example use case: a user has an already collected Zotero library and, upon connection, can create a "papers" dataset containing the scientific papers. Then, the user can simply add more papers to Zotero using their own methods, for example, the browser plugin, and later RagFlow will sync that, and the downstream agents will get the knowledge in the papers.
Additional information
No response
Self Checks
Is your feature request related to a problem?
Describe the feature you'd like
The system currently supports many data sources, and I suggest adding Zotero (and its official/WebDAV storage options) as another. The basic feature includes supplying a user ID and API key to fetch from the Zotero official API and selecting the attachment storage location. Attachments can come from the web library or a configured WebDAV source. Later, additional features may include filtering by tags, sub-library, etc.
Describe implementation you've considered
Zotero bills the library and attachment storage separately. Users get unlimited library entries and cloud syncing, and they have a web API to fetch library entries. After fetching, you can filter the items to ones with PDF attachments only. You can extract the attachment ID from the response body. Zotero has 2 storage options for attachments: either in the SaaS web library, where you can download from the web API using the attachment ID, or from WebDAV storage. In this case, the easiest way is to get the ZIP directly using the item ID, then unpack it and ingest the PDF file inside. The main web library API also exposes filtering by tags, etc. I can submit a PR to implement this feature if the feature request is accepted.
Documentation, adoption, use case
Additional information
No response