[Bug]: `refresh_ref_docs()` / `arefresh_ref_docs()` drop kwargs after the first document in a batch

[gautamvarmadatla]

Bug Description

insert_kwargs and update_kwargs passed to refresh_ref_docs() are silently dropped after the first matching document because the method calls .pop() on the shared update_kwargs dict inside the document loop. So, the first inserted or updated document receives the expected kwargs, but subsequent documents in the same batch receive {} without any error.

Version

0.14.21

Steps to Reproduce

from typing import Any, List

from llama_index.core import Document, VectorStoreIndex
from llama_index.core.schema import BaseNode, TransformComponent


class RecordKwargs(TransformComponent):
    def __call__(self, nodes: List[BaseNode], **kwargs: Any) -> List[BaseNode]:
        print(f"transform received: {kwargs}")
        return nodes


docs = [Document(text=f"doc {i}") for i in range(3)]
index = VectorStoreIndex([], transformations=[RecordKwargs()])

print("refresh_ref_docs with insert_kwargs={'my_flag': True}:")
index.refresh_ref_docs(docs, insert_kwargs={"my_flag": True})

Relevant Logs/Tracebacks

refresh_ref_docs with insert_kwargs={'my_flag': True}:
transform received: {'my_flag': True}
transform received: {}
transform received: {}
[True, True, True]

[dosubot]

Good catch — this is a confirmed bug. Both refresh_ref_docs and arefresh_ref_docs in BaseIndex call .pop() on the shared update_kwargs dict inside the document loop, so the key is removed after the first iteration and all subsequent documents get {}.

The fix is to extract the kwargs before the loop using .get() instead of .pop() inside it:

# Before the loop
_insert_kwargs = update_kwargs.get("insert_kwargs", {})
_update_kwargs = update_kwargs.get("update_kwargs", {})

for i, document in enumerate(documents):
    existing_doc_hash = self._docstore.get_document_hash(document.get_doc_id())
    if existing_doc_hash is None:
        self.insert(document, **_insert_kwargs)
        refreshed_documents[i] = True
    elif existing_doc_hash != document.hash:
        self.update_ref_doc(document, **_update_kwargs)
        refreshed_documents[i] = True

The same change is needed in the async variant arefresh_ref_docs. No existing PR appears to address this yet — a fix would be welcome if you'd like to submit one!

To reply, just mention @dosu.

---

Share context across your team and agents. Try Dosu.

[badalraj9]

I'd like to work on this. Will fix both refresh_ref_docs and arefresh_ref_docs using .get() instead of .pop() inside the loop.