Skip to content

fix(llm): replace retry with tenacity in ollama.py#367

Merged
imbajin merged 2 commits into
apache:mainfrom
bitflicker64:fix/ollama-retry-tenacity
Jun 23, 2026
Merged

fix(llm): replace retry with tenacity in ollama.py#367
imbajin merged 2 commits into
apache:mainfrom
bitflicker64:fix/ollama-retry-tenacity

Conversation

@bitflicker64

Copy link
Copy Markdown
Contributor

Summary

ollama.py was using the abandoned retry PyPI package (last maintained 2016)
while openai.py and litellm.py in the same directory already use tenacity.
This PR brings ollama.py in line with the established patterns.

Changes

hugegraph-llm/src/hugegraph_llm/models/llms/ollama.py

  • Replace from retry import retry with tenacity imports
  • Replace @retry(tries=3, delay=1) with stop_after_attempt(3),
    wait_exponential(min=4, max=10), and
    retry_if_exception_type((ollama.ResponseError, httpx.ConnectError, httpx.TimeoutException))
    — matching the pattern in openai.py
  • Narrow except Exception to the specific retriable exception types
  • Replace print(f"Retrying LLM call {e}") with log.error(...) (consistent
    with openai.py; log was already imported and used in the same file)
  • Fix misleading "Retrying LLM call" message in agenerate_streaming,
    which has no @retry decorator
  • Replace """Comment""" placeholder docstrings with real descriptions

hugegraph-llm/pyproject.toml

  • Remove "retry", add "tenacity" as an explicit dependency

pyproject.toml

  • Replace "retry~=0.9.2" constraint with "tenacity~=8.5.0"

Closes #365

@dosubot dosubot Bot added size:M This PR changes 30-99 lines, ignoring generated files. dependencies Pull requests that update a dependency file labels Jun 22, 2026
@github-actions github-actions Bot added the llm label Jun 22, 2026

@imbajin imbajin left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: yes. Summary: The Ollama retry policy changes need deterministic regression coverage. Evidence: uv run ruff check . passed; existing Ollama tests are external-service tests and do not cover the new Tenacity retry predicate.

self.async_client = ollama.AsyncClient(host=f"http://{host}:{port}", **kwargs)

@retry(tries=3, delay=1)
@retry(

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Add regression coverage for the new retry policy

This changes both the retry library and the retry predicate/delay for generate() and agenerate(), but the PR does not add a mock-based test for retryable Ollama/httpx failures or for non-retryable errors. The module guidance asks code changes to cover changed behavior; please add a deterministic unit test that stubs client.chat / async_client.chat so CI proves the new Tenacity policy is wired correctly without requiring an Ollama service.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

second commit adds TestOllamaClientRetryPolicy: mock-based, no live service, covers retryable/non-retryable exceptions and transient recovery for both generate() and agenerate().

@imbajin imbajin left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: no. Summary: No obvious issues found in the current head. Evidence: git diff --check passed; Ollama retry policy tests passed (9 passed, 2 skipped); ruff check and format passed for touched files.

@dosubot dosubot Bot added the lgtm This PR has been approved by a maintainer label Jun 23, 2026
@imbajin imbajin changed the title fix(llm): replace retry with tenacity in ollama.py and fix inconsiste… fix(llm): replace retry with tenacity in ollama.py Jun 23, 2026
@imbajin imbajin merged commit 5f4b54a into apache:main Jun 23, 2026
18 checks passed
haohao0103 pushed a commit to haohao0103/incubator-hugegraph-ai that referenced this pull request Jun 30, 2026
## Summary  
  
`ollama.py` was using the abandoned `retry` PyPI package (last
maintained 2016)
while `openai.py` and `litellm.py` in the same directory already use
`tenacity`.
This PR brings `ollama.py` in line with the established patterns.  
  
Closes apache#365
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file lgtm This PR has been approved by a maintainer llm size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] ollama.py: standardize retry logic and fix inconsistencies to match openai.py/litellm.py

2 participants