fix: resume-able bulk insert from failed chunk

[MicBun]

resolves: https://github.com/truflation/website/issues/3697#issuecomment-4334258361

Summary by CodeRabbit

• New Features

  • Enhanced BulkInserter with separate retry budgets for different error types (transient, node catch-up, and infrastructure failures).
  • Added configurable progress logging to track chunk processing and throughput.
  • Increased default retry attempts for improved resilience.

• Documentation

  • Updated API reference and examples with new configuration parameters and their defaults.
  • Added worst-case delay estimates for catch-up scenarios.

[coderabbitai]

Warning

Rate limit exceeded

@MicBun has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 28 minutes and 16 seconds before requesting another review.

To keep reviews running without waiting, you can enable usage-based add-on for your organization. This allows additional reviews beyond the hourly cap. Account admins can enable it under billing.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 007bee67-b990-4d7a-a116-23516f4d6177

📥 Commits

Reviewing files that changed from the base of the PR and between 91afc0d and 37cac20.

📒 Files selected for processing (3)

• bindings/bindings.go
• src/trufnetwork_sdk_py/bulk_inserter.py
• tests/fixtures/test_trufnetwork.py

📝 Walkthrough

Walkthrough

The BulkInserter API is expanded across Go bindings, Python SDK, and documentation with differentiated retry/backoff policies: non-catchup transient attempts increase to 15 (from 5), catchup backoff increases to 15 seconds (from 5), a separate catchup_max_attempts=20 budget is introduced for "node is catching up" errors, infra_max_attempts=10 scopes pre-broadcast infrastructure retries, and progress_log_every_n=500 enables periodic progress logging. The underlying sdk-go dependency is upgraded.

Changes

Cohort / File(s)	Summary
Documentation & Examples README.md, docs/api-reference.md, examples/bulk_insert_example/README.md	Updated BulkInserter configuration examples and API reference to document expanded retry/backoff parameters with increased default values, new catchup_max_attempts, infra_max_attempts, and progress_log_every_n settings.
Go Bindings bindings/bindings.go	Extended NewBulkInserter function signature to accept catchupMaxAttempts, infraMaxAttempts, and progressLogEveryN parameters; added default stderr-bound INFO logger via contractsapi.WithLogger; updated documentation for tuning parameters.
Python SDK src/trufnetwork_sdk_py/bulk_inserter.py	Updated BulkInserter class constructor to accept new retry/backoff parameters (catchup_max_attempts, infra_max_attempts, progress_log_every_n) with adjusted defaults; passes parameters to Go-side NewBulkInserter.
Dependency Upgrade go.mod	Upgraded github.com/trufnetwork/sdk-go from v0.6.5-0.20260424050133-2cce51f9dc50 to v0.7.1-0.20260428121615-6a692c7bf875.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

• sdk-py#78 — Modifies go.mod to update the github.com/trufnetwork/sdk-go dependency version.
• sdk-py#111 — Extends BulkInserter API and introduces the bindings functions and Python class structures being modified here.

Suggested labels

enhancement

Suggested reviewers

• pr-time-tracker

Poem

🐰 With whiskers twitching, retries bloom,
From five attempts to fifteen's room,
Catch-ups get their budget wide,
Progress logs peek far and far,
Infra errors fail fast—no more hide! ✨

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Title check	⚠️ Warning	The title claims to fix resumable bulk insert from failed chunks, but the actual changes focus on expanding retry/backoff behavior with separate budgets and new configuration parameters.	Revise the title to accurately reflect the main change: 'refactor: expand bulk inserter retry and backoff configuration' or similar to describe the parameter additions and retry policy restructuring.
Docstring Coverage	⚠️ Warning	Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch bulkInsertResume

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[holdex]

Time Submission Status

Member	Status	Time	Action	Last Update
MicBun	✅ Submitted	3h	Update time	Apr 28, 2026, 1:12 PM

---

You can submit time with the command. Example:

@holdex pr submit-time 15m

See available commands to help comply with our Guidelines.

[holdex]

Bug Report Checklist

Status	Commit Link	Bug Author	HR Card
✅ Submitted	commit link	@MicBun	HR Card

If this commit or author is incorrect, reply in this PR with:

@holdex bug dispute <correct-commit-url> && bug author @correct-user

Use git blame to identify the right commit and author before posting.

See available commands to help comply with our Guidelines.

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@bindings/bindings.go`:
- Line 231: The comment for progressLogEveryN in bindings.go claims the default
is 0 but the Python wrapper (bulk_inserter.py) sets a default of 500, causing
inconsistency; either update the Go-side comment to note that the Python wrapper
overrides the default or change the Python default to 0 to match Go. Locate the
symbol progressLogEveryN in bindings.go and either adjust its inline comment to
state "default 0 (overridden to 500 in Python bulk_inserter.py)" or change the
default value in bulk_inserter.py (the argument/variable that sets the 500
default) so both sides match. Ensure the chosen change is reflected in both
files' comments/argument defaults to avoid future drift.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 1e0391b4-34be-497b-a272-ce6f1c4cabc2

📥 Commits

Reviewing files that changed from the base of the PR and between 7799703 and 91afc0d.

⛔ Files ignored due to path filters (1)

• go.sum is excluded by !**/*.sum

📒 Files selected for processing (6)

• README.md
• bindings/bindings.go
• docs/api-reference.md
• examples/bulk_insert_example/README.md
• go.mod
• src/trufnetwork_sdk_py/bulk_inserter.py

[MicBun]

@holdex bug commit 91c6bd7 && bug author @MicBun

[MicBun]

@holdex pr submit-time 3h