Skip to content

docs: administration: document input rate metrics and rate gate - #2637

Merged
patrick-stephens merged 2 commits into
fluent:masterfrom
eschabell:erics_in_gate_rate_rate_metrics_updates
Aug 11, 2026
Merged

docs: administration: document input rate metrics and rate gate#2637
patrick-stephens merged 2 commits into
fluent:masterfrom
eschabell:erics_in_gate_rate_rate_metrics_updates

Conversation

@eschabell

@eschabell eschabell commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator

Add the rate_window and rate_gate settings shared by all input plugins to the pipeline section, covering the pause and resume thresholds and how backpressure scales the effective limit. Add the five new fluentbit_input_rate_* metrics to the monitoring reference.

Note: this update needed to cover code changes without doc PR

Summary by CodeRabbit

  • Documentation
    • Added guidance for monitoring input ingestion rates and rate-limiting behavior.
    • Documented new Prometheus metrics for byte rate, record rate, busy chunks, limiting status, and retry attempts.
    • Included configuration options, defaults, threshold behavior, backpressure handling, and an HTTP input example.

@eschabell eschabell self-assigned this Jul 31, 2026
@eschabell
eschabell requested a review from a team as a code owner July 31, 2026 16:03
@eschabell eschabell added waiting-on-review Waiting on a review from mainteners 5.0.9 labels Jul 31, 2026
@coderabbitai

coderabbitai Bot commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 09e4c7b8-ae1c-4fe5-8e4b-0c8635102958

📥 Commits

Reviewing files that changed from the base of the PR and between 6d81f90 and 347afbb.

📒 Files selected for processing (2)
  • administration/configuring-fluent-bit/yaml/pipeline-section.md
  • administration/monitoring.md
🚧 Files skipped from review as they are similar to previous changes (1)
  • administration/monitoring.md

📝 Walkthrough

Walkthrough

The documentation adds input ingestion-rate metrics, optional rate-gate configuration, rate-gate behavior, and five V2 Prometheus metrics.

Changes

Input rate documentation

Layer / File(s) Summary
Rate metrics and rate-gate documentation
administration/configuring-fluent-bit/yaml/pipeline-section.md, administration/monitoring.md
Documents ingestion-rate measurement, byte and record limits, backpressure adjustment, pause and resume thresholds, rate-gate states, retry attempts, and an HTTP configuration example.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

Suggested reviewers: patrick-stephens

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the documentation of input rate metrics and the rate gate.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@eschabell

Copy link
Copy Markdown
Collaborator Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Aug 1, 2026

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@administration/configuring-fluent-bit/yaml/pipeline-section.md`:
- Line 99: Update the gate behavior description to explain the combined-limit
semantics: pausing occurs when either the byte-rate or record-rate limit is
exceeded, and resumption requires both measured rates to fall below their
respective effective limits multiplied by rate_gate.resume_ratio.
- Line 97: Update the rate gate activation wording near the rate_gate
configuration description to state that rate_gate must be enabled and at least
one of rate_gate.max_bytes or rate_gate.max_records must be set to a non-zero
value; preserve that zero means unlimited.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: be8a90a6-52e4-4804-8bd1-619b83cdca10

📥 Commits

Reviewing files that changed from the base of the PR and between 0eb06a6 and 6d81f90.

📒 Files selected for processing (2)
  • administration/configuring-fluent-bit/yaml/pipeline-section.md
  • administration/monitoring.md

Comment thread administration/configuring-fluent-bit/yaml/pipeline-section.md Outdated
Comment thread administration/configuring-fluent-bit/yaml/pipeline-section.md Outdated
@eschabell
eschabell force-pushed the erics_in_gate_rate_rate_metrics_updates branch 2 times, most recently from 62885b2 to 047e6d3 Compare August 1, 2026 08:41
@eschabell

Copy link
Copy Markdown
Collaborator Author

@patrick-stephens ready for review!

Comment thread administration/configuring-fluent-bit/yaml/pipeline-section.md Outdated
Comment thread administration/configuring-fluent-bit/yaml/pipeline-section.md
@eschabell eschabell added 5.1 and removed 5.0.9 labels Aug 6, 2026
Add the rate_window and rate_gate settings shared by all input plugins
to the pipeline section, covering the pause and resume thresholds and
how backpressure scales the effective limit. Add the five new
fluentbit_input_rate_* metrics to the monitoring reference. Also
clarify rate gate pause and resume semantics and rate gate activation
requirements.

Note: this update needed to cover code changes without doc PR

Signed-off-by: Eric D. Schabell <eric@schabell.org>
…example

Signed-off-by: Eric D. Schabell <eric@schabell.org>
@eschabell
eschabell force-pushed the erics_in_gate_rate_rate_metrics_updates branch from 047e6d3 to 347afbb Compare August 9, 2026 18:29
@eschabell

Copy link
Copy Markdown
Collaborator Author

@patrick-stephens fixed conflicts, back to you for review.

@patrick-stephens
patrick-stephens merged commit 21a0519 into fluent:master Aug 11, 2026
9 checks passed
@eschabell eschabell removed the waiting-on-review Waiting on a review from mainteners label Aug 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants