Skip to content

fix(storage): limit unclustered block selection in recluster compact phase#19641

Open
zhyass wants to merge 1 commit intodatabendlabs:mainfrom
zhyass:feat_recluster
Open

fix(storage): limit unclustered block selection in recluster compact phase#19641
zhyass wants to merge 1 commit intodatabendlabs:mainfrom
zhyass:feat_recluster

Conversation

@zhyass
Copy link
Copy Markdown
Member

@zhyass zhyass commented Mar 30, 2026

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

Recluster now caps the number of unclustered blocks selected for compaction per round, preventing the compact phase from becoming a bottleneck when tables have many unclustered segments.

  • Added MAX_UNCLUSTERED_BLOCKS_PER_RECLUSTER (1000) as the upper bound, further capped by compact_max_block_selection setting
  • Recluster select_segments now tracks unclustered_block_num and breaks early once the limit is reached, allowing incremental progress across multiple invocations

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

This change is Reviewable

@github-actions github-actions bot added the pr-bugfix this PR patches a bug in codebase label Mar 30, 2026
@zhyass zhyass requested review from SkyFan2002 and dantengsky March 30, 2026 15:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-bugfix this PR patches a bug in codebase

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant