fix(storage): support pagination in Kubernetes storage GC (#4979) - #4990
Open
Retr0-XD wants to merge 1 commit into
Open
fix(storage): support pagination in Kubernetes storage GC (#4979)#4990Retr0-XD wants to merge 1 commit into
Retr0-XD wants to merge 1 commit into
Conversation
The Kubernetes API enforces a 500-item hard limit per list request. Garbage collection in Dex's Kubernetes storage was not handling pagination, causing it to only process the first 500 objects and miss expired items beyond that limit. Changes: - Add 'Continue' field to ListMeta struct to capture continuation tokens - Add listNWithContinue() method that accepts optional continue token - Modify GarbageCollect() to loop through all pages for each resource type: * AuthRequests, AuthCodes, DeviceRequests, DeviceTokens, AuthSessions - Each resource type now properly paginates until no more items exist This ensures garbage collection processes all expired objects in large deployments with more than 500 items of any single resource type. Fixes: dexidp#4979 Signed-off-by: Sakthi Harish <sakthi.harish@edgeverve.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes incomplete garbage collection in Kubernetes storage backend when deployments have more than 500 expired objects.
Problem: The Kubernetes API enforces a 500-item hard limit per list request. The GC function was not handling pagination, so it only processed the first 500 objects.
Solution: Add pagination support by capturing and using continuation tokens in the GarbageCollect function for all resource types.
Changes: Added Continue field to ListMeta, new listNWithContinue() method, and pagination loops in GarbageCollect().
Fixes #4979