Fetch GlobalIngressIP from API server to avoid stale cache - #4123
Conversation
When the service controller re-queues a GlobalIngressIP immediately after initial allocation, the syncer's cache may not yet reflect the updated status with the allocated IP. This causes the controller to attempt re-allocation instead of updating the internal service. Fix by always fetching the latest GlobalIngressIP from the API server at the start of the Create operation, before trimming status conditions or processing. This ensures we have fresh data even when the cache is stale. Fixes: submariner-io#4122 Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
|
🤖 Created branch: z_pr4123/tpantelis/gip-allocate-ips-race |
WalkthroughThe controller now stores a GlobalIngressIP resource client, refreshes the resource from the API server during create processing, and requeues when retrieval or conversion fails. ChangesGlobalIngressIP refresh
Estimated code review effort: 2 (Simple) | ~10 minutes Sequence Diagram(s)sequenceDiagram
participant CreateProcessing
participant globalIngressIPController
participant KubernetesAPIServer
CreateProcessing->>globalIngressIPController: Refresh GlobalIngressIP
globalIngressIPController->>KubernetesAPIServer: Retrieve namespaced resource
KubernetesAPIServer-->>globalIngressIPController: Return current GlobalIngressIP
globalIngressIPController-->>CreateProcessing: Return converted resource or error
CreateProcessing->>CreateProcessing: Requeue on refresh error
Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
🧹 Nitpick comments (1)
pkg/globalnet/controllers/global_ingressip_controller.go (1)
171-178: 🗄️ Data Integrity & Integration | 🔵 Trivial | 🏗️ Heavy liftAdd a regression test for the stale-cache Create path.
Use a queued
GlobalIngressIPwith an emptyStatus.AllocatedIPwhile the API-server object already has an allocated IP. Assert that the controller reuses the live IP, updates the internal Service, and does not allocate a second IP. Also verify that refresh failures requeue the operation.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@pkg/globalnet/controllers/global_ingressip_controller.go` around lines 171 - 178, Add regression coverage for the GlobalIngressIP create/reconcile path around fetchLatestFromAPIServer: queue a stale object with empty Status.AllocatedIP while the API-server version has an allocated IP, then assert the live IP is reused, the internal Service is updated, and no second allocation occurs. Add a refresh-failure case asserting the operation is requeued.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@pkg/globalnet/controllers/global_ingressip_controller.go`:
- Around line 171-178: Add regression coverage for the GlobalIngressIP
create/reconcile path around fetchLatestFromAPIServer: queue a stale object with
empty Status.AllocatedIP while the API-server version has an allocated IP, then
assert the live IP is reused, the internal Service is updated, and no second
allocation occurs. Add a refresh-failure case asserting the operation is
requeued.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 1a88f302-c15a-48b5-9d96-a5e0955566eb
📒 Files selected for processing (2)
pkg/globalnet/controllers/global_ingressip_controller.gopkg/globalnet/controllers/types.go
|
🤖 Closed branches: [z_pr4123/tpantelis/gip-allocate-ips-race] |
The submariner fix[1] was merged, using the special "devel" version to
test the fix before the next release.
This uses the "devel" tag for all images:
% subctl show versions --context dr1
✓ Showing versions
COMPONENT REPOSITORY CONFIGURED RUNNING ARCH
submariner-gateway quay.io/submariner devel devel-a5d2aba37b1d arm64
submariner-routeagent quay.io/submariner devel devel-a5d2aba37b1d arm64
submariner-globalnet quay.io/submariner devel devel-a5d2aba37b1d arm64
submariner-metrics-proxy quay.io/submariner devel devel-dc2b634fb6bb arm64
submariner-operator quay.io/submariner devel devel-8bef08878f9c arm64
submariner-lighthouse-agent quay.io/submariner devel devel-b1b4669a6542 arm64
submariner-lighthouse-coredns quay.io/submariner devel devel-b1b4669a6542 arm64
[1] submariner-io/submariner#4123
Signed-off-by: Nir Soffer <nsoffer@redhat.com>
The submariner fix[1] was merged, using the special "devel" version to
test the fix before the next release.
This uses the "devel" tag for all images:
% subctl show versions --context dr1
✓ Showing versions
COMPONENT REPOSITORY CONFIGURED RUNNING ARCH
submariner-gateway quay.io/submariner devel devel-a5d2aba37b1d arm64
submariner-routeagent quay.io/submariner devel devel-a5d2aba37b1d arm64
submariner-globalnet quay.io/submariner devel devel-a5d2aba37b1d arm64
submariner-metrics-proxy quay.io/submariner devel devel-dc2b634fb6bb arm64
submariner-operator quay.io/submariner devel devel-8bef08878f9c arm64
submariner-lighthouse-agent quay.io/submariner devel devel-b1b4669a6542 arm64
submariner-lighthouse-coredns quay.io/submariner devel devel-b1b4669a6542 arm64
[1] submariner-io/submariner#4123
Signed-off-by: Nir Soffer <nsoffer@redhat.com>
The submariner fix[1] was merged, using the special "devel" version to
test the fix before the next release.
This uses the "devel" tag for all images:
% subctl show versions --context dr1
✓ Showing versions
COMPONENT REPOSITORY CONFIGURED RUNNING ARCH
submariner-gateway quay.io/submariner devel devel-a5d2aba37b1d arm64
submariner-routeagent quay.io/submariner devel devel-a5d2aba37b1d arm64
submariner-globalnet quay.io/submariner devel devel-a5d2aba37b1d arm64
submariner-metrics-proxy quay.io/submariner devel devel-dc2b634fb6bb arm64
submariner-operator quay.io/submariner devel devel-8bef08878f9c arm64
submariner-lighthouse-agent quay.io/submariner devel devel-b1b4669a6542 arm64
submariner-lighthouse-coredns quay.io/submariner devel devel-b1b4669a6542 arm64
[1] submariner-io/submariner#4123
Signed-off-by: Nir Soffer <nsoffer@redhat.com>
The submariner fix[1] was merged, using the special "devel" version to
test the fix before the next release.
This uses the "devel" tag for all images:
% subctl show versions --context dr1
✓ Showing versions
COMPONENT REPOSITORY CONFIGURED RUNNING ARCH
submariner-gateway quay.io/submariner devel devel-a5d2aba37b1d arm64
submariner-routeagent quay.io/submariner devel devel-a5d2aba37b1d arm64
submariner-globalnet quay.io/submariner devel devel-a5d2aba37b1d arm64
submariner-metrics-proxy quay.io/submariner devel devel-dc2b634fb6bb arm64
submariner-operator quay.io/submariner devel devel-8bef08878f9c arm64
submariner-lighthouse-agent quay.io/submariner devel devel-b1b4669a6542 arm64
submariner-lighthouse-coredns quay.io/submariner devel devel-b1b4669a6542 arm64
[1] submariner-io/submariner#4123
Signed-off-by: Nir Soffer <nsoffer@redhat.com>
The submariner fix[1] was merged, using the special "devel" version to
test the fix before the next release.
This uses the "devel" tag for all images:
% subctl show versions --context dr1
✓ Showing versions
COMPONENT REPOSITORY CONFIGURED RUNNING ARCH
submariner-gateway quay.io/submariner devel devel-a5d2aba37b1d arm64
submariner-routeagent quay.io/submariner devel devel-a5d2aba37b1d arm64
submariner-globalnet quay.io/submariner devel devel-a5d2aba37b1d arm64
submariner-metrics-proxy quay.io/submariner devel devel-dc2b634fb6bb arm64
submariner-operator quay.io/submariner devel devel-8bef08878f9c arm64
submariner-lighthouse-agent quay.io/submariner devel devel-b1b4669a6542 arm64
submariner-lighthouse-coredns quay.io/submariner devel devel-b1b4669a6542 arm64
[1] submariner-io/submariner#4123
Signed-off-by: Nir Soffer <nsoffer@redhat.com>
The submariner fix[1] was merged, using the special "devel" version to
test the fix before the next release.
This uses the "devel" tag for all images:
% subctl show versions --context dr1
✓ Showing versions
COMPONENT REPOSITORY CONFIGURED RUNNING ARCH
submariner-gateway quay.io/submariner devel devel-a5d2aba37b1d arm64
submariner-routeagent quay.io/submariner devel devel-a5d2aba37b1d arm64
submariner-globalnet quay.io/submariner devel devel-a5d2aba37b1d arm64
submariner-metrics-proxy quay.io/submariner devel devel-dc2b634fb6bb arm64
submariner-operator quay.io/submariner devel devel-8bef08878f9c arm64
submariner-lighthouse-agent quay.io/submariner devel devel-b1b4669a6542 arm64
submariner-lighthouse-coredns quay.io/submariner devel devel-b1b4669a6542 arm64
[1] submariner-io/submariner#4123
Signed-off-by: Nir Soffer <nsoffer@redhat.com>
The submariner fix[1] was merged, using the special "devel" version to
test the fix before the next release.
This uses the "devel" tag for all images:
% subctl show versions --context dr1
✓ Showing versions
COMPONENT REPOSITORY CONFIGURED RUNNING ARCH
submariner-gateway quay.io/submariner devel devel-a5d2aba37b1d arm64
submariner-routeagent quay.io/submariner devel devel-a5d2aba37b1d arm64
submariner-globalnet quay.io/submariner devel devel-a5d2aba37b1d arm64
submariner-metrics-proxy quay.io/submariner devel devel-dc2b634fb6bb arm64
submariner-operator quay.io/submariner devel devel-8bef08878f9c arm64
submariner-lighthouse-agent quay.io/submariner devel devel-b1b4669a6542 arm64
submariner-lighthouse-coredns quay.io/submariner devel devel-b1b4669a6542 arm64
[1] submariner-io/submariner#4123
Signed-off-by: Nir Soffer <nsoffer@redhat.com>
When the service controller re-queues a
GlobalIngressIPimmediately after initial allocation, the syncer's cache may not yet reflect the updated status with the allocated IP. This causes the controller to attempt re-allocation instead of updating the internal service.Fix by always fetching the latest
GlobalIngressIPfrom the API server at the start of the Create operation, before trimming status conditions or processing. This ensures we have fresh data even when the cache is stale.Fixes: #4122
Summary by CodeRabbit