Skip to content

fix(its): correct InstanceStatus convergence semantics - #10799

Merged
leon-ape merged 7 commits into
mainfrom
bugfix/10758-instance-status-contract
Aug 25, 2026
Merged

fix(its): correct InstanceStatus convergence semantics#10799
leon-ape merged 7 commits into
mainfrom
bugfix/10758-instance-status-contract

Conversation

@leon-ape

@leon-ape leon-ape commented Aug 24, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • define InstanceStatus.UpToDate as full convergence of an Active/Present instance to the current InstanceSet desired state: Pod target, dynamic configs, and PVC expansion
  • invalidate only existing instances affected by those desired-state changes; preserve unaffected observations and keep add/remove/offline/released represented by DesiredState and CurrentState
  • fail closed UpToDate for a desired Active identity whose current Pod/Instance is missing during generation handoff, without rewriting runtime health fields
  • keep Ready, Available, Failed, and Role as current runtime observations, independent of desired-state convergence
  • align legacy ITS, ITS2, and the Instance controller on the same public InstanceStatus contract
  • close ITS2 generation-handoff gaps and publish the desired Pod revision for every Active instance, including absent or not-yet-observed children
  • keep handed-off but unconverged ITS2 instances in both the rolling availability window and the roleful member-update plan, independent of role or ordinal traversal order

Contract and design

status.observedGeneration continues to mean that the controller has observed and started processing the InstanceSet generation. It does not make the entire status an atomic snapshot of that generation, and this change adds no generation field or reconciler reordering.

The supported existing-instance update domains are:

  • Pod changes, including revision changes and revision-excluded in-place fields such as resources and metadata/restart inputs
  • dynamic config changes represented by per-instance config hashes
  • PVC expansion represented by PVC targets and VolumeExpansion

Legacy ITS checks revision-excluded Pod fields directly. ITS2 compares the desired and current child Instance in the same supported domains. If a child generation has not yet been observed, its previous parent UpToDate=true cannot cross the handoff even when the child spec has already been submitted and matches the desired spec.

InstanceStatus.CurrentRevision and UpdateRevision remain in the Pod-revision domain. ITS2 deterministically computes the desired revision from each Active desired child spec, matching legacy ITS publication timing instead of waiting for child status. Legacy aggregate revision maps remain in the Pod-revision domain, while ITS2 aggregate revision maps remain in their existing Instance-spec revision domain.

Runtime health helpers require a current observed Instance generation and their runtime condition, but do not require UpToDate. ITS2 update planning and updated-replica accounting still require full convergence.

Rolling coordination explicitly combines health and convergence. Before sorting or traversing candidates, ITS2 counts existing handed-off-but-unconverged participants against maxUnavailable and, for roleful updates, against the member-update plan. Runtime unavailability by itself counts only against the availability budget. This preserves Serial one-by-one behavior even when maxUnavailable is greater than one.

Scope and downstream audit

  • no Apps or Ops logic is introduced into workload status producers
  • Apps role, config, and volume-expansion consumers already use their independent fields with Active/Present filtering
  • current main Ops has no direct per-instance UpToDate or UpdateRevision consumer
  • fix(ops): use public status for upgrade and lifecycle completion #10760 remains an independent draft consumer of the separate UpToDate, Ready, Available, and Failed dimensions and carries no ITS/ITS2 producer workaround
  • no new test files are introduced; coverage is added to the existing legacy ITS, ITS2, Instance, shared status-builder, and controller-util test files

Validation

  • Instance, legacy ITS, ITS2, shared instancestatus, and controllerutil test suites
  • full controllers/workloads envtest suite
  • pkg/operations/... and controllers/operations/...
  • make generate, make manifests, and make doc
  • generated Instance/InstanceSet CRDs, Helm CRDs, and API reference synchronized
  • all required PR checks passing at the current head

Fixes #10800.

Follow-up to #10758. Independent of #10760.

@leon-ape
leon-ape requested a review from a team as a code owner August 24, 2026 04:53
@apecloud-bot

Copy link
Copy Markdown
Collaborator

Auto Cherry-pick Instructions

Usage:
  - /nopick: Not auto cherry-pick when PR merged.
  - /pick: release-x.x [release-x.x]: Auto cherry-pick to the specified branch when PR merged.

Example:
  - /nopick
  - /pick release-1.1

CLA Recheck Instructions

Usage:
  - /recheck-cla: Trigger a re-check of CLA status for this pull request.
Example:
  - /recheck-cla

@github-actions github-actions Bot added the size/XXL Denotes a PR that changes 1000+ lines. label Aug 24, 2026
@codecov

codecov Bot commented Aug 24, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 78.57143% with 51 lines in your changes missing coverage. Please review.
✅ Project coverage is 65.62%. Comparing base (13138f4) to head (0bed8c9).

Files with missing lines Patch % Lines
pkg/controller/instanceset/reconciler_status.go 71.05% 14 Missing and 8 partials ⚠️
...ntroller/instanceset/reconciler_revision_update.go 76.47% 5 Missing and 3 partials ⚠️
pkg/controller/instance/reconciler_status.go 81.25% 3 Missing and 3 partials ⚠️
pkg/controller/instanceset2/reconciler_status.go 45.45% 4 Missing and 2 partials ⚠️
...troller/instanceset2/reconciler_revision_update.go 91.48% 2 Missing and 2 partials ⚠️
pkg/controller/instanceset2/reconciler_update.go 85.00% 2 Missing and 1 partial ⚠️
pkg/controller/instance/revision_utils.go 0.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #10799      +/-   ##
==========================================
+ Coverage   65.26%   65.62%   +0.36%     
==========================================
  Files         510      510              
  Lines       64016    64185     +169     
==========================================
+ Hits        41780    42122     +342     
+ Misses      18497    18318     -179     
- Partials     3739     3745       +6     
Flag Coverage Δ
unittests 65.62% <78.57%> (+0.36%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@leon-ape leon-ape added the nopick Not auto cherry-pick when PR merged label Aug 24, 2026
@leon-ape leon-ape changed the title fix(workloads): scope InstanceStatus invalidation fix(workloads): correct InstanceStatus convergence semantics Aug 25, 2026
@leon-ape leon-ape changed the title fix(workloads): correct InstanceStatus convergence semantics fix(its): correct InstanceStatus convergence semantics Aug 25, 2026
@leon-ape

Copy link
Copy Markdown
Collaborator Author

/approve

@apecloud-bot apecloud-bot added the approved PR Approved Test label Aug 25, 2026
@leon-ape
leon-ape merged commit 4180a61 into main Aug 25, 2026
37 checks passed
@leon-ape
leon-ape deleted the bugfix/10758-instance-status-contract branch August 25, 2026 09:59
@github-actions github-actions Bot added this to the Release 1.2.0 milestone Aug 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved PR Approved Test nopick Not auto cherry-pick when PR merged size/XXL Denotes a PR that changes 1000+ lines.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

InstanceStatus UpToDate can retain stale convergence across InstanceSet generations

2 participants