Problem
A RebuildInstance OpsRequest using the in-place path (rebuildFrom[].inPlace: true) with a backupName that references a nonexistent Backup never fails: the ops controller retries the reconcile forever and the OpsRequest stays in Running.
Scenario
- User submits an in-place rebuild from a backup, but the backup name is typo'd, or
- the referenced Backup object was deleted after the OpsRequest was submitted.
Either way the Backup does not exist when the reconcile path resolves it — and it will never appear, so retrying cannot help.
Deterministic reproducibility
Webhook/validation does not guard this: validateRebuildInstance (pkg/operations/validation.go) only validates the component/instances, not the referenced backup. So a missing backup at reconcile time is a deterministic, permanent condition, not a transient one.
Reproduce
- Create a cluster with a component that supports rebuild.
- Create a
RebuildInstance OpsRequest with inPlace: true and backupName: does-not-exist.
- Observe the OpsRequest stays
Running forever; the controller keeps requeueing on a backups.dataprotection.kubeblocks.io "does-not-exist" not found error.
Framework mechanics / root cause
In pkg/operations/rebuild_instance.go, prepareInplaceRebuildHelper:
- the
cli.Get of the Backup named by rebuildFrom[].backupName returns the bare NotFound error (~L598);
- the ActionSet named by
backup.Status.BackupMethod.actionSetName is fetched via dputils.GetActionSetByName, which also returns the bare NotFound error (~L610).
In the operations framework, a plain error from ReconcileAction means "transient, requeue and retry"; only intctrlutil.NewFatalError marks the instance progress as Failed (see rebuildInstancesInPlace, which catches ErrorTypeFatal and sets FailedProgressStatus, but returns any other error for requeue). The adjacent validations in the same function (backup not Full/Incremental, backup not Completed, empty backupMethod) already return NewFatalError; only these two NotFound lookups leak through as retryable.
Relationship to #10510
#10510 fixed a different lookup in a different phase: the instance (pod) lookup in the Action phase. This issue is about the backup / actionset lookup in the reconcile phase (ReconcileAction → rebuildInstancesInPlace → prepareInplaceRebuildHelper), which #10510 did not touch.
Expected behavior
A nonexistent backup (or a deleted ActionSet referenced by the backup) should be classified as a fatal error so the affected instances are marked Failed and the OpsRequest transitions to Failed, instead of retrying forever.
Problem
A
RebuildInstanceOpsRequest using the in-place path (rebuildFrom[].inPlace: true) with abackupNamethat references a nonexistent Backup never fails: the ops controller retries the reconcile forever and the OpsRequest stays inRunning.Scenario
Either way the Backup does not exist when the reconcile path resolves it — and it will never appear, so retrying cannot help.
Deterministic reproducibility
Webhook/validation does not guard this:
validateRebuildInstance(pkg/operations/validation.go) only validates the component/instances, not the referenced backup. So a missing backup at reconcile time is a deterministic, permanent condition, not a transient one.Reproduce
RebuildInstanceOpsRequest withinPlace: trueandbackupName: does-not-exist.Runningforever; the controller keeps requeueing on abackups.dataprotection.kubeblocks.io "does-not-exist" not founderror.Framework mechanics / root cause
In
pkg/operations/rebuild_instance.go,prepareInplaceRebuildHelper:cli.Getof the Backup named byrebuildFrom[].backupNamereturns the bare NotFound error (~L598);backup.Status.BackupMethod.actionSetNameis fetched viadputils.GetActionSetByName, which also returns the bare NotFound error (~L610).In the operations framework, a plain error from
ReconcileActionmeans "transient, requeue and retry"; onlyintctrlutil.NewFatalErrormarks the instance progress as Failed (seerebuildInstancesInPlace, which catchesErrorTypeFataland setsFailedProgressStatus, but returns any other error for requeue). The adjacent validations in the same function (backup not Full/Incremental, backup not Completed, empty backupMethod) already returnNewFatalError; only these two NotFound lookups leak through as retryable.Relationship to #10510
#10510 fixed a different lookup in a different phase: the instance (pod) lookup in the
Actionphase. This issue is about the backup / actionset lookup in the reconcile phase (ReconcileAction→rebuildInstancesInPlace→prepareInplaceRebuildHelper), which #10510 did not touch.Expected behavior
A nonexistent backup (or a deleted ActionSet referenced by the backup) should be classified as a fatal error so the affected instances are marked Failed and the OpsRequest transitions to
Failed, instead of retrying forever.