Skip to content

fix(kubernetes): recover BatchSandbox after Pod restart - #1523

Open
luppyw wants to merge 1 commit into
opensandbox-group:mainfrom
luppyw:fix/1497-batchsandbox-recovery
Open

fix(kubernetes): recover BatchSandbox after Pod restart#1523
luppyw wants to merge 1 commit into
opensandbox-group:mainfrom
luppyw:fix/1497-batchsandbox-recovery

Conversation

@luppyw

@luppyw luppyw commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Summary

Testing

  • Unit tests
  • Documentation build (cd docs && pnpm docs:build)
  • Integration tests (envtest asset download timed out against GitHub during make test)
  • e2e / manual verification (requires the Kind/Docker environment)

Breaking Changes

  • None
  • Yes (describe impact and migration path)

Checklist

  • Linked Issue or clearly described motivation
  • Added/updated docs (if needed)
  • Added/updated tests (if needed)
  • Security impact considered
  • Backward compatibility considered

Persist failed Pod UIDs so a recovered original Pod can clear a stale Failed phase without treating a replacement Pod as recovery.

Fixes opensandbox-group#1497
@github-actions github-actions Bot added component/k8s For kubernetes runtime documentation Improvements or additions to documentation size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Aug 14, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1e29c5ea28

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

}
}

status.FailedPodUIDs = nil

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Clear failedPodUIDs in the merge patch

When a failed pod recovers, setting FailedPodUIDs to nil is not enough to remove the persisted status field because updateStatus marshals the struct into a JSON merge patch and the field is tagged omitempty, so failedPodUIDs is omitted rather than sent as null. After recovery, the API server keeps the old UID list, the desired status keeps comparing unequal to the stored status, and the reconciler will keep trying to patch the same object instead of becoming idempotent.

AGENTS.md reference: kubernetes/AGENTS.md:L166-L166

Useful? React with 👍 / 👎.

// current Failed phase. The controller uses these UIDs to distinguish an
// in-place recovery from a replacement Pod that reuses the same name.
// +optional
FailedPodUIDs []types.UID `json:"failedPodUIDs,omitempty"`

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep the Helm CRD schema in sync

This status field is added to the Go type and the Kustomize CRD, but the bundled Helm CRD copy at kubernetes/charts/opensandbox-controller/templates/crds/batchsandboxes.yaml still lacks status.failedPodUIDs. In Helm-installed clusters, that unknown status field is pruned by the CRD schema, so the controller cannot persist pod UID provenance and the transient Pod recovery path remains ineffective for Helm users.

AGENTS.md reference: kubernetes/AGENTS.md:L164-L164

Useful? React with 👍 / 👎.

@luppyw

luppyw commented Aug 14, 2026

Copy link
Copy Markdown
Contributor Author

@Pangjiping Could you please review this PR when you have time? If the changes look correct and the required checks pass, would you be willing to merge it? I will address any requested changes promptly.

简体中文

@Pangjiping 方便时能否帮忙审核一下这个 PR?如果修改没有问题且必要检查通过,麻烦帮忙合并;如果需要调整,我会及时跟进。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

component/k8s For kubernetes runtime documentation Improvements or additions to documentation size/M Denotes a PR that changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BatchSandbox remains Failed after the same Pod recovers to Running and Ready

1 participant