Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

kubeadm: handle dup unix:// prefix in node annotation #110656

Merged
merged 1 commit into from Jun 24, 2022
Merged

kubeadm: handle dup unix:// prefix in node annotation #110656

merged 1 commit into from Jun 24, 2022

Conversation

pacoxu
Copy link
Member

@pacoxu pacoxu commented Jun 20, 2022

What type of PR is this?

/kind bug

What this PR does / why we need it:

Which issue(s) this PR fixes:

xref #107295 & kubernetes/kubeadm#2426

Special notes for your reviewer:

Does this PR introduce a user-facing change?

kubeadm: handle dup `unix://` prefix in node annotation

@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. kind/bug Categorizes issue or PR as related to a bug. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. area/kubeadm sig/cluster-lifecycle Categorizes an issue or PR as relevant to SIG Cluster Lifecycle. and removed do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Jun 20, 2022
@pacoxu
Copy link
Member Author

pacoxu commented Jun 20, 2022

/cc @neolit123

@pacoxu pacoxu mentioned this pull request Jun 20, 2022
Merged
neolit123
neolit123 reviewed Jun 20, 2022
View reviewed changes
Copy link
Member

@neolit123 neolit123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can just remove all prefixes with strings.Replace and then re-add it once

@neolit123
Copy link
Member

/triage accepted
/priority important-soon

@k8s-ci-robot k8s-ci-robot added triage/accepted Indicates an issue or PR is ready to be actively worked on. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. and removed needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Jun 20, 2022
@pacoxu pacoxu force-pushed the kubeadm-dup-unix branch 2 times, most recently from 6febe31 to a74d7bc Compare June 21, 2022 03:03
neolit123
neolit123 reviewed Jun 21, 2022
View reviewed changes
cmd/kubeadm/app/cmd/phases/upgrade/node/kubeletconfig.go
Comment on lines +109 to +110
newSocket := strings.ReplaceAll(nro.CRISocket, kubeadmapiv1.DefaultContainerRuntimeURLScheme+"://", "")
newSocket = kubeadmapiv1.DefaultContainerRuntimeURLScheme + "://" + newSocket
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one potential problem here would be if the user provided socket path was not unix:// (linux) or npipe (win).

for example, user value:

URL://URL://path

would become:

DEFAULTURL://path

but i don't think we support anything other than unix:// (linux) or npipe (win), so it should be fine.

neolit123
neolit123 reviewed Jun 21, 2022
View reviewed changes
cmd/kubeadm/app/cmd/upgrade/common.go
dupURLScheme := strings.HasPrefix(cfg.NodeRegistration.CRISocket, kubeadmapiv1.DefaultContainerRuntimeURLScheme+"://"+kubeadmapiv1.DefaultContainerRuntimeURLScheme+"://")
if dupURLScheme {
socket := strings.ReplaceAll(cfg.NodeRegistration.CRISocket, kubeadmapiv1.DefaultContainerRuntimeURLScheme+"://", "")
cfg.NodeRegistration.CRISocket = kubeadmapiv1.DefaultContainerRuntimeURLScheme + "://" + socket
Copy link
Member

@neolit123 neolit123 Jun 21, 2022 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the logic in runKubeletConfigPhase() for upgrade node will write the fixed cri socket to remove dups.
but during apply are we going to fix it as well?
this cfg here might be only for local use and not updated in the cluster CM, but i could be wrong.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to my test, this will be called during kubeadm config image pull. To fix it during kubeadm config image pull would be werid, but it should be fixed.

  • The upgrade node will be fixed by the runKubeletConfigPhase().
  • upgrade apply will call PerformPostUpgradeTasks that will call AnnotateCRISocket to overwrite the annotation.

Thats why this can fix both.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to my test, this will be called during kubeadm config image pull. To fix it during kubeadm config image pull would be werid, but it should be fixed.

this here means the common enforceRequirements.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to my test, this will be called during kubeadm config image pull

hm, we should not mutate the CRI socket during actions that are not upgrade.

basically if the socket is corrupted (too many prefixes) upgrade should fix it early in the upgrade phase (enforceRequirements is a good location it seems)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated.

BTW

  • directly run kubeadm config image pull can success with dup unix:// prefix annotation on the node.
  • but kubeadm upgrade apply v1.xx will fail to run kubeadm config image pull
    • with this fix, we will update the annotation in enforceRequirements. This can be fixed.

neolit123
neolit123 reviewed Jun 24, 2022
View reviewed changes
Copy link
Member

@neolit123 neolit123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. I have not tested this, but please send additional changes if needed.

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 24, 2022
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: neolit123, pacoxu

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 24, 2022
@k8s-ci-robot k8s-ci-robot merged commit 34b4125 into kubernetes:master Jun 24, 2022
@k8s-ci-robot k8s-ci-robot added this to the v1.25 milestone Jun 24, 2022
@pacoxu pacoxu deleted the kubeadm-dup-unix branch June 24, 2022 07:36
@pacoxu pacoxu mentioned this pull request Aug 24, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@neolit123 neolit123 neolit123 left review comments

@fabriziopandini fabriziopandini Awaiting requested review from fabriziopandini

@RA489 RA489 Awaiting requested review from RA489

Assignees

@neolit123 neolit123

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/kubeadm cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. lgtm "Looks good to me", indicates that a PR is ready to be merged. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/cluster-lifecycle Categorizes an issue or PR as relevant to SIG Cluster Lifecycle. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
None yet
Milestone
v1.25
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@pacoxu @neolit123 @k8s-ci-robot