Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for slash as sysctl separator to Pod securityContext field and to PodSecurityPolicy #106834

Merged
merged 2 commits into from Jul 15, 2022

Conversation

mengjiao-liu
Copy link
Member

@mengjiao-liu mengjiao-liu commented Dec 6, 2021

What type of PR is this?

/kind feature

What this PR does / why we need it:

Starting from Kubernetes version 1.23, the kubelet supports the use of either / or .
as separators for sysctl names.
For example, you can represent the same sysctl name as kernel.shm_rmid_forced using a period as the separator, or as kernel/shm_rmid_forced using a slash as a separator.
For more sysctl parameter conversion method details, please refer to the page sysctl.d(5) from the Linux man-pages project.

In 1.25, use relaxed validation everywhere, in other words, Pod SecurityContext and PodSecurityPolicy can supports slash as sysctl separator. And I added the corresponding e2e test.

Ref #102393 (comment)

Which issue(s) this PR fixes:

Fixes #102373

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Pod SecurityContext and PodSecurityPolicy supports slash as sysctl separator.

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:


@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. kind/feature Categorizes issue or PR as related to a new feature. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. area/kubelet labels Dec 6, 2021
@k8s-ci-robot k8s-ci-robot added the sig/apps Categorizes an issue or PR as relevant to SIG Apps. label Dec 6, 2021
@k8s-ci-robot k8s-ci-robot added sig/auth Categorizes an issue or PR as relevant to SIG Auth. sig/node Categorizes an issue or PR as relevant to SIG Node. and removed do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Dec 6, 2021
@mengjiao-liu
Copy link
Member Author

/test pull-kubernetes-e2e-gce-ubuntu-containerd

@enj enj added this to Needs Triage in SIG Auth Old Dec 6, 2021
@ehashman ehashman added this to Waiting on Author in SIG Node PR Triage Dec 6, 2021
@k8s-ci-robot k8s-ci-robot added area/test sig/testing Categorizes an issue or PR as relevant to SIG Testing. labels Dec 7, 2021
@mengjiao-liu
Copy link
Member Author

/sig api-machinery

@k8s-ci-robot k8s-ci-robot added the sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. label Dec 7, 2021
@fedebongio
Copy link
Contributor

/remove-sig api-machinery

@k8s-ci-robot k8s-ci-robot removed the sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. label Dec 7, 2021
@mengjiao-liu mengjiao-liu force-pushed the sysctl-allow-slashes branch 2 times, most recently from 2d8bdca to c89d219 Compare December 8, 2021 02:28
@k8s-ci-robot k8s-ci-robot added area/conformance Issues or PRs related to kubernetes conformance tests sig/architecture Categorizes an issue or PR as relevant to SIG Architecture. labels Dec 8, 2021
@mengjiao-liu mengjiao-liu changed the title [WIP]Pod SecurityContext and PodSecurityPolicy supports slash as sysctl separator Pod SecurityContext and PodSecurityPolicy supports slash as sysctl separator Dec 8, 2021
@mengjiao-liu mengjiao-liu changed the title Pod SecurityContext and PodSecurityPolicy supports slash as sysctl separator Add support for slash as sysctl separator to Pod securityContext field and to PodSecurityPolicy Mar 29, 2022
@mengjiao-liu
Copy link
Member Author

Eg, you could /retitle Add support for slash as sysctl separator to Pod securityContext field and to PodSecurityPolicy

Thanks for your suggestion, it has been changed.

@mengjiao-liu
Copy link
Member Author

mengjiao-liu commented Mar 29, 2022

are there minimum kernel / container runtime / OS versions required for slash-delimited sysctl support?

There are few articles on the Internet about linux sysctl separators, so I also checked the source code of the linux kernel part to verify.

linux kernel support sysctl:
sysctl appeared in 1.3.57 and is fully supported ever since.

/proc/sys
              This directory (present since 1.3.57) contains a number of
              files and subdirectories corresponding to kernel
              variables.  These variables can be read and in some cases
              modified using the /proc filesystem, and the (deprecated)
              [sysctl(2)](https://man7.org/linux/man-pages/man2/sysctl.2.html) system call.

              String values may be terminated by either '\0' or '\n'.

              Integer and long values may be written either in decimal
              or in hexadecimal notation (e.g., 0x3FFF).  When writing
              multiple integer or long values, these may be separated by
              any of the following whitespace characters: ' ', '\t', or
              '\n'.  Using other separators leads to the error EINVAL.

ref:

The sysctl also uses the conversion of slashes and dot separator ,at least 10 years ago. I found the source code of procps-ng Sysctl, the earliest version is 3.3.0 I can found, and the conversion of . and / has been implemented at this time, at least 10 years ago(Maybe earlier, but I can't find the source code further forward).

if so, what is the impact of using a slash-delimited sysctl on a node/kernel/os/container runtime that doesn't support slash-delimited sysctls?
is the impact limited to an error running the container, or can it cause problems beyond the scope of that container?

I think that the kernel versions generally used now are greater than version 1.35.7, and the bottom layer of the kernel supports sysctl including slashes.

As far as my own understanding is concerned, the linux kernel finally needs to convert dots into slashes to find files under /proc/sys, so it must support slashes.

The runc code opencontainers/runc@f7d1401 only adds namespace verification to sysctl, and then uses syscall to call sysctl in 2015.

The runc code just adds namespace verification to sysctl, and then calls sysctl.

runc 1.1.0 allow slashes in sysctl names, to better match sysctl(8)'s behaviour. (opencontainers/runc#3254, opencontainers/runc#3257)

In order to be compatible with runc which does not support slashes to separate namespaces and keys, I have converted the slashes into dot separators before the kubelet calls runc.

sysctl.ConvertPodSysctlsVariableToDotsSeparator(pod.Spec.SecurityContext)

So I don't think it has effect on k8s already using sysctl functionality.
However, I am not very familiar with the linux kernel source code, maybe someone who is more familiar with this part of the code can help to confirm it again? @liggitt

@k8s-ci-robot k8s-ci-robot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Mar 30, 2022
@enj
Copy link
Member

enj commented Jun 13, 2022

/milestone v1.25

@k8s-ci-robot k8s-ci-robot added this to the v1.25 milestone Jun 13, 2022
@enj enj moved this from Needs Triage to Pending other SIGs in SIG Auth Old Jun 13, 2022
@pacoxu pacoxu moved this from Waiting on Author to Needs Reviewer in SIG Node PR Triage Jun 20, 2022
@mengjiao-liu
Copy link
Member Author

/retest

@pacoxu pacoxu moved this from Needs Reviewer to Done in SIG Node PR Triage Jun 30, 2022
@pacoxu
Copy link
Member

pacoxu commented Jun 30, 2022

/lgtm
move to Done in sig-node board.

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 30, 2022
@mengjiao-liu
Copy link
Member Author

Please review again. @liggitt @lavalamp

Copy link
Member

@liggitt liggitt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/approve
for API changes

/hold for one comment on the e2e test

test/e2e/common/node/sysctl.go Show resolved Hide resolved
@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jul 14, 2022
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: liggitt, mengjiao-liu, mrunalp

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 14, 2022
@liggitt
Copy link
Member

liggitt commented Jul 15, 2022

/hold cancel

@k8s-ci-robot k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jul 15, 2022
@mengjiao-liu
Copy link
Member Author

/test pull-kubernetes-e2e-kind

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/conformance Issues or PRs related to kubernetes conformance tests area/kubelet area/test cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/feature Categorizes issue or PR as related to a new feature. lgtm "Looks good to me", indicates that a PR is ready to be merged. priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/apps Categorizes an issue or PR as relevant to SIG Apps. sig/architecture Categorizes an issue or PR as relevant to SIG Architecture. sig/auth Categorizes an issue or PR as relevant to SIG Auth. sig/node Categorizes an issue or PR as relevant to SIG Node. sig/security Categorizes an issue or PR as relevant to SIG Security. sig/testing Categorizes an issue or PR as relevant to SIG Testing. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
Status: API review completed, 1.24
Archived in project
Archived in project
SIG Auth Old
Closed / Done
Development

Successfully merging this pull request may close these issues.

Regex for validating Sysctl values in deployments might not be correct.