New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
dynamic resource allocation #111023
dynamic resource allocation #111023
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@pohly: 4 invalid OWNERS files
In response to this:
What type of PR is this?
/kind feature
/kind api-changeWhat this PR does / why we need it:
This implements the API and control plane part (kube-scheduler, kube-controller-manager, kube-apiserver, controller side of example driver) for Dynamic Resource Allocation.
Which issue(s) this PR fixes:
Related-to: kubernetes/enhancements#3063
Special notes for your reviewer:
This is using the revised API from pohly/enhancements#13. Before we move forward with the implementation, I would like to get confirmation from @alculquicondor and @thockin that this approach is okay and clarify a few remaining questions around the API. I'll add some of my own questions as PR comments.
We also need to decide how we want to merge this: everything in one big PR or split up along reasonable boundaries? Right now, the kubelet changes would be a good fit for a separate PR (different developer and reviewers, current functionality from this PR can be used without kubelet changes as the extended Pod spec will simply get ignored by kubelet).
Either way, the PR is not complete yet. The functionality for the control plane is all there, but tests and API validation are incomplete.
Does this PR introduce a user-facing change?
When the new `DynamicResourceAllocation` feature is enabled, additional resources can be requested for pods through the new ResourceClaim API and third-party resource drivers.
Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:
- [KEP]: https://github.com/kubernetes/enhancements/issues/3063 - [Usage]: TODO
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
bd80049
to
dc8b94b
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@pohly: 2 invalid OWNERS files
In response to this:
What type of PR is this?
/kind feature
/kind api-changeWhat this PR does / why we need it:
This implements the API and control plane part (kube-scheduler, kube-controller-manager, kube-apiserver, controller side of example driver) for Dynamic Resource Allocation.
Which issue(s) this PR fixes:
Related-to: kubernetes/enhancements#3063
Special notes for your reviewer:
This is using the revised API from pohly/enhancements#13. Before we move forward with the implementation, I would like to get confirmation from @alculquicondor and @thockin that this approach is okay and clarify a few remaining questions around the API. I'll add some of my own questions as PR comments.
We also need to decide how we want to merge this: everything in one big PR or split up along reasonable boundaries? Right now, the kubelet changes would be a good fit for a separate PR (different developer and reviewers, current functionality from this PR can be used without kubelet changes as the extended Pod spec will simply get ignored by kubelet).
Either way, the PR is not complete yet. The functionality for the control plane is all there, but tests and API validation are incomplete.
Does this PR introduce a user-facing change?
When the new `DynamicResourceAllocation` feature is enabled, additional resources can be requested for pods through the new ResourceClaim API and third-party resource drivers.
Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:
- [KEP]: https://github.com/kubernetes/enhancements/issues/3063 - [Usage]: TODO
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
dc8b94b
to
af09c21
Compare
@pohly: The following tests failed, say
Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here. |
The publishing bot tests want the package to exist in the merged branch before adding rules to publish it. |
/skip pull-publishing-bot-validate NOTE: this is by design, this CI job will fail in this PR as this PR is adding the new staging repo. |
/skip |
No particular benefit and no relevant changes, it's just to stay up-to-date and to avoid having to pull that in when merging kubernetes#111023 which indirectly depends on the newer release.
// +listType=set | ||
// +featureGate=DynamicResourceAllocation | ||
// +optional | ||
Claims []ResourceClaim `json:"claims,omitempty" protobuf:"bytes,3,opt,name=claims"` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We just ran across this while reviewing #107776
The ResourceRequirements
type is also used within the PersistentVolumeSpec type (which is also used indirectly in pod specs via ephemeral volumes), and the "clear if feature-gate is disabled" logic is only wired to podspec-handling. That means in 1.26 PVC objects or PodSpecs with ephemeral volumes defined can persist data in this field even if the feature is disabled.
Can you open a PR to clear this field in PVC and ephemeral volume create/update flows, and file a follow-up issue to consider splitting the type used in PVCs so it doesn't continue inheriting work done that is targeted at pods?
Have we decided whether to implement Topology support for DRA or not? @pohly |
With topology you mean sub-node topologuy like NUMA? This is out-of-scope for this KEP. It operates at the same level as current scheduler, which is nodes. However, there are some aspects of DRA that can be used to implement a controller which makes holistic decisions about all pending claims needed by a pod. See @klueska's presentations at KubeCon and the Slack discussions. |
What type of PR is this?
/kind feature
/kind api-change
What this PR does / why we need it:
This implements the API and control plane part (kube-scheduler, kube-controller-manager, kube-apiserver, controller side of example driver) for Dynamic Resource Allocation.
Which issue(s) this PR fixes:
Related-to: kubernetes/enhancements#3063
Special notes for your reviewer:
Let's try to gather who has reviewed what:
dependencies: @dims, @liggitt- no longer needed, dependencies were removedThis is using the revised API from kubernetes/enhancements#3502.
To try out the feature, build Kubernetes, then in one console run:
In another:
In yet another:
And finally:
Does this PR introduce a user-facing change?
Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.: