Skip to content

Support for resource pools v2 #4977

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
stefannica wants to merge 41 commits into
base: develop
Choose a base branch
from
+10,218 −6,925
Open

Support for resource pools v2 #4977

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
6b04525
Implement resource pools v2 API and resource manager backend
stefannica May 27, 2026
bc309fe
Fixed bugs and introduced pydantic settings for the RM store
stefannica May 29, 2026
1a7736e
Reworked resource settings and component ID resource request references
stefannica May 29, 2026
bca402d
Remove resource subjects, fix some bugs
stefannica May 31, 2026
e44d80c
Add metadata and timestamps to some resource pool related models
stefannica Jun 1, 2026
38dfa6d
Support filtering resource requests by multiple status values
stefannica Jun 1, 2026
ca4f561
Fix DB migration order
stefannica Jun 2, 2026
67a3c02
Introduced resource request terminate endpoint
stefannica Jun 2, 2026
0188dcc
Rework the dynamic orchestrator and heartbeat to wire in resource req…
stefannica Jun 4, 2026
eeb64f9
Restructured the RM store and formalized user-owned requests
stefannica Jun 7, 2026
c1f7ae8
Wire resource allocations into orchestrators and step operators
stefannica Jun 8, 2026
f499836
Update docs
stefannica Jun 8, 2026
c93b852
Docs updates
stefannica Jun 8, 2026
dd64b77
Update docs
stefannica Jun 10, 2026
3eeeff1
Support multiple subjects for the same policy
stefannica Jun 12, 2026
4af5afa
Add teams support to resource pools
stefannica Jun 15, 2026
758eb62
Attack user/owner to resource pool entities
stefannica Jun 15, 2026
c8f0e62
Removed legacy resource pools support and updated to external RM
stefannica Jun 22, 2026
d4ecad3
Add new tests
stefannica Jun 24, 2026
98db173
Skip org from user account subject description
stefannica Jun 25, 2026
4db9b8e
Add configurable time based waiting in resource pools
stefannica Jun 25, 2026
d1c3173
Add pipeline run as preemption group to all resource requests
stefannica Jun 25, 2026
d94f638
Add updated support for resource request settings
stefannica Jun 29, 2026
a2b8ab9
Service connector overrides for step operators
stefannica Jun 30, 2026
a637552
Add support for suspended k8s jobs during provisioning
stefannica Jul 20, 2026
f866c9e
Set runtime state to resource requests
stefannica Jul 15, 2026
edb2020
Complete multi-step operator selection
stefannica Jul 20, 2026
e77356a
Fix docstrings
stefannica Jul 20, 2026
88782a0
Merge and reorder DB migration scripts and fix linter errors
stefannica Jul 20, 2026
b4951f5
Update RM API to cover resource group classes
stefannica Jul 20, 2026
69519ff
Fix resource request listing
stefannica Jul 20, 2026
cf00f99
Small fixes and code cleanup
stefannica Jul 21, 2026
448c4bf
Rework the resource request lease renewal based on the heartbeat
stefannica Jul 21, 2026
ab14a88
Remove step_run as a resource type
stefannica Jul 21, 2026
1bfc43f
Update docs
stefannica Jul 24, 2026
c35398a
Reduce slop and fix spelling
stefannica Jul 24, 2026
cdc3742
Revert dynamic step operator, service connector and pending job support
stefannica Jul 24, 2026
1be47fd
Remove info log
stefannica Jul 24, 2026
e584a95
Rework allocation aware orchestrator and step operator methods
stefannica Jul 24, 2026
cd2bed4
Reverted more code
stefannica Jul 24, 2026
d7e7acd
Fix spelling
stefannica Jul 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .typos.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ registr = "registr"
tokenzation = "tokenzation"
Conveyer = "Conveyer"
froms = "froms" # used by sqlalchemy in a method name
queueing = "queueing"

[default]
locale = "en-us"
6 changes: 4 additions & 2 deletions docs/book/getting-started/zenml-pro/deploy-workspace-resource-pools.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,9 @@ helm upgrade zenml oci://public.ecr.aws/zenml/zenml \

## Related resources

- [Resource pools](resource-pools.md) — product overview and concepts
- [Resource pools — Core concepts](resource-pools-core-concepts.md)
- [Resource pools](resource-pools.md) — product overview
- [Core concepts](resource-pools-core-concepts.md)
- [Admin guide](resource-pools-admin-guide.md)
- [User guide](resource-pools-user-guide.md)
- [Self-hosted Deployment on Kubernetes with Helm](deploy-workspace-k8s.md)
- [Helm chart on Artifact Hub](https://artifacthub.io/packages/helm/zenml/zenml)
Loading
Loading