Honour process.resourceLabels in nf-seqera executor#7048
Merged
pditommaso merged 16 commits intomasterfrom Apr 20, 2026
Merged
Conversation
Spec for honouring process.resourceLabels in nf-seqera with cumulative semantics: config-level baseline on the Sched run, per-task delta on Sched task. Removes the now-redundant seqera.executor.labels option. Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
Task-by-task TDD plan implementing the spec from docs/superpowers/specs/2026-04-17-seqera-resource-labels-design.md. Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
…ess.resourceLabels Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
Picks up the published artifact so the build resolves without the includeBuild composite. The transitive sched-api 0.52.0-SNAPSHOT is now also published. Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
✅ Deploy Preview for nextflow-docs-staging ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
- Labels.toStringMap now accepts Object and throws IllegalArgumentException when the value is not a Map, giving a clear error when process.resourceLabels is misconfigured (e.g. as a list). - SeqeraExecutor.getRunResourceLabels wraps the cached map in Collections.unmodifiableMap, matching Labels.getEntries. Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
3 tasks
Member
|
Don't forget to remove |
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
Member
Author
|
Well spotted, done |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Make the
nf-seqeraexecutor honour the standard Nextflowprocess.resourceLabelsdirective with cumulative semantics:process.resourceLabelsbaseline is attached once to the Sched run viaCreateRunRequest.labels.task.config.getResourceLabels()and the run baseline, on the newTask.labelsfield. Identical key/value entries are stripped; differing or new entries are sent so the scheduler's task-over-run merge gives the correct final values.seqera.executor.labelsconfig option (breaking change for pluginnf-seqera, replaced entirely byprocess.resourceLabels).This fixes a gap where the
process.resourceLabelsdirective — emitted byAbstractComputePlatformProvider.addConfigResourceLabels()for every CE type and honoured by AWS Batch / GCP Batch / Azure / K8s — was effectively dead on the Seqera Compute path.Motivation
SeqeraTaskHandler.submit()previously built the schedulerTaskwith onlyname,image,command,environment,resourceRequirement,resourceLimit,machineRequirement, andnextflow(taskId/hash/workDir)— it never readtask.config.getResourceLabels(). The plugin's only label path was at the run level (seqera.executor.labels+autoLabels), so per-process labels emitted via the standard directive were silently dropped on Seqera Compute while working on every other supported executor.Cumulative model rationale: per-task scheduler payloads stay small (only the delta is sent); same-key collisions between the config baseline and a per-process override resolve naturally on the scheduler side (task labels override run labels), preserving Nextflow's existing directive semantics.
Changes
Plugin (
plugins/nf-seqera)Labels.withProcessResourceLabels(Map<String,?>)— new instance method that adds config-level labels to the run-label map, coercing values viaString.valueOfto satisfysched-api'sMap<String,String>typing.Labels.toStringMap(Map<String,?>)— static helper returning a coerced map (orCollections.emptyMap()for null/empty input).Labels.delta(task, run)— static helper returning entries intaskthat are missing fromrunor have a different value. Returnsnullwhen the result would be empty (so callers can omit the field).SeqeraExecutor:runResourceLabels(initialised to empty map);@PackageScope void computeRunResourceLabels()readssession.config.process.resourceLabelsand stores the coerced baseline;getRunResourceLabels();createRun()invokescomputeRunResourceLabels()and callslabels.withProcessResourceLabels(runResourceLabels)after the autoLabels block.SeqeraTaskHandler.submit()computesLabels.delta(Labels.toStringMap(task.config.getResourceLabels()), executor.runResourceLabels)and attaches it toTask.labelsonly when non-empty.seqera.executor.labelsconfig option,ExecutorOpts.labelsfield/getter, andLabels.withUserLabels(Map)removed; obsolete tests dropped fromExecutorOptsTest,LabelsTest, andSeqeraConfigTest.VERSIONbumped to0.18.0;changelog.txtupdated.sched-clientbumped to0.52.0-SNAPSHOT(which exposesTask.labels).Tests (Spock)
LabelsTestcoverswithProcessResourceLabels(coercion, null/empty, override semantics) plustoStringMapanddelta(null/empty handling, full-when-empty-baseline, missing/differing-only deltas).SeqeraExecutorTestadds:computeRunResourceLabels()populating the cached map, including non-String value coercion;createRun()test assertingCreateRunRequest.labelscontains both the config-levelprocess.resourceLabelsentries and thenextflow.io/*auto-label entries whenautoLabelsis enabled.SeqeraTaskHandlerTest:submit()attachesTask.labelscontaining only the delta and leavesTask.labelsunset when task labels equal the run baseline;Mock(SeqeraExecutor)blocks tightened withgetRunResourceLabels() >> [:]to reflect the runtime invariant.Docs
docs/reference/process.mdaddsseqera-executorto theresourceLabelssupport list.Breaking change
The
seqera.executor.labelsconfig option is removed. Users on pluginnf-seqera0.17.x must migrate to the standard Nextflow directive:The directive may be set at the config level (applied to all processes), under
withName:/withLabel:selectors, or inside individualprocess {}bodies — all forms accumulate per Nextflow's normal merging rules. The config-level entries become the run baseline; selector/process-body entries become per-task deltas.Test plan
./gradlew :plugins:nf-seqera:test— greendocs/superpowers/specs/2026-04-17-seqera-resource-labels-design.md)sched-client:0.52.0-SNAPSHOTartifact resolves from the published Maven repoprocess.resourceLabelsbaseline plus per-process directives, verifying the resulting run labels and per-task labels in the schedulerNotes
docs/superpowers/{specs,plans}/2026-04-17-seqera-resource-labels-*.mdfor review traceability.settings.gradleretains an opt-in commentedincludeBuild '../sched'hint for local development against an unreleased sched-client; CI builds use the published Maven artifact.