Support ad-hoc CWL workflows by Nazim-crim · Pull Request #997 · crim-ca/weaver

Nazim-crim · 2026-07-14T21:04:13Z

https://crim-ca.atlassian.net/browse/NRCAN06-88

Add support for ad-hoc CWL workflow execution through the POST /jobs endpoint using multipart/mixed
or multipart/related content types. Users can now deploy and execute a Process in a single request by
submitting both the CWL workflow definition (with Content-Profile: http://www.opengis.net/def/ogcapi-processes/2.0/process-description)
and execution parameters (with Content-Profile: http://www.opengis.net/def/ogcapi-processes/2.0/execute)
in a multipart request body. The workflow is automatically deployed (with temporary ad-hoc tagging) before
execution, eliminating the need for separate deployment and execution steps for one-time workflow runs
resolves Support ad-hoc CWL workflows #834

codecov · 2026-07-15T19:22:18Z

Codecov Report

❌ Patch coverage is 89.13043% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.44%. Comparing base (874914a) to head (255d92e).
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
weaver/processes/utils.py	84.37%	1 Missing and 9 partials ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##           master     #997   +/-   ##
=======================================
  Coverage   88.44%   88.44%           
=======================================
  Files          88       88           
  Lines       20874    20961   +87     
  Branches     2760     2789   +29     
=======================================
+ Hits        18462    18540   +78     
  Misses       1726     1726           
- Partials      686      695    +9

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

fmigneault

Great PR! Thanks for the updates. Some semantic adjustment needed for the process description profile URI that is misused.

fmigneault · 2026-07-17T14:27:17Z

+.. _app_pkg_adhoc:
+
+Ad-hoc Workflow Execution
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


Can you move the section under processes.rst.
I think it would make more sense there in between "Execution KVP-encoded" and "Execution Steps".
For consistency, the title could be "Execution of Ad-hoc Workflow"

fmigneault · 2026-07-17T14:39:22Z

+Ad-hoc Workflow Execution
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+`Weaver` supports *ad-hoc* workflow execution, allowing users to deploy and execute a :term:`CWL` workflow in a single


Please add this:

# in references.rst (near corresponding definitions) .. |ogc-api-proc-part3-cwl-ad-hoc-exec| replace:: :term:`CWL` *ad-hoc* workflow execution .. _ogc-api-proc-part3-cwl-ad-hoc-exec: https://docs.ogc.org/DRAFTS/21-009.html#section_cwl_workflows

Then, reword as

`Weaver` supports |ogc-api-proc-part3-cwl-ad-hoc-exec|_ from the |ogc-api-proc-part3|_ standard, which allows ...

fmigneault · 2026-07-17T14:42:30Z

+1. One or more :term:`CWL` parts (workflow and any dependent tools) with
+   ``Content-Profile: http://www.opengis.net/def/ogcapi-processes/2.0/process-description``


The Content-Profile: http://www.opengis.net/def/ogcapi-processes/2.0/process-description header would apply to cases where additional process metadata is provided (i.e.: #990)

Here, the CWL parts just need to have the relevant application/cwl[+json|yaml].

Therefore, there could optionally be other "Process Description" parts using the indicated Content-Profile.

Consider adding a reference to proc_ogc_api_multi_cwl section by indicating that it follows the same concepts and procedure defined there.

fmigneault · 2026-07-17T14:43:40Z

+
+1. One or more :term:`CWL` parts (workflow and any dependent tools) with
+   ``Content-Profile: http://www.opengis.net/def/ogcapi-processes/2.0/process-description``
+2. One execution request part with ``Content-Profile: http://www.opengis.net/def/ogcapi-processes/2.0/execute``


Make sure to update the URIs with HTTPS.

fmigneault · 2026-07-17T14:49:20Z

+    Content-Profile: http://www.opengis.net/def/ogcapi-processes/2.0/process-description
+


To be removed since that is not the corresponding profile for the CWL part contents.

If the code already supports dealing with multiple application/json of different Content-Profile, you can move this under a separte section like:

--adhoc-boundary Content-Type: application/json Content-Profile: http://www.opengis.net/def/ogcapi-processes/2.0/process-description { "id": "adhoc-workflow", "description": "Example ad-hoc workflow deployed and executed inline." }

fmigneault · 2026-07-17T18:07:43Z

+    :param interpreted_parts: ``list`` of interpreted parts (content_type, content_id, content_location, profile, data)
+    :param root_workflow_cid: Content-ID of root workflow from ``start`` parameter (if any)


code quotes for Content-ID and the tuple

fmigneault · 2026-07-17T18:09:14Z

+        if profile == sd.OGC_API_PROC_PROFILE_EXECUTE_URI:
+            if execution_request is not None:


Planing ahead, the profile should be a list where matching must be "any of" those URI.

fmigneault · 2026-07-17T18:11:56Z

+            isinstance(part_data, dict) and "cwlVersion" in part_data and "$graph" in part_data
+        ):
+            deployment_parts.append((content_type_part, content_id, content_location, profile, part_data))
+        # JSON/YAML parts without profile could be execution request (fallback)


for clairity, indicate that the YAML is expecte to have been pre-converted to JSON

fmigneault · 2026-07-17T18:13:26Z

+            "title": "Missing CWL packages",
+            "description": (
+                "Multipart request must contain at least one CWL package part for ad-hoc workflow execution. "
+                f"Use Content-Profile: {sd.OGC_API_PROC_PROFILE_PROC_DESC_URI} header "


to adjust as per other comments
hint toward the CWL media-types instead

fmigneault · 2026-07-17T18:15:28Z

+    for _, content_id, _, _, part_data in deployment_parts:
+        if isinstance(part_data, dict):


Should there be an extra check of application/cwl[+json|+yaml] here?

add adhoc execution wip

f1d240b

Nazim-crim self-assigned this Jul 14, 2026

Nazim-crim added 3 commits July 14, 2026 17:25

Fix format

3ff5288

Fix lint

31656b0

Fix number of tuple for interpreted part

f763ab3

Nazim-crim added 3 commits July 16, 2026 11:33

Increase cov

1a2d1d1

Merge branch 'master' into ad-hoc-workflow

cc9020a

Update changelog

17f5750

Nazim-crim requested a review from fmigneault July 16, 2026 17:41

Nazim-crim marked this pull request as ready for review July 16, 2026 17:41

Add doc exemple for ad hoc execution

255d92e

fmigneault requested changes Jul 17, 2026

View reviewed changes

fmigneault mentioned this pull request Jul 18, 2026

Part 3: proposal for functional CWL ad-hoc execution with multipart opengeospatial/ogcapi-processes#607

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Support ad-hoc CWL workflows #997

Support ad-hoc CWL workflows #997
Nazim-crim wants to merge 8 commits into
masterfrom
ad-hoc-workflow

Nazim-crim commented Jul 14, 2026 •

edited by fmigneault

Loading

Uh oh!

codecov Bot commented Jul 15, 2026 •

edited

Loading

Uh oh!

fmigneault left a comment

Uh oh!

fmigneault Jul 17, 2026

Uh oh!

fmigneault Jul 17, 2026

Uh oh!

fmigneault Jul 17, 2026

Uh oh!

fmigneault Jul 17, 2026

Uh oh!

fmigneault Jul 17, 2026

Uh oh!

fmigneault Jul 17, 2026

Uh oh!

fmigneault Jul 17, 2026

Uh oh!

fmigneault Jul 17, 2026

Uh oh!

fmigneault Jul 17, 2026

Uh oh!

fmigneault Jul 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		1. One or more :term:`CWL` parts (workflow and any dependent tools) with
		``Content-Profile: http://www.opengis.net/def/ogcapi-processes/2.0/process-description``

		Content-Profile: http://www.opengis.net/def/ogcapi-processes/2.0/process-description

		:param interpreted_parts: ``list`` of interpreted parts (content_type, content_id, content_location, profile, data)
		:param root_workflow_cid: Content-ID of root workflow from ``start`` parameter (if any)

		if profile == sd.OGC_API_PROC_PROFILE_EXECUTE_URI:
		if execution_request is not None:

		for _, content_id, _, _, part_data in deployment_parts:
		if isinstance(part_data, dict):

Uh oh!

Conversation

Nazim-crim commented Jul 14, 2026 • edited by fmigneault Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov Bot commented Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

fmigneault left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Nazim-crim commented Jul 14, 2026 •

edited by fmigneault

Loading

codecov Bot commented Jul 15, 2026 •

edited

Loading