Fix generation and self-healing bugs found in hands-on testing

[binod-adhikari]

Summary

I ran generate-workflow end-to-end against a real site (an apartment listing site's apply-now flow, with a dismissible popup and a nav-driven multi-page task) using Anthropic models, then replayed the generated workflow. This surfaced three bugs blocking the two features that matter most for this project's value proposition — semantic workflow generation and self-healing replay — plus two smaller bugs in dead code that only appeared once the first fix let it actually run.

• workflow_creation_prompt.md: an unescaped literal {variable} in prose (line 26) broke prompt_content.format(goal=..., actions=...) in HealingService.create_workflow_definition with KeyError: 'variable', since str.format() treats any single-brace token as a substitution placeholder, not just the intentional {goal}/{actions}. Escaped it to {{variable}} to match the rest of the template.

• healing/service.py: _history_to_workflow_definition always tags the agent's screenshot as image/jpeg regardless of actual format, so Anthropic rejects it: messages.1.content.1.image.source.base64: The image was specified using the image/jpeg media type, but the image appears to be a image/png image. browser-use's get_screenshot() returns PNG; fixed the hardcoded MIME type to match.

• workflow/service.py: _fallback_to_agent — the self-healing path — was fully implemented but entirely commented out, so any failed deterministic step (e.g. a selector that no longer matches after a page changes) crashed the whole run instead of healing, even though the surrounding log messages ("Attempting fallback with agent") implied it was active. Re-enabled it, and fixed two bugs in that dead code that only surfaced once it actually executed:

  • _run_agent_step(agent_step_config) was missing the required step_index argument.
  • self.steps doesn't exist on Workflow (the attribute is self.schema.steps, per the comment a few lines away at for step_index, step_dict in enumerate(self.schema.steps): # self.steps now holds dictionaries).

Result after all four fixes

Generated a workflow from a live multi-step task (navigate → dismiss popup → click nav → scroll → extract). On replay, two of the five steps' selectors didn't match (expected — they were generated from one page load and matched against a fresh one) and both healed successfully through two different paths: the agent-based fallback fixed here handled one, and the existing semantic-executor hierarchical-selector fallback handled the other. The workflow completed end to end with correct extracted data.

Test plan

• Ran generate-workflow against a live site with ChatAnthropic as agent/extraction/workflow LLM — completes without the KeyError or image MIME error.
• Ran Workflow.load_from_file(...).run(...) against the generated workflow — a deterministic step with a stale selector now heals via _fallback_to_agent instead of crashing, and the run completes successfully.
• No existing automated tests were run against this change (couldn't find a test suite covering workflow/service.py's replay path) — happy to add coverage if there's a preferred pattern for it in this repo.

🤖 Generated with Claude Code

---

Summary by cubic

Fixes four bugs that blocked semantic workflow generation and self-healing replay in end-to-end testing. Failed deterministic steps previously crashed the entire run; they now fall back to the agent and heal.

Bug Fixes

• Escaped a literal {variable} in workflow_creation_prompt.md so str.format() no longer raises KeyError: 'variable' during workflow generation.
• Uses image/png instead of a hardcoded image/jpeg MIME type for screenshots, which Anthropic previously rejected as a format mismatch.
• Re-enables _fallback_to_agent in workflow/service.py, which was fully implemented but commented out.
• Fixes two latent bugs in the fallback path: passes the required step_index to _run_agent_step, and references self.schema.steps instead of the non-existent self.steps.

After these fixes, a workflow generated from a live multi-step task replayed end to end with both stale selectors healing successfully via the agent fallback and the existing semantic-executor fallback.

^{Written for commit be4620f. Summary will update on new commits.}

[Copilot]

Pull request overview

Fixes multiple issues that prevented end-to-end workflow generation and self-healing replay from working reliably (prompt templating crash, Anthropic image MIME mismatch, and an inactive/buggy agent fallback path during deterministic step failures).

Changes:

• Escapes a literal {variable} token in the workflow-creation prompt template to avoid str.format() KeyError.
• Corrects screenshot data URL MIME type to image/png for Anthropic compatibility.
• Re-enables and fixes the _fallback_to_agent self-healing path, wiring it into deterministic step failure handling.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

File	Description
workflows/workflow_use/workflow/service.py	Re-enables agent fallback for failed deterministic steps and restores previously-dead fallback code paths.
workflows/workflow_use/healing/service.py	Updates screenshot MIME type used in prompt construction for Anthropic image validation.
workflows/workflow_use/healing/prompts/workflow_creation_prompt.md	Escapes braces in prompt prose to prevent str.format() crashes during workflow creation.

Suppressed comments (1)

workflows/workflow_use/workflow/service.py:699

• Same as above: this warning always claims an agent fallback will be attempted, but the code only does so when self.fallback_to_agent is true. Consider making the message conditional so logs accurately reflect behavior.

					action_name = step_resolved.type or '[Unknown Action]'
					logger.warning(
						f'Deterministic step {step_index + 1} ({action_name}) failed: {e}. Attempting fallback with agent.'
					)

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[cubic-dev-ai]

3 issues found across 3 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="workflows/workflow_use/workflow/service.py">

<violation number="1" location="workflows/workflow_use/workflow/service.py:488">
P2: When an input step contains a credential or other runtime secret, this info log records it in `fallback_task` through `failed_params`. Log only step metadata and redact action parameters instead.</violation>

<violation number="2" location="workflows/workflow_use/workflow/service.py:682">
P1: When the action succeeds but the next-step readiness check fails, this fallback replays the current action. Separate post-action readiness errors from action errors before invoking `_fallback_to_agent`, or a click/input can be performed twice.</violation>
</file>

<file name="workflows/workflow_use/healing/prompts/workflow_creation_prompt.md">

<violation number="1" location="workflows/workflow_use/healing/prompts/workflow_creation_prompt.md:29">
P3: Correct this explanation: `str.format()` treats `{{` and `}}` as escaped literal braces, so `{{variable}}` renders `{variable}` instead of substituting a value.</violation>
</file>

<sub>Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic</sub>

[cubic-dev-ai]

P1: When the action succeeds but the next-step readiness check fails, this fallback replays the current action. Separate post-action readiness errors from action errors before invoking _fallback_to_agent, or a click/input can be performed twice.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At workflows/workflow_use/workflow/service.py, line 682:

<comment>When the action succeeds but the next-step readiness check fails, this fallback replays the current action. Separate post-action readiness errors from action errors before invoking `_fallback_to_agent`, or a click/input can be performed twice.</comment>

<file context>
@@ -672,7 +678,12 @@ async def _execute_step(self, step_index: int, step_resolved: WorkflowStep) -> A
 						)
-						raise ValueError(f'Deterministic step {step_index + 1} ({action_name}) failed: {e}')
+						if self.fallback_to_agent:
+							result = await self._fallback_to_agent(step_resolved, step_index, e)
+							if not result.is_successful():
+								raise ValueError(f'Deterministic step {step_index + 1} ({action_name}) failed even after fallback')
</file context>

[cubic-dev-ai]

P2: When an input step contains a credential or other runtime secret, this info log records it in fallback_task through failed_params. Log only step metadata and redact action parameters instead.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At workflows/workflow_use/workflow/service.py, line 488:

<comment>When an input step contains a credential or other runtime secret, this info log records it in `fallback_task` through `failed_params`. Log only step metadata and redact action parameters instead.</comment>

<file context>
@@ -418,79 +428,75 @@ async def _run_extraction_step(self, step, step_index: int) -> ActionResult:
+			failed_value=failed_value,
+			step_description=step_description,
+		)
+		logger.info(f'Agent fallback task: {fallback_task}')
+
+		# Prepare agent step config based on the failed step, adding task
</file context>

[cubic-dev-ai]

P3: Correct this explanation: str.format() treats {{ and }} as escaped literal braces, so {{variable}} renders {variable} instead of substituting a value.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At workflows/workflow_use/healing/prompts/workflow_creation_prompt.md, line 29:

<comment>Correct this explanation: `str.format()` treats `{{` and `}}` as escaped literal braces, so `{{variable}}` renders `{variable}` instead of substituting a value.</comment>

<file context>
@@ -23,10 +23,10 @@ You are a master at building re-executable workflows from browser automation ste
    - ✅ CORRECT: `"value": "{{email}}"` or `"target_text": "{{repo_name}}"`
    - ❌ WRONG: `"value": "{{{{email}}}}"` or `"value": "email"`
-   - Python's str.format() substitutes {variable} with actual values at runtime
+   - Python's str.format() substitutes {{variable}} with actual values at runtime
 
 5. **Prefer direct navigation over search engines!**
</file context>

Suggested change

	- Python's str.format() substitutes {{variable}} with actual values at runtime
	- Python's str.format() treats `{{` and `}}` as escaped literal braces, so `{{variable}}` renders `{variable}` instead of substituting a value