Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 11 additions & 4 deletions agents/code.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,10 +84,14 @@ the review agent — if the triage was wrong, your code will fail review.

You MUST produce a JSON file at `$FULLSEND_OUTPUT_DIR/agent-result.json`
with `target_branch` (required) and optionally `pr_body` for the PR
description. The `code-implementation` skill describes the schema and
the exact steps where you write each field. The post-script reads this
file to determine the PR target branch and description. Without this
file, the validation loop rejects the run and retries.
description. When the agent cannot proceed without human intervention,
set `needs_input` to `true` and provide a `needs_input_reason` string
explaining the blocker. The `code-implementation` skill describes the
schema and the exact steps where you write each field. The post-script
reads this file to determine the PR target branch and description, or
to apply the `fs-code-needs-input` label when the agent signals it
needs help. Without this file, the validation loop rejects the run and
retries.

After writing the file, validate it before exiting:

Expand All @@ -109,6 +113,9 @@ Your exit state is the handoff contract:
- **Clean commit on the feature branch + valid structured output** → the
post-script pushes and creates the PR (after its own authoritative secret
scan).
- **`needs_input: true` in structured output** → the post-script applies the
`fs-code-needs-input` label to the issue, posts a comment with the reason,
and exits without creating a PR. No code changes are expected.
- **No commit** → the post-script reads your transcript and exit code to
report the failure. Structured output should still be written when possible
so the post-script knows which branch was targeted.
Expand Down
1 change: 1 addition & 0 deletions docs/code.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ on issues (not PRs).
|-------|---------|
| `ready-to-code` | Triggers the code agent. Applied by the [triage](triage.md) agent for low-risk categories (bug, documentation, performance), or manually by a human for feature work after prioritization. Not applied when the triage result sets `requires_workflow_changes`, since the code agent cannot modify workflow files. |
| `ready-for-review` | Applied by the code agent after pushing a PR. In per-repo installs, triggers the [review agent](review.md) when applied to a PR. Also marks workflow state for humans and the [retro agent](retro.md). |
| `fs-code-needs-input` | Applied by the post-script when the code agent signals `needs_input: true`. Indicates the agent cannot proceed without human intervention. The post-script also posts a comment with the specific blocker. Retry with `/fs-code` after addressing the blocker. |

## Configuration

Expand Down
23 changes: 22 additions & 1 deletion schemas/code-result.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,27 @@
"type": "boolean",
"default": true,
"description": "Whether the PR should close the linked issue on merge. Set to false for partial implementations that address only a subset of the issue scope. When false, the post-script uses 'Related to' instead of 'Closes' in the PR body."
},
"needs_input": {
"type": "boolean",
"default": false,
"description": "When true, the agent is signaling that it cannot proceed without human input. The post-script applies the fs-code-needs-input label to the issue, posts a comment with the reason, and skips PR creation. No code changes are expected."
},
"needs_input_reason": {
"type": "string",
"maxLength": 2000,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[medium] api-contract

The schema description for needs_input_reason states 'Required when needs_input is true', but the JSON Schema has no if/then conditional to enforce this constraint. The triage-result.schema.json uses allOf/if/then for identical conditional requirement patterns. The post-script compensates with a fallback ('No reason provided').

Suggested fix: Add a conditional requirement using JSON Schema if/then, or change the description from 'Required' to 'Recommended'.

"description": "Human-readable explanation of why the agent needs input. Required when needs_input is true. Describes the specific blocker (broken tooling, uninterpretable issue, missing scan-secrets, etc.)."
}
},
"allOf": [
{
"if": {
"properties": { "needs_input": { "const": true } },
"required": ["needs_input"]
},
"then": {
"required": ["needs_input_reason"]
}
}
}
]
}
194 changes: 194 additions & 0 deletions scripts/post-code-test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -1436,6 +1436,200 @@ else
echo "PASS: script-has-noop-comment"
fi

# ---------------------------------------------------------------------------
# Test: needs_input handling — verify the post-script contains the
# needs_input detection logic and label/comment infrastructure.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[low] test-adequacy

Tests reimplement detection and comment logic in local helper functions. No integration test for the full needs_input path (result file to label created to comment posted to exit 0 without PR).

# ---------------------------------------------------------------------------

# Verify needs_input detection is present in the post-code script
if ! grep -q 'needs_input' "${POST_SCRIPT}"; then
echo "FAIL: script-has-needs-input"
echo " ${POST_SCRIPT} missing needs_input handling"
FAILURES=$((FAILURES + 1))
else
echo "PASS: script-has-needs-input"
fi

# Verify fs-code-needs-input label is created
if ! grep -q 'fs-code-needs-input' "${POST_SCRIPT}"; then
echo "FAIL: script-has-needs-input-label"
echo " ${POST_SCRIPT} missing fs-code-needs-input label"
FAILURES=$((FAILURES + 1))
else
echo "PASS: script-has-needs-input-label"
fi

# ---------------------------------------------------------------------------
# Test helper — reimplements the needs_input detection logic from
# post-code.src.sh so we can test it without a git repo or network access.
# ---------------------------------------------------------------------------
detect_needs_input() {
local result_json="$1"

local needs_input
needs_input="$(echo "${result_json}" | jq -r '.needs_input // false' 2>/dev/null || echo "false")"

if [ "${needs_input}" = "true" ]; then
local reason
reason="$(echo "${result_json}" | jq -r '.needs_input_reason // "No reason provided"' 2>/dev/null || echo "No reason provided")"
echo "needs_input:${reason}"
else
echo "proceed"
fi
}

run_needs_input_test() {
local test_name="$1"
local result_json="$2"
local expected_prefix="$3"

local actual
actual="$(detect_needs_input "${result_json}")"

if [[ "${actual}" != ${expected_prefix}* ]]; then
echo "FAIL: ${test_name}"
echo " result_json: '${result_json}'"
echo " expected prefix: '${expected_prefix}'"
echo " actual: '${actual}'"
FAILURES=$((FAILURES + 1))
return
fi

echo "PASS: ${test_name}"
}

# --- needs_input detection test cases ---

# needs_input=true with reason → signal needs_input
run_needs_input_test "needs-input-true-with-reason" \
'{"target_branch":"main","needs_input":true,"needs_input_reason":"Build tooling is broken"}' \
"needs_input:Build tooling is broken"

# needs_input=true without reason → signal needs_input with default
run_needs_input_test "needs-input-true-no-reason" \
'{"target_branch":"main","needs_input":true}' \
"needs_input:No reason provided"

# needs_input=false → proceed normally
run_needs_input_test "needs-input-false" \
'{"target_branch":"main","needs_input":false}' \
"proceed"

# needs_input absent → proceed normally
run_needs_input_test "needs-input-absent" \
'{"target_branch":"main"}' \
"proceed"

# needs_input=true with detailed reason → signal with full reason
run_needs_input_test "needs-input-scan-secrets-missing" \
'{"target_branch":"main","needs_input":true,"needs_input_reason":"scan-secrets is not installed in the sandbox image"}' \
"needs_input:scan-secrets is not installed in the sandbox image"

# needs_input=true for uninterpretable issue
run_needs_input_test "needs-input-uninterpretable" \
'{"target_branch":"main","needs_input":true,"needs_input_reason":"Issue description is not interpretable: no clear problem statement or actionable scope"}' \
"needs_input:Issue description is not interpretable"

# ---------------------------------------------------------------------------
# Test helper — reimplements the needs_input comment body construction from
# post-code.src.sh so we can test it without a GitHub API.
# ---------------------------------------------------------------------------
build_needs_input_comment() {
local reason="$1"
local issue_number="$2"
local repo_full_name="$3"

cat <<EOF
🛑 **Needs human input** — code agent cannot proceed

The code agent evaluated issue #${issue_number} but determined it cannot make progress without human intervention.

**Reason:** ${reason}

**Workflow run:** https://github.com/${repo_full_name}/actions/runs/unknown

Please address the blocker above, then retry with \`/fs-code\`.
EOF
}

run_needs_input_comment_test() {
local test_name="$1"
local reason="$2"
local issue_number="$3"
local repo_full_name="$4"
local check_pattern="$5"
local expect_present="$6" # "yes" or "no"

local actual
actual="$(build_needs_input_comment "${reason}" "${issue_number}" "${repo_full_name}")"

if [ "${expect_present}" = "yes" ]; then
if ! echo "${actual}" | grep -qF "${check_pattern}"; then
echo "FAIL: ${test_name}"
echo " expected to find: '${check_pattern}'"
echo " in body:"
echo "${actual}" | sed 's/^/ /'
FAILURES=$((FAILURES + 1))
return
fi
else
if echo "${actual}" | grep -qF "${check_pattern}"; then
echo "FAIL: ${test_name}"
echo " expected NOT to find: '${check_pattern}'"
echo " in body:"
echo "${actual}" | sed 's/^/ /'
FAILURES=$((FAILURES + 1))
return
fi
fi

echo "PASS: ${test_name}"
}

# --- needs_input comment test cases ---

# Comment should include the reason
run_needs_input_comment_test "needs-input-comment-includes-reason" \
"Build tooling is broken: make setup failed" \
"42" "my-org/my-repo" \
"Build tooling is broken" "yes"

# Comment should include the issue number
run_needs_input_comment_test "needs-input-comment-includes-issue" \
"Build tooling is broken" \
"505" "my-org/my-repo" \
"#505" "yes"

# Comment should include retry instruction
run_needs_input_comment_test "needs-input-comment-includes-retry" \
"Build tooling is broken" \
"42" "my-org/my-repo" \
"/fs-code" "yes"

# Comment should include workflow run URL
run_needs_input_comment_test "needs-input-comment-includes-run-url" \
"Build tooling is broken" \
"42" "my-org/my-repo" \
"my-org/my-repo/actions/runs/" "yes"

# Comment should include the stop emoji header
run_needs_input_comment_test "needs-input-comment-includes-header" \
"Build tooling is broken" \
"42" "my-org/my-repo" \
"Needs human input" "yes"

# Comment should NOT contain "No PR created" (that's the no-op path)
run_needs_input_comment_test "needs-input-comment-no-noop-text" \
"Build tooling is broken" \
"42" "my-org/my-repo" \
"No PR created" "no"

# Comment should NOT contain PUSH_TOKEN
run_needs_input_comment_test "needs-input-comment-no-token-leak" \
"Build tooling is broken" \
"42" "my-org/my-repo" \
"PUSH_TOKEN" "no"

# --- Branch validation test cases ---

# Auto-correct: agent writes main, default is master, no allowed list → corrected
Expand Down
43 changes: 43 additions & 0 deletions scripts/post-code.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2264,6 +2264,49 @@ Retry with \`/fs-code\` if appropriate."
fi
}

# ---------------------------------------------------------------------------
# 0. Check for needs_input signal
#
# The agent may signal that it cannot proceed without human input. When
# needs_input is true, apply the fs-code-needs-input label to the issue,
# post a comment explaining the blocker, and exit without creating a PR.
# This is an expected path — exit 0, not a failure.
# ---------------------------------------------------------------------------
AGENT_NEEDS_INPUT="false"
if [ -n "${RESULT_FILE:-}" ] && [ -f "${RESULT_FILE}" ]; then
AGENT_NEEDS_INPUT="$(jq -r '.needs_input // false' "${RESULT_FILE}" 2>/dev/null || echo "false")"
fi

if [ "${AGENT_NEEDS_INPUT}" = "true" ]; then
NEEDS_INPUT_REASON="$(jq -r '.needs_input_reason // "No reason provided"' "${RESULT_FILE}" 2>/dev/null || echo "No reason provided")"
gha_echo notice "Agent signaled needs_input — posting comment and applying label"

safe_issue_number="$(_sanitize_workflow_value "${ISSUE_NUMBER}")"
_post_failure_ensure_token

run_url="$(forge_get_workflow_run_url)"
sanitized_reason="$(sanitize_failure_detail "${NEEDS_INPUT_REASON}")"

needs_input_body="🛑 **Needs human input** — code agent cannot proceed

The code agent evaluated issue #${safe_issue_number} but determined it cannot make progress without human intervention.

**Reason:** ${sanitized_reason}

**Workflow run:** ${run_url}

Please address the blocker above, then retry with \`/fs-code\`."

forge_create_label "fs-code-needs-input" "Code agent needs human input to proceed" "FBCA04"
forge_add_label "fs-code-needs-input"

if ! forge_post_issue_comment "${needs_input_body}"; then
gha_echo warning "Failed to post needs_input comment to issue #${safe_issue_number}"
fi

exit 0
fi

# ---------------------------------------------------------------------------
# 1. Verify feature branch
# ---------------------------------------------------------------------------
Expand Down
43 changes: 43 additions & 0 deletions scripts/post-code.src.sh
Original file line number Diff line number Diff line change
Expand Up @@ -366,6 +366,49 @@ Retry with \`/fs-code\` if appropriate."
fi
}

# ---------------------------------------------------------------------------
# 0. Check for needs_input signal
#
# The agent may signal that it cannot proceed without human input. When
# needs_input is true, apply the fs-code-needs-input label to the issue,
# post a comment explaining the blocker, and exit without creating a PR.
# This is an expected path — exit 0, not a failure.
# ---------------------------------------------------------------------------
AGENT_NEEDS_INPUT="false"
if [ -n "${RESULT_FILE:-}" ] && [ -f "${RESULT_FILE}" ]; then
AGENT_NEEDS_INPUT="$(jq -r '.needs_input // false' "${RESULT_FILE}" 2>/dev/null || echo "false")"
fi

if [ "${AGENT_NEEDS_INPUT}" = "true" ]; then
NEEDS_INPUT_REASON="$(jq -r '.needs_input_reason // "No reason provided"' "${RESULT_FILE}" 2>/dev/null || echo "No reason provided")"
gha_echo notice "Agent signaled needs_input — posting comment and applying label"

safe_issue_number="$(_sanitize_workflow_value "${ISSUE_NUMBER}")"
_post_failure_ensure_token

run_url="$(forge_get_workflow_run_url)"
sanitized_reason="$(sanitize_failure_detail "${NEEDS_INPUT_REASON}")"

needs_input_body="🛑 **Needs human input** — code agent cannot proceed

The code agent evaluated issue #${safe_issue_number} but determined it cannot make progress without human intervention.

**Reason:** ${sanitized_reason}

**Workflow run:** ${run_url}

Please address the blocker above, then retry with \`/fs-code\`."

forge_create_label "fs-code-needs-input" "Code agent needs human input to proceed" "FBCA04"
forge_add_label "fs-code-needs-input"

if ! forge_post_issue_comment "${needs_input_body}"; then
gha_echo warning "Failed to post needs_input comment to issue #${safe_issue_number}"
fi

exit 0
fi

# ---------------------------------------------------------------------------
# 1. Verify feature branch
# ---------------------------------------------------------------------------
Expand Down
21 changes: 21 additions & 0 deletions scripts/validate-code-output-test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,27 @@ run_test "schema-invalid-json" \
'not json' \
"false"

# needs_input schema cases
run_test "schema-valid-needs-input-true-with-reason" \
'{"target_branch":"main","needs_input":true,"needs_input_reason":"scan-secrets is not installed"}' \
"true"

run_test "schema-valid-needs-input-false" \
'{"target_branch":"main","needs_input":false}' \
"true"

run_test "schema-valid-needs-input-absent" \
'{"target_branch":"main"}' \
"true"

run_test "schema-fail-needs-input-true-without-reason" \
'{"target_branch":"main","needs_input":true}' \
"false"

run_test "schema-valid-needs-input-reason-only" \
'{"target_branch":"main","needs_input_reason":"some reason"}' \
"true"

# The run_test helper always creates output/agent-result.json, so testing a
# missing output directory requires a separate helper.
run_test_no_output_dir() {
Expand Down
Loading
Loading