diff --git a/bin/fm-brief.sh b/bin/fm-brief.sh index 63ca1f054e..efdc8262ff 100755 --- a/bin/fm-brief.sh +++ b/bin/fm-brief.sh @@ -330,6 +330,7 @@ The report is the only thing that survives, so anything worth keeping must be in 5. If you hit the same obstacle twice, append \`blocked: {why}\` and stop; firstmate will help. 6. If a decision belongs to a human (product choices, destructive actions), append \`needs-decision: {summary of options}\` and stop. Firstmate will reply with the decision. + Give it a key when more than one decision or blocker can be open at once: \`needs-decision [key=]: {summary}\`. A decision or blocker you opened stays open until a \`resolved\` line carrying its exact key lands; a later \`done:\` or \`working:\` line never closes it, even when the answer is what started that work. Firstmate's reply normally writes that closing line at answer time; when a blocker or wait clears WITHOUT a firstmate reply, append \`resolved: {how it cleared}\` yourself (same \`[key=]\` if you opened it with one) as you resume. 7. Never stop, restart, or update the shared \`no-mistakes\` daemon - it is one instance serving @@ -447,6 +448,7 @@ $RULE1 5. If you hit the same obstacle twice, append \`blocked: {why}\` and stop; firstmate will help. 6. If a decision belongs above the implementation worker (product choices, destructive actions, ask-user findings), append \`needs-decision: {summary of options}\` and stop. Firstmate will reply with the decision. + Give it a key when more than one decision or blocker can be open at once: \`needs-decision [key=]: {summary}\`. A decision or blocker you opened stays open until a \`resolved\` line carrying its exact key lands; a later \`done:\` or \`working:\` line never closes it, even when the answer is what started that work. Firstmate's reply normally writes that closing line at answer time; when a blocker or wait clears WITHOUT a firstmate reply, append \`resolved: {how it cleared}\` yourself (same \`[key=]\` if you opened it with one) as you resume. 7. Never stop, restart, or update the shared \`no-mistakes\` daemon - it is one instance serving diff --git a/bin/fm-send.sh b/bin/fm-send.sh index 512df6245c..7629ec62c3 100755 --- a/bin/fm-send.sh +++ b/bin/fm-send.sh @@ -20,8 +20,12 @@ # 3 = the text was typed into the live endpoint and Enter was sent, but the # submit read-back stayed unconfirmed (verify the pane before any resend, and # never re-type blindly; a marked request's pending-reply expectation stays -# armed because this outcome is not a proven failure); any other nonzero = the -# send failed and nothing may be assumed delivered. +# armed because this outcome is not a proven failure); a nonzero whose stderr +# says the text was delivered and not to resend = a post-delivery bookkeeping +# step failed after the submit was confirmed (the pending-reply delivery commit, +# a failed closing append, or a named --resolve-key that was not open) - the +# answer already landed, so reconcile the named record and do not resend; any +# other nonzero = the send failed and nothing may be assumed delivered. # Submission dispatches through the target's recorded backend; the tmux adapter # shares its composer/submit core with the away-mode daemon via bin/fm-tmux-lib.sh. # Tune with FM_SEND_RETRIES (default 3) / FM_SEND_SLEEP (0.4). @@ -83,19 +87,32 @@ # transferred from the live status log to its durable captain-held task, which # the status ledger alone can no longer close. # -# Each named key must therefore currently be open in ONE of the two ledgers: open -# in this home's status log per status_open_decisions (bin/fm-classify-lib.sh), or -# a still-open captain-held task resolved as above. A key in neither is refused -# before sending, so a mistyped key cannot deliver an answer while silently -# orphaning the decision. A failed or unconfirmed send never closes a key (a remote +# A named key is closed only when it is currently open in ONE of the two ledgers: +# this home's status log per status_open_decisions (bin/fm-classify-lib.sh), or +# a still-open captain-held task resolved as above. A key in neither closes +# nothing, and fm-send says so loudly on stderr and exits nonzero - but it still +# DELIVERS the answer, and still closes whichever other named keys are open. +# Delivering there is deliberate. Cancelling the whole send did not avoid the +# mis-state this flag exists to prevent - a decision that looks answered while +# its record stays open - it produced the worse version of it: the decision +# stayed open anyway AND no worker ever received the answer the captain +# believed was given. Delivering the text is never more dangerous than the +# plain steer that is always allowed; the flag's power is the CLOSE, and that +# is exactly what an unmatched key withholds. "Delivered, not closed" is also +# what the post-delivery append-failure path below already accepts as the safe +# direction, because an open decision re-surfaces in every later OPEN DECISIONS +# listing until it is really answered. The diagnostic stays loud so the key gets +# reconciled instead of assumed closed, and says not to resend. +# A failed or unconfirmed send never closes a key (a remote # delivered-with-pending-confirmation outcome counts as delivered - see the -# remote paragraph above); a -# delivered answer whose closing append fails exits nonzero with the exact -# manual close command, leaving the decision open to re-surface (the safe -# direction). A send without the flag never closes anything: a routine steer, -# working:, or done: event still cannot clear a captain decision. The flag is -# refused with --key, with an explicit backend target (no task ledger in this -# home), and with an empty message. +# remote paragraph above); a delivered answer whose closing append fails exits +# nonzero with the exact manual close command, leaving the decision open to +# re-surface (the safe direction). A send without the flag never closes anything: +# a routine steer, working:, or done: event still cannot clear a captain +# decision. Malformed usage IS refused before anything is sent - a key outside +# the slug charset, a duplicate key, --key, an explicit backend target (no task +# ledger in this home), and an empty message - because those are argument errors +# with no answer to deliver, not ledger-state mismatches. # # After a successful text submit fm-send pauses FM_SEND_SETTLE seconds (default 1, # 0 disables) before returning: submit confirmation only proves the text was @@ -382,20 +399,25 @@ if [ -n "$TARGET_SELECTOR" ] && [ -n "$TARGET_META" ] && [ "$(fm_meta_get "$TARG fi # Validate the answerer-closes request before any durable mutation or send: the -# target must have a task ledger in THIS home, the send must carry an answer -# message, and every named key must be open right now in that ledger per the -# ONE authoritative fold (status_open_decisions). Refusing here, before the -# send, is what keeps a mistyped key loud instead of delivering an answer that -# silently leaves its decision open. +# target must have a task ledger in THIS home and the send must carry an answer +# message. Those are argument errors, refused outright. +# Each named key is then checked against the ONE authoritative fold +# (status_open_decisions) and split into the keys this send may close and the +# keys it may not. An unmatched key is announced here, before the send, so the +# reason is on the record even if delivery later fails - but it does not cancel +# the answer (see the header contract). RESOLVE_STATUS_FILE= # Which ledger each answered key belongs to. A key still open in the status log # is owned by the status log: fm-captain-hold's `complete` closes that live copy # at the moment it transfers a decision to its durable captain-held task, so # "still open in status" and "already held" are the two sides of one transfer, # never both at once. Checking the backlog only for keys the status log no -# longer owns also keeps the common path free of any backlog read. +# longer owns also keeps the common path free of any backlog read. A key in +# neither ledger is unmatched: the answer is still delivered, and nothing is +# closed for that key. RESOLVE_STATUS_KEYS= RESOLVE_HOLD_KEYS= +RESOLVE_UNMATCHED_KEYS= # Resolve a --resolve-key key that the status log no longer owns to the # captain-held task that carries it: the key as a task id itself (the collapsed @@ -443,19 +465,20 @@ if [ -n "$RESOLVE_KEYS" ]; then esac # Not open in the status log. A decision already transferred to its durable # captain-held task is exactly this case, and it is answerable - just - # through the other ledger - so check there before refusing. + # through the other ledger - so check there before treating it as unmatched. if resolved_hold_id=$(fm_send_hold_resolved_id "$RESOLVE_TASK_ID" "$k"); then RESOLVE_HOLD_KEYS="${RESOLVE_HOLD_KEYS}${RESOLVE_HOLD_KEYS:+ }$resolved_hold_id" continue fi - echo "error: --resolve-key '$k': no open decision or blocker with that key in $RESOLVE_STATUS_FILE, and no captain-held task '$k' or '$RESOLVE_TASK_ID-decision-$k' still open (already closed or mistyped). Re-check the OPEN DECISIONS listing, then resend without that key or with the right one; nothing was sent." >&2 - exit 1 + RESOLVE_UNMATCHED_KEYS="${RESOLVE_UNMATCHED_KEYS}${RESOLVE_UNMATCHED_KEYS:+ }$k" + echo "warning: --resolve-key '$k': no open decision or blocker with that key in $RESOLVE_STATUS_FILE, and no captain-held task '$k' or '$RESOLVE_TASK_ID-decision-$k' still open (already closed, mistyped, or transferred). This does not cancel the answer; nothing will be closed for that key, and the delivery outcome is reported when the send completes." >&2 done fi # Close each answered decision in this home's ledger, only after delivery is -# fully confirmed. An append failure exits nonzero with the manual close -# command; the decision then stays open and re-surfaces, never silently lost. +# fully confirmed, and only for the keys that were actually open. An append +# failure exits nonzero with the manual close command; the decision then stays +# open and re-surfaces, never silently lost. # The close is this home's own bookkeeping, written by the very turn that # answered the decision, so it goes through the guarded self-announced append # (bin/fm-wake-lib.sh) and does not wake this same session again; any @@ -666,7 +689,7 @@ else fi # Delivery is fully confirmed: close each answered decision in this home's # ledger (answerer-closes; see the header contract). - if [ -n "$RESOLVE_KEYS" ]; then + if [ -n "$RESOLVE_STATUS_KEYS" ] || [ -n "$RESOLVE_HOLD_KEYS" ]; then fm_send_close_resolved_keys "$RESOLVE_ANSWER_TEXT" || exit 1 fm_send_feed_resolved_holds "$RESOLVE_ANSWER_TEXT" || exit 1 fi @@ -683,3 +706,17 @@ else # disables it. Scoped to this path only, never the shared submit core. [ "${FM_SEND_SETTLE:-1}" = 0 ] || sleep "${FM_SEND_SETTLE:-1}" fi + +# The answer landed but at least one named key was not open, so the ledger is +# not in the state the caller expected: exit nonzero rather than reporting a +# clean success. The decision, if it exists under another key, stays open and +# re-surfaces in the next OPEN DECISIONS listing - which is also where the key +# that actually closes it is printed, next to its own close command. +if [ -n "$RESOLVE_UNMATCHED_KEYS" ]; then + RESOLVE_UNMATCHED_RENDERED= + for k in $RESOLVE_UNMATCHED_KEYS; do + RESOLVE_UNMATCHED_RENDERED="${RESOLVE_UNMATCHED_RENDERED}${RESOLVE_UNMATCHED_RENDERED:+ }--resolve-key '$k'" + done + echo "error: the answer was delivered to $T, but no decision was closed for $RESOLVE_UNMATCHED_RENDERED (not open in $RESOLVE_STATUS_FILE). Do not resend the answer. Re-check the OPEN DECISIONS listing and close the real key with its printed command, or leave it open if it is already closed." >&2 + exit 1 +fi diff --git a/bin/fm-wake-drain.sh b/bin/fm-wake-drain.sh index 203765be80..36c4eccb21 100755 --- a/bin/fm-wake-drain.sh +++ b/bin/fm-wake-drain.sh @@ -130,8 +130,18 @@ EOF # fm-classify-lib.sh's "incremental (cursor-backed) open-decisions fold"). # Bounded and silent: prints nothing when no decision is open, which is the # common case. +# +# Printed key == answerable key. Every entry renders the key the fold actually +# decided - INCLUDING the "default" bucket a keyless line opens - and carries +# its own ready-to-run close command underneath, because the two must never +# disagree: an entry that showed no key while its note text still contained a +# "[key=...]" token advertised that token as the key, and fm-send closes only a +# key this same fold reports open. Rendering the fold's key and spelling the +# exact command for it removes the inference that mismatch needed. A "[key=...]" +# inside a note is worker prose, quoted verbatim rather than rewritten - the +# command line under the note is what says which key is real. print_open_decisions_section() { - local snapshot=${1:-} open task key verb note line item_bytes=220 global_bytes=4000 + local snapshot=${1:-} open task key verb note line cmd item_bytes=220 global_bytes=4000 local output='' used=0 shown=0 omitted=0 bytes if [ -n "$snapshot" ]; then @@ -143,20 +153,34 @@ print_open_decisions_section() { while IFS=$(printf '\t') read -r task key verb note; do [ -n "$task" ] || continue - line="$task" - [ "$key" = default ] || line="$line [key=$key]" - line="$line $verb: $note" + line="$task [key=$key] $verb: $note" # The shared cut counts the item's own characters; the trailing newline this # section's global budget also pays for is this caller's, so the per-item # allowance passed down is one short of the cap. fm_cap_line_var "$line" $((item_bytes - 1)) line=$FM_LINE_CAP_LINE - bytes=$(( ${#line} + 1 )) + # Never capped: a truncated command would be a command that does not run, + # and both operands are bounded already (a task id and a slug-charset key). + # The key is slug-charset by the fold's own check, but the task id is just a + # state-dir filename, and this line is printed to be RUN: an id outside the + # plain-slug charset, or one that starts with a dash (fm-send would parse it + # as a flag), gets a pointer instead of a command rather than a string an + # agent would paste into a shell. + case "$task" in + -*|*[!A-Za-z0-9._-]*) + cmd=" close it: answer that task with bin/fm-send.sh --resolve-key $key (its id is not a plain slug; check $STATE)" + ;; + *) cmd=" close it: bin/fm-send.sh $task --resolve-key $key ''" ;; + esac + # An entry is shown only when its note AND its command both fit, so no + # decision is ever listed without the command that closes it. + bytes=$(( ${#line} + 1 + ${#cmd} + 1 )) if [ $((used + bytes)) -gt "$global_bytes" ]; then omitted=$((omitted + 1)) continue fi output="$output$line +$cmd " used=$((used + bytes)) shown=$((shown + 1)) @@ -174,7 +198,7 @@ EOF # the send that answers a listed decision also closes it, so closure never # depends on the busy worker writing a matching resolved line (contract: # bin/fm-send.sh header). - printf "OPEN DECISIONS: close one by answering it: bin/fm-send.sh --resolve-key ''\n" || return 1 + printf 'OPEN DECISIONS: answer each one with the close command printed under it; the key that closes a decision is the bracketed key before its verb, never a [key=...] inside the note text.\n' || return 1 } # Print the RECORD DIVERGENCE section: every captain call whose two records diff --git a/bin/fm-watch.sh b/bin/fm-watch.sh index d1d59d3ceb..442300e3fa 100755 --- a/bin/fm-watch.sh +++ b/bin/fm-watch.sh @@ -487,9 +487,21 @@ clear_pause_tracking() { # } # Reconcile a declared pause or captain-held status with authoritative crew state. -# After fm-crew-state has fallen back to stopped or unknown, paused classification is -# recovered only for a confidently dead ordinary crew, or for a secondmate, whose -# endpoint liveness this function deliberately never reads. +# The authoritative read decides first, and agent liveness never overrules it: a +# crew that DECLARED its own external wait is normally still alive holding +# something (a dev server, a session), so a live agent is that pause's expected +# shape rather than evidence against it. Liveness only breaks the remaining tie - +# a paused or captain-held status whose crew fm-crew-state can no longer confirm +# (stopped or unknown), where paused classification is recovered only for a +# confidently dead ordinary crew, or for a secondmate, whose endpoint liveness +# this function deliberately never reads, and a live or unreadable ordinary crew +# still surfaces, so a captain hold firstmate wrote over a crew that never +# declared the wait itself cannot silence a live decision gate. +# Once a decision grants the bounded cadence, the cheap .paused- path holds +# it without re-reading anything until the recheck marker ages past +# STALE_ESCALATE_SECS. Re-deciding it per stale hash was the 2026-08-12 churn bug: +# a held dev server repaints, so every repaint was a fresh first sighting that +# re-read liveness and queued another contentless stale wake. pause_state_class() { # local win=$1 task=$2 key last recheck_file class agent_alive kind key=$(window_key "$win") @@ -500,29 +512,21 @@ pause_state_class() { # crew_absorb_class "$task" return fi - # Read once past the declared-wait gate and reused by both liveness gates below, - # so a mate's stale poll costs one metadata scan rather than one per gate, and the - # far more common no-declaration path above still costs none. - kind=$(window_kind "$win") if [ -e "$STATE/.paused-$key" ] && [ "$(age_of "$recheck_file")" -lt "$STALE_ESCALATE_SECS" ]; then - if [ "$kind" != secondmate ]; then - agent_alive=$(fm_backend_agent_alive "$(window_backend "$win")" "$win" 2>/dev/null) || agent_alive=unknown - if [ "$agent_alive" != dead ]; then - rm -f "$recheck_file" - printf 'none' - return - fi - fi printf 'paused' return fi + # Read once past both cheap gates, so only the poll that actually re-decides a + # declared wait pays the metadata scan the liveness gate below needs; the far + # more common no-declaration and held-cadence paths above cost none. + kind=$(window_kind "$win") class=$(crew_absorb_class "$task") if [ "$class" = working ]; then rm -f "$recheck_file" printf 'working' return fi - if [ "$kind" != secondmate ]; then + if [ "$class" != paused ] && [ "$kind" != secondmate ]; then agent_alive=$(fm_backend_agent_alive "$(window_backend "$win")" "$win" 2>/dev/null) || agent_alive=unknown if [ "$agent_alive" != dead ]; then rm -f "$recheck_file" @@ -531,13 +535,15 @@ pause_state_class() { # fi fi # Recover paused classification for a declared wait that authoritative crew state - # could not name. Reaching here already proves the only two admissible cases: an - # ordinary crew whose agent the gate above confirmed dead, so no live decision gate - # is being silenced, or a secondmate, whose endpoint liveness is deliberately never - # read and so cannot supply that confirmation. Without the mate case a mate's - # captain hold - which has no current-state mapping and so arrives as `none` - - # would be silenced by every caller rather than taking the bounded re-surface - # cadence, and a forgotten hold would rot invisibly. + # could not name. A crew the authoritative read already named paused skipped the + # gate above and needs no recovery, so reaching here still `none` proves the only + # two admissible cases: an ordinary crew whose agent the gate above confirmed + # dead, so no live decision gate is being silenced, or a secondmate, whose + # endpoint liveness is deliberately never read and so cannot supply that + # confirmation. Without the mate case a mate's captain hold - which has no + # current-state mapping and so arrives as `none` - would be silenced by every + # caller rather than taking the bounded re-surface cadence, and a forgotten hold + # would rot invisibly. [ "$class" = none ] && class=paused case "$class" in paused) date +%s > "$recheck_file" ;; diff --git a/docs/architecture.md b/docs/architecture.md index 0f1cf8dd9a..2fdcdb6269 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -26,8 +26,9 @@ A concurrent replacement remains armed, every non-merged or invalid observation No-verb wakes, such as `working:` notes and bare turn-ended signals, are benign only when `bin/fm-crew-state.sh` reports positive evidence that the crew is still working: an actively running no-mistakes step attributed to that crew's current code, or an exact busy verdict from the semantic busy-state contract. A `kind=secondmate` task's status signal is the parent-directed reply stream and is never absorbed as provably working; only its bare turn-ended signal retains the ordinary absorb rule. A crew that declares `paused:` for a known external wait, or carries a verified `captain-held` transfer, is separately absorbed while idle and re-surfaced only on the longer pause cadence, rather than being treated as a possible wedge. -For an ordinary crew that has stopped, the normal-mode watcher first surfaces one stale wake, then applies that same cadence to an unchanged `paused:` or durable `captain-held` endpoint only when the backend confidently reports its agent dead. -Live or inconclusive liveness remains fail-open at that initial surface, and a secondmate's endpoint liveness is still never read at all; a mate is admitted to that same cadence only to serve a declared wait's bounded re-surface, so a forgotten pause or captain hold on a mate cannot rot invisibly. +That declaration is authoritative whether or not the crew's agent is still alive, because a crew holding something for its own external wait is normally still running. +Agent liveness only breaks the remaining tie: for a `paused:` or durable `captain-held` endpoint whose crew `bin/fm-crew-state.sh` can no longer confirm, the normal-mode watcher first surfaces one stale wake and grants that cadence only when the backend confidently reports the agent dead. +Live or inconclusive liveness remains fail-open at that initial surface, so a captain hold over a crew that never declared the wait itself cannot silence a live decision gate; a secondmate's endpoint liveness is still never read at all, and a mate is admitted to that same cadence only to serve a declared wait's bounded re-surface, so a forgotten pause or captain hold on a mate cannot rot invisibly. Its initial normal-mode status signal still surfaces through the no-verb path, while away mode self-handles that routine signal and owns the later recheck. Fresh stale panes use the same current-state read before trusting the status log, so an active run or a proven busy worker outranks an old captain-relevant status-log line left behind before validation. No-change heartbeats are also benign. @@ -48,6 +49,9 @@ The explicit resolution is written by the actor that answers, not the busy worke This home's answerer close, pending-reply escalation close, and captain-held transfer use the provenance-guarded append owned by `bin/fm-wake-lib.sh`, so they advance the watcher marker only across their own bytes when all earlier bytes were already announced; pending or interleaved foreign bytes fail toward an ordinary wake. A turn-ended-only queue row omits its historical status annotation when that status file exactly matches the same seen marker. Any direct or remaining historical annotation prints every status line unread at the presentation cursor instead of replaying only the latest line. +Because that closure is keyed, the listing and the send must agree on the key: the section renders the key the fold decided, including the shared `default` bucket a keyless line opens, beside the close command for exactly that key, so it can never advertise a key that would close nothing. +That command line degrades to a pointer naming the key when the decision's task id is not safe to paste as a shell command, so the section never prints a string an agent would run as something other than the close it describes. +An answer naming a key that is not open is still delivered and closes nothing for that key, while every other key the same send names and the fold still reports open closes normally, exiting nonzero: that nonzero is a ledger mismatch, not an undelivered send, and `bin/fm-send.sh`'s header owns the exit-status contract. `bin/fm-crew-state.sh ` is the cheap current-state read for an actionable heartbeat review: it attributes a no-mistakes run, active or terminal, only when it matches the crew's branch and current code identity, then keeps that run-step authoritative even if the pane has closed. The script header owns the exact run-head ancestry rules. During no-mistakes' `ci` monitor phase, it also reads the ci step log tail because `axi status` reports both "still waiting on checks" and "checks green, waiting on merge" as `ci,running`. diff --git a/tests/fm-send-resolve-key.test.sh b/tests/fm-send-resolve-key.test.sh index 910f812b21..8f181e5df5 100755 --- a/tests/fm-send-resolve-key.test.sh +++ b/tests/fm-send-resolve-key.test.sh @@ -13,7 +13,14 @@ # scenario where the worker never writes a matching resolved line. # 2. A routine steer without the flag never closes anything, and a working:/ # done: line still cannot clear a captain decision. -# 3. A key that is not open refuses BEFORE anything is sent (mistype safety). +# 3. A key that is not open closes nothing and exits nonzero, while the answer +# itself is still delivered - that nonzero is not "unsent" - and a mixed +# batch closes only its open keys. Cancelling the send used to drop the +# captain's answer entirely, which is the worse half of the same mis-state +# this flag exists to prevent. +# 3b. Whatever key the drain PRINTS is a key fm-send ACCEPTS: the printed close +# commands are replayed verbatim, never retyped, for the inline-marker +# form, the keyless fallback, and a note carrying key-shaped prose. # 4. A failed or unconfirmed send never closes a key. # 5. A local secondmate answer is marked+corr'd yet closes the same way, and # the closing line carries the plain answer, not marker or corr bytes. @@ -242,7 +249,13 @@ test_routine_steer_never_closes() { pass "fm-send: a send without --resolve-key never closes a decision, and working/done still cannot" } -test_not_open_key_refuses_before_send() { +# A key that is not open closes nothing and exits nonzero, but the ANSWER still +# reaches the worker. Refusing the whole send used to drop the captain's answer +# on the floor - the mis-state this flag exists to prevent is a decision that +# looks answered while its record stays open, and cancelling delivery produced +# the worse version of it: nothing answered, nothing closed, and a captain who +# believed the decision was handled. +test_not_open_key_still_delivers_but_closes_nothing() { local dir fb log home err rc out dir="$TMP_ROOT/not-open"; mkdir -p "$dir" fb=$(make_stubs "$dir"); log="$dir/send.log"; err="$dir/send.err" @@ -254,17 +267,105 @@ test_not_open_key_refuses_before_send() { env PATH="$fb:$PATH" \ FM_ROOT_OVERRIDE="$home" FM_HOME="$home" FM_SEND_LOG="$log" FM_SEND_SETTLE=0 \ "$SEND" t4 --resolve-key mistyped "the answer" >/dev/null 2>"$err"; rc=$? - [ "$rc" -ne 0 ] || fail "a not-open key should refuse" - assert_contains "$(cat "$err")" "--resolve-key 'mistyped'" "the refusal should name the bad key" - assert_contains "$(cat "$err")" "nothing was sent" "the refusal should state nothing was sent" - [ ! -s "$log" ] || fail "a refused answer still typed text: $(cat "$log")" + [ "$rc" -ne 0 ] || fail "a not-open key should still exit nonzero" + assert_contains "$(cat "$err")" "--resolve-key 'mistyped'" "the diagnostic should name the bad key" + assert_contains "$(cat "$err")" "Do not resend the answer" \ + "the diagnostic should say the answer already landed" + assert_contains "$(cat "$err")" "the answer was delivered" \ + "the diagnostic should say the answer already landed, not that the send failed" + assert_not_contains "$(cat "$err")" "nothing may be assumed delivered" \ + "a delivered unmatched-key must not be described as undelivered" + assert_not_contains "$(cat "$err")" "not sent" \ + "a delivered unmatched-key must not be treated as unsent" + assert_contains "$(cat "$log")" "the answer" "the answer itself should still reach the worker" if grep -F 'resolved' "$home/state/t4.status" >/dev/null; then - fail "a refused answer still closed something: $(cat "$home/state/t4.status")" + fail "an unmatched key still closed something: $(cat "$home/state/t4.status")" fi out=$(drain_out "$home") printf '%s' "$out" | grep -F '[key=real-key]' >/dev/null \ - || fail "the real decision disappeared after a refused answer: $out" - pass "fm-send --resolve-key: a key that is not open refuses loudly before anything is sent" + || fail "the real decision disappeared after an unmatched key: $out" + pass "fm-send --resolve-key: an unmatched key delivers the answer, closes nothing, and exits nonzero" +} + +# Mixed batch: the open keys close exactly as they would have, the unmatched one +# is reported, and the answer is delivered once. Partial closure is the point - +# each key names an independently open decision, so an unrelated mistyped key +# must not withhold closure from the decisions this answer really did answer. +test_partially_matched_keys_close_only_what_is_open() { + local dir fb log home err rc out + dir="$TMP_ROOT/partial"; mkdir -p "$dir" + fb=$(make_stubs "$dir"); log="$dir/send.log"; err="$dir/send.err" + home=$(setup_home partial) + fm_write_meta "$home/state/t10.meta" "window=sess:fm-t10" "kind=ship" + { + printf 'needs-decision [key=open-one]: first\n' + printf 'needs-decision [key=open-two]: second\n' + } > "$home/state/t10.status" + + : > "$log" + env PATH="$fb:$PATH" \ + FM_ROOT_OVERRIDE="$home" FM_HOME="$home" FM_SEND_LOG="$log" FM_SEND_SETTLE=0 \ + "$SEND" t10 --resolve-key open-one --resolve-key gone "do the first one" \ + >/dev/null 2>"$err"; rc=$? + [ "$rc" -ne 0 ] || fail "a batch containing an unmatched key should exit nonzero" + assert_contains "$(cat "$err")" "--resolve-key 'gone'" "the diagnostic should name only the unmatched key" + grep -F 'resolved [key=open-one]: answered: do the first one' "$home/state/t10.status" >/dev/null \ + || fail "the matched key was not closed:"$'\n'"$(cat "$home/state/t10.status")" + if grep -F '[key=gone]' "$home/state/t10.status" >/dev/null; then + fail "an unmatched key was written into the ledger: $(cat "$home/state/t10.status")" + fi + out=$(drain_out "$home") + printf '%s' "$out" | grep -F '[key=open-two]' >/dev/null \ + || fail "an untouched decision was cleared by a partially matched batch: $out" + if printf '%s' "$out" | grep -F '[key=open-one]' >/dev/null; then + fail "the matched key is still listed as open: $out" + fi + pass "fm-send --resolve-key: a mixed batch closes the open keys and reports only the unmatched one" +} + +# End-to-end agreement between the two halves of the contradiction that started +# this: whatever key the drain PRINTS for a decision is a key fm-send ACCEPTS. +# The test does not retype the key - it runs the exact command the drain printed +# under each decision, for the inline-marker form the briefs tell workers to +# write, for the keyless fallback, and for the trap case where the note text +# carries key-shaped prose the fold deliberately ignores. +test_drain_printed_command_is_accepted_verbatim() { + local dir fb log home rc out cmd task key n + dir="$TMP_ROOT/agreement"; mkdir -p "$dir" + fb=$(make_stubs "$dir"); log="$dir/send.log" + home=$(setup_home agreement) + for task in inline plain prose; do + fm_write_meta "$home/state/$task.meta" "window=sess:fm-$task" "kind=ship" + done + printf 'needs-decision: [key=totals-pool-separation] display-only or engine bug\n' \ + > "$home/state/inline.status" + printf 'needs-decision: which banner color\n' > "$home/state/plain.status" + printf 'needs-decision: pick a [key=red] or [key=blue] theme\n' > "$home/state/prose.status" + + n=0 + while IFS= read -r cmd; do + case "$cmd" in + *'close it: bin/fm-send.sh '*) ;; + *) continue ;; + esac + # " close it: bin/fm-send.sh --resolve-key ''" + cmd=${cmd#*bin/fm-send.sh } + task=${cmd%% *} + key=${cmd#*--resolve-key } + key=${key%% *} + run_send "$fb" "$home" "$log" "$task" --resolve-key "$key" "answering $task"; rc=$? + expect_code 0 "$rc" "the drain printed --resolve-key $key for $task, but fm-send did not accept it" + n=$((n + 1)) + done </dev/null; then + fail "replaying every printed close command left a decision open: $out" + fi + pass "fm-send --resolve-key: every key the drain prints is accepted verbatim by the real send" } test_failed_send_does_not_close() { @@ -289,6 +390,36 @@ test_failed_send_does_not_close() { pass "fm-send --resolve-key: a failed send never closes the decision" } +# The exit-status contract tells a caller to classify a nonzero by whether +# stderr says the text was delivered and not to resend. +# An unmatched key is announced BEFORE the send, so that pre-send line must not +# assert delivery: a send that then fails would otherwise emit both a delivery +# claim and "text not sent", and the caller would drop an answer that never +# landed. +test_failed_send_with_unmatched_key_never_claims_delivery() { + local dir fb log home err rc + dir="$TMP_ROOT/fail-unmatched"; mkdir -p "$dir" + fb=$(make_stubs "$dir"); log="$dir/send.log"; err="$dir/send.err" + home=$(setup_home fail-unmatched) + fm_write_meta "$home/state/t11.meta" "window=sess:fm-t11" "kind=ship" + printf 'needs-decision [key=real-key]: choose\n' > "$home/state/t11.status" + + : > "$log" + env PATH="$fb:$PATH" FM_FAKE_TMUX_SEND_FAIL=1 \ + FM_ROOT_OVERRIDE="$home" FM_HOME="$home" FM_SEND_LOG="$log" FM_SEND_SETTLE=0 \ + "$SEND" t11 --resolve-key mistyped "the answer" >/dev/null 2>"$err"; rc=$? + [ "$rc" -ne 0 ] || fail "a failed send with an unmatched key should exit nonzero" + assert_contains "$(cat "$err")" "--resolve-key 'mistyped'" \ + "the unmatched key should still be named before the send" + assert_contains "$(cat "$err")" "not sent" \ + "a failed send must be reported as unsent" + assert_not_contains "$(cat "$err")" "delivered" \ + "a failed send must never claim the answer was delivered" + assert_not_contains "$(cat "$err")" "resend" \ + "a failed send must never tell the caller not to resend" + pass "fm-send --resolve-key: a failed send with an unmatched key is never reported as delivered" +} + test_multiple_keys_close_together() { local dir fb log home rc out dir="$TMP_ROOT/multi"; mkdir -p "$dir" @@ -501,8 +632,11 @@ test_answer_close_is_self_announced test_colon_first_key_position_is_answerable test_answer_starts_work_never_orphans test_routine_steer_never_closes -test_not_open_key_refuses_before_send +test_not_open_key_still_delivers_but_closes_nothing +test_partially_matched_keys_close_only_what_is_open +test_drain_printed_command_is_accepted_verbatim test_failed_send_does_not_close +test_failed_send_with_unmatched_key_never_claims_delivery test_multiple_keys_close_together test_local_secondmate_answer_marked_and_closed test_remote_secondmate_answer_closes_locally diff --git a/tests/fm-wake-drain-open-decisions.test.sh b/tests/fm-wake-drain-open-decisions.test.sh index 4db2c40954..12034c113e 100755 --- a/tests/fm-wake-drain-open-decisions.test.sh +++ b/tests/fm-wake-drain-open-decisions.test.sh @@ -6,6 +6,11 @@ # open/resolved statement); these tests exercise the real drain script over # crafted status logs and assert on its printed output, not on the fold's own # source text. +# They also pin the printed-key/answerable-key agreement: the section renders +# the key the fold decided (including the "default" bucket) and the exact +# command that closes it, so it can never advertise a key fm-send would refuse. +# The agreement is proven end to end - drain output executed through the real +# fm-send - in tests/fm-send-resolve-key.test.sh. set -u # shellcheck source=tests/wake-helpers.sh @@ -32,11 +37,154 @@ test_buried_decision_still_surfaces() { grep -F 'OPEN DECISIONS' "$out" >/dev/null || fail "buried decision produced no OPEN DECISIONS section" grep -F 'task1' "$out" | grep -F '[key=api-shape]' | grep -F 'pick REST or RPC' >/dev/null \ || fail "buried needs-decision was not surfaced with its task, key, and note" - grep -F "close one by answering it: bin/fm-send.sh --resolve-key " "$out" >/dev/null \ - || fail "open section is missing the answerer-closes hint" + grep -F "close it: bin/fm-send.sh task1 --resolve-key api-shape ''" "$out" >/dev/null \ + || fail "open section is missing the decision's own answerer-closes command" pass "a needs-decision buried under later routine/other-key lines still reports as open" } +# The section used to hide the key whenever the fold landed on "default", so a +# decision whose NOTE still contained a "[key=...]" token advertised that token +# as its key - and the generic hint invited answering with it. fm-send then +# refused the key the listing had just shown. Every entry must now render the +# key the fold decided and carry the command that closes exactly that key. +test_printed_key_and_command_agree_for_every_key_form() { + local dir state out line prev + dir=$(make_case key-command-agreement) + state="$dir/state" + out="$dir/drain.out" + # 1. The inline-marker form the generated briefs tell workers to write. + printf 'needs-decision: [key=totals-pool-separation] display-only or engine bug\n' \ + > "$state/inline.status" + # 2. The no-marker fallback, which folds to the shared "default" bucket. + printf 'needs-decision: which banner color\n' > "$state/plain.status" + # 3. The trap: no stated key, but key-shaped tokens inside the note text. + printf 'needs-decision: pick a [key=red] or [key=blue] theme\n' > "$state/prose.status" + + FM_STATE_OVERRIDE="$state" "$DRAIN" > "$out" || fail "drain failed on mixed key forms" + + grep -F 'inline [key=totals-pool-separation] needs-decision: display-only or engine bug' "$out" >/dev/null \ + || fail "the inline-marker decision did not list under its stated key: $(cat "$out")" + grep -F "close it: bin/fm-send.sh inline --resolve-key totals-pool-separation ''" "$out" >/dev/null \ + || fail "the inline-marker decision's close command did not use its stated key: $(cat "$out")" + + grep -F 'plain [key=default] needs-decision: which banner color' "$out" >/dev/null \ + || fail "the keyless decision did not list its actual (default) key: $(cat "$out")" + grep -F "close it: bin/fm-send.sh plain --resolve-key default ''" "$out" >/dev/null \ + || fail "the keyless decision's close command did not name the default key: $(cat "$out")" + + grep -F 'prose [key=default] needs-decision: pick a [key=red] or [key=blue] theme' "$out" >/dev/null \ + || fail "a note-only key mention changed how the decision listed: $(cat "$out")" + grep -F "close it: bin/fm-send.sh prose --resolve-key default ''" "$out" >/dev/null \ + || fail "the close command followed prose in the note instead of the folded key: $(cat "$out")" + if grep -F -- "--resolve-key red" "$out" >/dev/null || grep -F -- "--resolve-key blue" "$out" >/dev/null; then + fail "the section suggested closing with a key from note prose: $(cat "$out")" + fi + + # Nothing is listed without its own command: every decision line is followed + # by a close command, so a reader never has to infer the key from the note. + prev='' + while IFS= read -r line; do + case "$prev" in + ''|'OPEN DECISIONS'*|' close it: '*) ;; + *) + case "$line" in + ' close it: bin/fm-send.sh '*) ;; + *) fail "a listed decision had no close command under it: $prev" ;; + esac + ;; + esac + prev=$line + done < "$state/normal.status" + printf 'needs-decision [key=odd]: a hand-made status file\n' > "$state/we ird; echo pwned.status" + + FM_STATE_OVERRIDE="$state" "$DRAIN" > "$out" || fail "drain failed with an odd status filename" + + grep -F "close it: bin/fm-send.sh normal --resolve-key real ''" "$out" >/dev/null \ + || fail "the ordinary task lost its close command: $(cat "$out")" + grep -F 'a hand-made status file' "$out" >/dev/null \ + || fail "the odd-id decision was dropped instead of listed: $(cat "$out")" + if grep -F 'bin/fm-send.sh we ird' "$out" >/dev/null; then + fail "an unslug task id was pasted into a runnable command: $(cat "$out")" + fi + grep -F 'its id is not a plain slug' "$out" >/dev/null \ + || fail "the odd-id decision did not explain why it has no command: $(cat "$out")" + pass "a task id outside the plain-slug charset gets a pointer instead of a runnable command" +} + +# A leading dash is a legal filename character, but fm-send treats `-foo` as a +# flag, so the advertised close command would send nothing and leave the +# decision open. Treat it like any other unpasteable id. +test_leading_dash_task_id_gets_a_pointer_not_a_command() { + local dir state out + dir=$(make_case leading-dash-id) + state="$dir/state" + out="$dir/drain.out" + printf 'needs-decision [key=real]: a normal task\n' > "$state/normal.status" + printf 'needs-decision [key=dash]: a leading-dash task id\n' > "$state/-foo.status" + + FM_STATE_OVERRIDE="$state" "$DRAIN" > "$out" || fail "drain failed with a leading-dash status filename" + + grep -F "close it: bin/fm-send.sh normal --resolve-key real ''" "$out" >/dev/null \ + || fail "the ordinary task lost its close command: $(cat "$out")" + grep -F 'a leading-dash task id' "$out" >/dev/null \ + || fail "the leading-dash decision was dropped instead of listed: $(cat "$out")" + if grep -F 'close it: bin/fm-send.sh -foo' "$out" >/dev/null; then + fail "a leading-dash task id was pasted into a runnable command: $(cat "$out")" + fi + grep -F 'its id is not a plain slug' "$out" >/dev/null \ + || fail "the leading-dash decision did not explain why it has no command: $(cat "$out")" + pass "a leading-dash task id gets a pointer instead of a runnable command" +} + +# The section's global byte budget now pays for each entry's close command as +# well as its note, and the two are charged together on purpose: at the cap +# boundary an entry must be dropped whole rather than listed with no way to +# close it. +test_global_cap_never_lists_a_decision_without_its_command() { + local dir state out i decisions commands + dir=$(make_case global-cap) + state="$dir/state" + out="$dir/drain.out" + i=0 + while [ "$i" -lt 40 ]; do + { + printf 'needs-decision [key=k%02d]: decision %02d needs a call' "$i" "$i" + awk 'BEGIN { while (n++ < 12) printf " and-then-some" }' + printf '\n' + } > "$state/task$(printf '%02d' "$i").status" + i=$((i + 1)) + done + + FM_STATE_OVERRIDE="$state" "$DRAIN" > "$out" || fail "drain failed on a flooded decision set" + + decisions=$(grep -c ' needs-decision: ' "$out") + commands=$(grep -c ' close it: bin/fm-send.sh ' "$out") + [ "$decisions" -eq "$commands" ] \ + || fail "$decisions decisions listed but $commands close commands printed" + [ "$decisions" -gt 0 ] || fail "the flooded set listed nothing at all" + [ "$decisions" -lt 40 ] || fail "the flood was not large enough to reach the byte cap" + grep -E '^OPEN DECISIONS: [0-9]+ more omitted \(byte cap\)$' "$out" >/dev/null \ + || fail "entries were dropped without the omission disclosure: $(cat "$out")" + pass "at the global byte cap an entry is dropped whole, never listed without its close command" +} + test_explicit_resolution_closes_it() { local dir state out dir=$(make_case resolved) @@ -196,13 +344,17 @@ test_over_long_decision_note_is_capped_with_a_marker() { FM_STATE_OVERRIDE="$state" "$DRAIN" > "$out" || fail "drain failed on an over-long decision note" - line=$(grep -F 'task-long' "$out") + line=$(grep -F 'task-long [key=' "$out") case "$line" in 'task-long [key=api-shape] needs-decision: pick REST or RPC'*' [truncated]') : ;; *) fail "an over-long decision note was not capped with its lede intact: $line" ;; esac longest=${#line} [ "$longest" -le 219 ] || fail "a capped decision item ran $longest characters past its per-item budget" + # The note is what gets cut; its close command must survive whole, because a + # truncated command is a command that does not run. + grep -F "close it: bin/fm-send.sh task-long --resolve-key api-shape ''" "$out" >/dev/null \ + || fail "the close command was cut along with its over-long note: $(cat "$out")" printf 'needs-decision [key=short]: brief enough to keep whole\n' > "$state/task-short.status" FM_STATE_OVERRIDE="$state" "$DRAIN" > "$out" || fail "drain failed on a short decision note" @@ -216,7 +368,11 @@ test_over_long_decision_note_is_capped_with_a_marker() { } test_buried_decision_still_surfaces +test_printed_key_and_command_agree_for_every_key_form test_over_long_decision_note_is_capped_with_a_marker +test_unslug_task_id_gets_a_pointer_not_a_command +test_leading_dash_task_id_gets_a_pointer_not_a_command +test_global_cap_never_lists_a_decision_without_its_command test_explicit_resolution_closes_it test_later_unrelated_terminal_line_does_not_close_it test_reserved_key_namespace_is_owned_by_its_library diff --git a/tests/fm-wake-drain-unread-status.test.sh b/tests/fm-wake-drain-unread-status.test.sh index ca0e2ba5ec..f80258a215 100755 --- a/tests/fm-wake-drain-unread-status.test.sh +++ b/tests/fm-wake-drain-unread-status.test.sh @@ -253,8 +253,8 @@ test_open_decisions_fold_is_unchanged() { || fail "OPEN DECISIONS no longer surfaces a buried needs-decision: $(cat "$out")" grep -F 'task6 note: re-read acknowledgement' "$out" >/dev/null \ || fail "the unread note was not surfaced alongside the still-open decision: $(cat "$out")" - grep -F "close one by answering it: bin/fm-send.sh --resolve-key " "$out" >/dev/null \ - || fail "OPEN DECISIONS lost its answerer-closes hint" + grep -F "close it: bin/fm-send.sh task6 --resolve-key api-shape ''" "$out" >/dev/null \ + || fail "OPEN DECISIONS lost the answerer-closes command for the buried decision" printf 'resolved [key=api-shape]: went with REST\n' >> "$state/task6.status" FM_STATE_OVERRIDE="$state" "$DRAIN" > "$out" || fail "drain failed after resolving the keyed decision" diff --git a/tests/fm-watch-triage.test.sh b/tests/fm-watch-triage.test.sh index 1bee5cab9b..e139ec8b37 100755 --- a/tests/fm-watch-triage.test.sh +++ b/tests/fm-watch-triage.test.sh @@ -997,9 +997,10 @@ test_nonterminal_stale_paused_absorbed_then_resurfaced() { # fm-crew-state then authoritatively reports stopped rather than paused, but the # confirmed-dead agent plus the declared wait or captain-held transfer must retain # bounded pause handling. -# A still-live agent at an external-decision gate is the disconfirming case: it -# must surface once, while the unchanged hash must not append the same wake on -# every watcher re-arm. +# A still-live agent under a captain-held transfer is the disconfirming case: the +# crew never declared that wait itself, so an unconfirmable crew whose agent is +# still live must surface once, while the unchanged hash must not append the same +# wake on every watcher re-arm. test_exited_declared_pause_is_bounded_but_live_gate_surfaces() { local dir state fakebin out capture_file statusf window key pane_hash sig pid back round wakes bare dir=$(make_case exited-declared-pause); state="$dir/state"; fakebin="$dir/fakebin" @@ -1071,17 +1072,17 @@ test_exited_declared_pause_is_bounded_but_live_gate_surfaces() { window="test:fm-gate" printf 'idle external-decision gate\n' > "$capture_file" printf 'window=%s\nkind=ship\nharness=grok\nbackend=tmux\n' "$window" > "$state/gate.meta" - printf 'paused: waiting at an active external-decision gate\n' > "$statusf" + printf 'captain-held [key=gate]: waiting at an active external-decision gate\n' > "$statusf" sig=$(seen_sig "$statusf"); printf '%s' "$sig" > "$state/.seen-gate_status" key=$(printf '%s' "$window" | tr ':/.' '___') pane_hash=$(hash_text "idle external-decision gate") printf '%s' "$pane_hash" > "$state/.hash-$key" printf '1\n' > "$state/.count-$key" - # First sight must surface promptly so a live external-decision gate is not - # hidden behind the pause cadence. + # First sight must surface promptly so a live external-decision gate under a + # hold the crew never declared is not hidden behind the pause cadence. PATH="$fakebin:$PATH" FM_FAKE_TMUX_WINDOW="$window" FM_FAKE_TMUX_CAPTURE="$capture_file" \ - FM_FAKE_TMUX_CURRENT_COMMAND=grok FM_FAKE_CREW_STATE='state: paused · source: status-log · waiting at an active external-decision gate' \ + FM_FAKE_TMUX_CURRENT_COMMAND=grok FM_FAKE_CREW_STATE='state: unknown · source: none · captain hold, state unconfirmable' \ FM_STATE_OVERRIDE="$state" FM_CREW_STATE_BIN="$fakebin/fm-crew-state.sh" FM_PAUSE_RESURFACE_SECS=999 FM_POLL=1 FM_SIGNAL_GRACE=1 \ FM_CHECK_INTERVAL=999999 FM_HEARTBEAT=999999 "$WATCH" >> "$out" & pid=$! @@ -1094,7 +1095,7 @@ test_exited_declared_pause_is_bounded_but_live_gate_surfaces() { # a second possible-wedge wake. printf '%s\n' $(( $(date +%s) - 500 )) > "$state/.stale-since-$key" PATH="$fakebin:$PATH" FM_FAKE_TMUX_WINDOW="$window" FM_FAKE_TMUX_CAPTURE="$capture_file" \ - FM_FAKE_TMUX_CURRENT_COMMAND=grok FM_FAKE_CREW_STATE='state: paused · source: status-log · waiting at an active external-decision gate' \ + FM_FAKE_TMUX_CURRENT_COMMAND=grok FM_FAKE_CREW_STATE='state: unknown · source: none · captain hold, state unconfirmable' \ FM_STATE_OVERRIDE="$state" FM_CREW_STATE_BIN="$fakebin/fm-crew-state.sh" FM_STALE_ESCALATE_SECS=240 FM_PAUSE_RESURFACE_SECS=999 FM_POLL=1 FM_SIGNAL_GRACE=1 \ FM_CHECK_INTERVAL=999999 FM_HEARTBEAT=999999 "$WATCH" >> "$out" & pid=$! @@ -1109,7 +1110,142 @@ test_exited_declared_pause_is_bounded_but_live_gate_surfaces() { bare=$(awk -F '\t' -v w="$window" '$3 == "stale" && $4 == w && $5 == "stale: " w { n++ } END { print n + 0 }' "$state/.wake-queue") [ "$wakes" -eq 0 ] || fail "acknowledged external-decision surface replayed $wakes wakes" [ "$bare" -eq 0 ] || fail "acknowledged external-decision bare stale remained queued" - pass "exited declared-pause and captain-held panes use bounded pause cadence while a live decision gate still surfaces once" + pass "exited declared-pause and captain-held panes use bounded pause cadence while a live captain-held gate still surfaces once" +} + +# --- a LIVE crew's own declared pause is authoritative ---------------------- +# The live 2026-08-12 case: two crews held a running dev server and declared +# `paused:` while waiting on the captain. A running dev server repaints, so each +# poll settles on a NEW stale hash, and every new hash was a first sighting that +# re-read agent liveness and surfaced a contentless `stale: ` wake - +# roughly a dozen supervision turns during one quiescent captain review, while +# state/.watch-triage.log simultaneously logged the pane as absorbed. A crew that +# declared its own external wait is normally still alive, so agent liveness must +# not override the authoritative paused verdict; only the pause cadence may +# re-surface it. +test_live_declared_pause_absorbs_churny_stale_hashes() { + local dir state fakebin out capture_file statusf window key text pane_hash sig pid round wakes i + dir=$(make_case live-paused-churn); state="$dir/state"; fakebin="$dir/fakebin" + out="$dir/watch.out"; capture_file="$dir/pane.txt"; statusf="$state/held.status" + window="test:fm-held" + printf 'dev server listening, request log line 0' > "$capture_file" + printf 'window=%s\nkind=ship\nharness=grok\nbackend=tmux\n' "$window" > "$state/held.meta" + printf 'paused: holding the dev server while the captain reviews\n' > "$statusf" + sig=$(seen_sig "$statusf"); printf '%s' "$sig" > "$state/.seen-held_status" + key=$(printf '%s' "$window" | tr ':/.' '___') + + # One long-lived watcher, exactly as production runs it: the churn comes from + # the held dev server repainting the pane, not from re-arming the watcher. + PATH="$fakebin:$PATH" FM_FAKE_TMUX_WINDOW="$window" FM_FAKE_TMUX_CAPTURE="$capture_file" \ + FM_FAKE_TMUX_CURRENT_COMMAND=grok \ + FM_FAKE_CREW_STATE='state: paused · source: status-log · holding the dev server' \ + FM_STATE_OVERRIDE="$state" FM_CREW_STATE_BIN="$fakebin/fm-crew-state.sh" \ + FM_PAUSE_RESURFACE_SECS=3600 FM_STALE_ESCALATE_SECS=240 FM_POLL=1 FM_SIGNAL_GRACE=1 \ + FM_CHECK_INTERVAL=999999 FM_HEARTBEAT=999999 "$WATCH" > "$out" & + pid=$! + + round=1 + while [ "$round" -le 4 ]; do + # Each repaint settles on a stale hash the watcher has never triaged before, + # so every round is a fresh first sighting - the shape that used to re-read + # agent liveness and queue another contentless wake. + text="dev server listening, request log line $round" + printf '%s' "$text" > "$capture_file" + pane_hash=$(hash_text "$text") + i=0 + while [ "$i" -lt 100 ]; do + [ "$(cat "$state/.stale-$key" 2>/dev/null || true)" = "$pane_hash" ] && break + kill -0 "$pid" 2>/dev/null || break + sleep 0.1 + i=$((i + 1)) + done + kill -0 "$pid" 2>/dev/null \ + || { wait "$pid" 2>/dev/null || true; fail "a live declared pause woke firstmate on churny stale hash $round: $(cat "$out")"; } + [ "$(cat "$state/.stale-$key" 2>/dev/null || true)" = "$pane_hash" ] \ + || { reap "$pid"; fail "churny paused hash $round was never absorbed (stale suppressor not advanced)"; } + [ -e "$state/.paused-$key" ] || { reap "$pid"; fail "churny paused absorb $round lost the pause cadence marker"; } + [ ! -e "$state/.stale-since-$key" ] || { reap "$pid"; fail "churny paused absorb $round started the wedge timer"; } + round=$((round + 1)) + done + reap "$pid" + + [ ! -s "$out" ] || fail "a live declared pause printed a wake reason across churny hashes: $(cat "$out")" + wakes=$(awk -F '\t' -v w="$window" '$3 == "stale" && $4 == w { n++ } END { print n + 0 }' "$state/.wake-queue" 2>/dev/null || echo 0) + [ "$wakes" -eq 0 ] || fail "a live declared pause queued $wakes stale wakes across four churny hashes" + pass "a live crew's declared pause absorbs every churny stale hash instead of queueing bare stale wakes" +} + +# The disconfirming case for the absorb above: the same live, idle, churny pane +# WITHOUT a declared pause must still surface immediately, on the unchanged +# schedule, so the fix cannot silence a genuinely quiet crew. +test_live_idle_crew_without_declared_pause_still_surfaces() { + local dir state fakebin out drain_out capture_file statusf window key pane_hash sig pid + dir=$(make_case live-idle-no-pause); state="$dir/state"; fakebin="$dir/fakebin" + out="$dir/watch.out"; drain_out="$dir/drain.out"; capture_file="$dir/pane.txt" + statusf="$state/quiet.status"; window="test:fm-quiet" + printf 'dev server listening, request log line 1' > "$capture_file" + printf 'window=%s\nkind=ship\nharness=grok\nbackend=tmux\n' "$window" > "$state/quiet.meta" + printf 'working: wiring the dev server\n' > "$statusf" + sig=$(seen_sig "$statusf"); printf '%s' "$sig" > "$state/.seen-quiet_status" + key=$(printf '%s' "$window" | tr ':/.' '___') + pane_hash=$(hash_text "dev server listening, request log line 1") + printf '%s' "$pane_hash" > "$state/.hash-$key" + printf '1\n' > "$state/.count-$key" + + PATH="$fakebin:$PATH" FM_FAKE_TMUX_WINDOW="$window" FM_FAKE_TMUX_CAPTURE="$capture_file" \ + FM_FAKE_TMUX_CURRENT_COMMAND=grok \ + FM_FAKE_CREW_STATE='state: unknown · source: none · no run, idle pane' \ + FM_STATE_OVERRIDE="$state" FM_CREW_STATE_BIN="$fakebin/fm-crew-state.sh" \ + FM_PAUSE_RESURFACE_SECS=3600 FM_STALE_ESCALATE_SECS=240 FM_POLL=1 FM_SIGNAL_GRACE=1 \ + FM_CHECK_INTERVAL=999999 FM_HEARTBEAT=999999 "$WATCH" > "$out" & + pid=$! + wait_for_exit "$pid" 40 || fail "an idle crew without a declared pause did not surface" + grep -Fx "stale: $window" "$out" >/dev/null || fail "idle non-paused crew did not surface its stale wake" + [ ! -e "$state/.paused-$key" ] || fail "an idle crew without a declared pause was given the pause cadence" + FM_STATE_OVERRIDE="$state" "$DRAIN" > "$drain_out" 2>/dev/null || fail "drain after the non-paused stale failed" + grep "$(printf '\tstale\t')" "$drain_out" | grep -F "$window" >/dev/null \ + || fail "idle non-paused crew's stale wake was not queued" + pass "a live idle crew that declared no pause still surfaces its stale wake on the unchanged schedule" +} + +# A declared pause is absorbed, never suppressed forever: once the pause itself +# is older than the bounded window, the same live crew re-surfaces once as a +# labeled recheck so a wait that quietly stopped being true cannot rot invisibly. +test_live_declared_pause_resurfaces_on_long_cadence() { + local dir state fakebin out drain_out capture_file statusf window key pane_hash sig pid back + dir=$(make_case live-paused-resurface); state="$dir/state"; fakebin="$dir/fakebin" + out="$dir/watch.out"; drain_out="$dir/drain.out"; capture_file="$dir/pane.txt" + statusf="$state/held.status"; window="test:fm-held" + printf 'dev server listening, idle' > "$capture_file" + printf 'window=%s\nkind=ship\nharness=grok\nbackend=tmux\n' "$window" > "$state/held.meta" + printf 'paused: holding the dev server while the captain reviews\n' > "$statusf" + back=$(( $(date +%s) - 500 )) + if [ "$(uname)" = Darwin ]; then touch -mt "$(date -r "$back" '+%Y%m%d%H%M.%S')" "$statusf" + else touch -m -d "@$back" "$statusf"; fi + sig=$(seen_sig "$statusf"); printf '%s' "$sig" > "$state/.seen-held_status" + key=$(printf '%s' "$window" | tr ':/.' '___') + pane_hash=$(hash_text "dev server listening, idle") + printf '%s' "$pane_hash" > "$state/.hash-$key" + printf '1\n' > "$state/.count-$key" + + PATH="$fakebin:$PATH" FM_FAKE_TMUX_WINDOW="$window" FM_FAKE_TMUX_CAPTURE="$capture_file" \ + FM_FAKE_TMUX_CURRENT_COMMAND=grok \ + FM_FAKE_CREW_STATE='state: paused · source: status-log · holding the dev server' \ + FM_STATE_OVERRIDE="$state" FM_CREW_STATE_BIN="$fakebin/fm-crew-state.sh" \ + FM_PAUSE_RESURFACE_SECS=240 FM_STALE_ESCALATE_SECS=240 FM_POLL=1 FM_SIGNAL_GRACE=1 \ + FM_CHECK_INTERVAL=999999 FM_HEARTBEAT=999999 "$WATCH" > "$out" & + pid=$! + wait_for_exit "$pid" 40 || fail "a live declared pause past the bounded window never re-surfaced" + grep -F "stale: $window" "$out" >/dev/null || fail "the long-cadence recheck printed no stale wake" + grep -F "awaiting external" "$out" >/dev/null || fail "the long-cadence recheck was not labeled an external-wait recheck" + grep -F "possible wedge" "$out" >/dev/null && fail "a live declared pause was mislabeled a possible wedge" + grep -Fx "stale: $window" "$out" >/dev/null && fail "the long-cadence recheck degraded to a bare stale wake" + [ -e "$state/.paused-resurfaced-$key" ] || fail "the long-cadence recheck left no throttle marker" + [ ! -e "$state/.stale-since-$key" ] || fail "a live declared pause recheck used the wedge timer" + FM_STATE_OVERRIDE="$state" "$DRAIN" > "$drain_out" 2>/dev/null || fail "drain after the paused recheck failed" + grep "$(printf '\tstale\t')" "$drain_out" | grep -F "awaiting external" >/dev/null \ + || fail "the paused recheck was not queued as an external-wait recheck" + pass "a live declared pause whose wait has gone stale still re-surfaces once on the long cadence" } test_secondmate_paused_resurfaces_in_normal_mode() { @@ -2641,6 +2777,9 @@ test_busy_declared_pause_is_rechecked_not_wedge_escalated test_nonterminal_stale_not_working_surfaced test_nonterminal_stale_paused_absorbed_then_resurfaced test_exited_declared_pause_is_bounded_but_live_gate_surfaces +test_live_declared_pause_absorbs_churny_stale_hashes +test_live_idle_crew_without_declared_pause_still_surfaces +test_live_declared_pause_resurfaces_on_long_cadence test_secondmate_paused_resurfaces_in_normal_mode test_secondmate_captain_held_resurfaces_in_normal_mode test_secondmate_nonpaused_stale_remains_suppressed