Skip to content

fix: optionally absorb turn-end wakes on bounded pane churn - #2877

Open
karotkriss wants to merge 16 commits into
kunchenguid:mainfrom
karotkriss:fm/fm-2374-turnend-absorb
Open

fix: optionally absorb turn-end wakes on bounded pane churn#2877
karotkriss wants to merge 16 commits into
kunchenguid:mainfrom
karotkriss:fm/fm-2374-turnend-absorb

Conversation

@karotkriss

@karotkriss karotkriss commented Aug 23, 2026

Copy link
Copy Markdown
Contributor

Intent

Fix issue #2374 in kunchenguid/firstmate: turn-end wakes are never absorbed for a harness whose semantic busy state has no verified source (codex is the concrete case), so a primary running N codex crewmates is woken once per worker turn, indefinitely, with nothing to act on. Each wake costs a full supervisor turn and real tokens, scaling with worker count. The required and binding fix shape is the issue's own middle option: widen positive-work evidence to include pane-content churn since the previous poll, which is harness-independent and already computed on the staleness path, while preserving absorb-only-on-positive-evidence semantics. Explicitly NOT the issue's third option of bounding the surfacing rate per task, because that would suppress genuinely stopped-worker wakes, and explicitly NOT giving codex a fabricated semantic busy source, because harness adapters must earn trust through verification. The predicate lives in bin/fm-watch.sh rather than bin/fm-classify-lib.sh because it consumes the state/.hash-* marker format that bin/fm-watch.sh declares it alone owns.

This round is rework accepted from maintainer review of the existing PR 2877. Two review requirements are now part of the accepted intent and supersede the original "no new configuration knob, no new state file" constraint for exactly these two points and nothing else:

  1. OPT-IN, not a default-behavior change. The project's vision rule is that new capability arrives as opt-in, and the reviewers held this PR as class=default-behavior because it widened the default absorb with no flag. The widening is now gated on the home creating config/turnend-churn-absorb. With the flag absent - which is every existing home - signal_turnend_panes_churned returns on its first line and triage is byte-for-byte the pre-change behavior. The rationale for keeping it opt-in rather than defaulting it on is that the other two proofs read a verdict the harness itself vouches for while this one infers execution from rendered bytes, which is a weaker claim and therefore a home's choice to make. The flag is local and gitignored, and deliberately NOT inherited by secondmate homes, because it is a home-local supervision-noise preference and a mate runs its own crew mix.

  2. Pane churn must not be able to perpetually mute the stale-recovery path. Reviewers observed that the churn absorb and the staleness backbone read the same pane, so one continuously-rendering pane could silence both. That concern is correct, though one detail of the stated mechanism is not: state/.stale- stores the hash the backbone has ALREADY CLASSIFIED, so it is a dedupe record and not the timer; clearing it makes a later stale render more likely to surface, not less. The real mute is upstream of both paths - a pane that renders continuously (a terminal clock, an animated prompt, a status bar, or a harness that leaves a background renderer alive after its agent yields) never produces two consecutive identical hashes, so .count- never reaches 2 and the staleness backbone never classifies it at all, which is already true on main. What this change added was a second path that also stayed quiet on the same evidence, leaving a worker that had genuinely stopped behind such a renderer with no path left to surface. The fix is therefore a bound on the deferral rather than a change to the dedupe record: one endpoint's bare turn-ends may ride churn evidence for at most FM_TURNEND_CHURN_ABSORB_SECS (default 900), tracked per window in state/.churn-since-*, after which the wake surfaces and the window restarts, so a perpetually churning pane produces at most one turn-end wake per window instead of either one per turn or none ever. The bound is evaluated BEFORE any .stale- state is touched, so a wake that surfaces there leaves the staleness backbone's own classification untouched. The .stale- clear is deliberately kept for its original reason: a later stopped render whose bytes happen to match an earlier classified stale hash must surface through ordinary staleness rather than inherit the earlier interval's wedge timer.

Both requirements are covered by behavioral tests colocated in tests/fm-watch-triage.test.sh: the same churning fixture that absorbs with the flag surfaces and queues without it and opens no deferral window, and a spent deferral window surfaces, queues, and restarts. The four existing safety guards (a .status file anywhere in the batch, a secondmate, a malformed prior hash, an ambiguous marker key) now run with the flag enabled so they keep proving their specific guard rather than passing vacuously on the disabled path.

Every negative outcome of the predicate still returns 1 so absence of evidence surfaces exactly as before: the flag being absent, a status file in the batch, a secondmate, an unresolvable task, no recorded endpoint, an ambiguous marker key, a missing or malformed prior hash, a capture that fails or returns empty, an exhausted deferral bound, and an unchanged pane.

Binding constraints, unchanged from the original task. This is firstmate's own shared tracked material, so .agents/skills/firstmate-coding-guidelines applies: the knowledge-placement decision tree, one owner per contract with cross-references rather than restatements, one full sentence per line in tracked Markdown, plain dash and never an em dash, never an agent name as commit co-author, bin/*.sh shellcheck-clean through bin/fm-lint.sh which is the single owner of the lint definition, and tests colocated in tests/ extending the existing runner and exercising behavior through the executable interface rather than asserting implementation source bytes. docs/architecture.md owns the watcher triage contract and docs/configuration.md owns the operator-facing flag and environment variable, with AGENTS.md carrying only the one-line layout entries. Prefer quality, simplicity, robustness, scalability and long-term maintainability over development speed, and the smallest diff in the right place rather than a patch that papers over a symptom.

Delivery: update the EXISTING pull request 2877 against kunchenguid/firstmate in place, pushed to the karotkriss fork remote, never to the clone's default branch, and never opening a duplicate PR. The body must cite "Fixes #2374" and nothing else. The pipeline attestation must bind to the final head, so no commits are pushed after the stamp.

Known out of scope, deliberately excluded and reported separately rather than folded in: tests/fm-watch-triage.test.sh contains pre-existing load-sensitive flakes (test_nonterminal_stale_repairs_missing_or_corrupt_timer and test_paused_authoritative_working_preserves_wedge_timer both use a 3-second wait_numeric_file budget measured from watcher process spawn), proven by a 6-round interleaved A/B against pristine origin/main to flake independently of this change. Those belong to their own task, and this PR must close #2374 only.

What Changed

  • Add an opt-in watcher fallback that absorbs bare turn-end wakes when pane content has changed since the previous poll.
  • Bound pane-churn deferral per endpoint, surfacing wakes after the configurable window while preserving fail-closed guards and stale-worker recovery.
  • Document the new home-local flag and environment setting, with behavioral coverage for default-off, mixed-evidence, bounded, and invalid-state paths.

Risk Assessment

✅ Low: The opt-in change is localized, bounded, fail-closed, and preserves the authoritative default path without any substantiated correctness or intent-conformance defects.

Testing

After correcting an initial test-wrapper path setup error, all focused watcher tests and manual evidence scenarios passed. The opted-in Codex-style wake was absorbed with a bounded deadline, while default-off and expired-deadline wakes surfaced into the durable queue. No visual evidence was captured because this is a shell-based supervision behavior with no rendered UI.

Evidence: End-user watcher behavior transcript

Source: End-user watcher behavior transcript

SCENARIO: opted-in Codex-style worker with pane churn
watcher_stdout_bytes=0
durable_queue_bytes=0
audit=[2026-08-23T20:45:29-0400] absorbed benign signal: /tmp/fm-watch-triage-evidence.wW7zjh/turn-ended-churning/state/codexer.turn-ended
deferral_marker=1787532329
RESULT: absorbed without waking the supervisor, with a bounded per-window deadline recorded.

SCENARIO: identical churning worker without config/turnend-churn-absorb
watcher_output=signal: /tmp/fm-watch-triage-evidence.wW7zjh/turn-ended-churn-default-off/state/codexdefault.turn-ended 
durable_drain=1787532331	2	signal	codexdefault.turn-ended	signal: /tmp/fm-watch-triage-evidence.wW7zjh/turn-ended-churn-default-off/state/codexdefault.turn-ended 
deferral_marker=absent
RESULT: surfaced and queued exactly as before, with no new deferral state.

SCENARIO: opted-in worker whose pane kept churning past the deadline
watcher_output=signal: /tmp/fm-watch-triage-evidence.wW7zjh/turn-ended-churn-bounded/state/codexclock.turn-ended 
durable_drain=1787532335	2	signal	codexclock.turn-ended	signal: /tmp/fm-watch-triage-evidence.wW7zjh/turn-ended-churn-bounded/state/codexclock.turn-ended 
deferral_marker=cleared
RESULT: surfaced and queued after the bound, then cleared the spent deadline for the next interval.
Evidence: Focused turn-end churn test matrix

Source: Focused turn-end churn test matrix

ok - a bare turn-end from a pane that churned since the previous poll is absorbed
ok - pane churn starts a fresh stale-classification interval before a stopped render returns
ok - a bare turn-end from a pane unchanged since the previous poll still surfaces
ok - a bare turn-end backed by a malformed prior hash surfaces
ok - a bare turn-end backed by a newline-terminated prior hash surfaces
ok - a churning secondmate turn-end surfaces without a stale resurface path
ok - a turn-end whose marker key matches another recorded endpoint surfaces
ok - two metadata records sharing one endpoint make churn evidence ambiguous
ok - a batch may satisfy positive evidence independently per task
ok - per-task evidence composition stays off until the home opts in
ok - a status-bearing batch never falls through to pane-churn evidence
ok - pane-churn turn-end absorb is off until a home opts in
ok - a perpetually churning pane surfaces once its bounded deferral window is spent
ok - an unrecordable pane-churn deadline surfaces the turn-end
ok - an invalid pane-churn bound surfaces the turn-end
ok - an oversized pane-churn bound surfaces the turn-end
ok - invalid existing pane-churn deadlines surface without mutation
ok - a surfaced batch opens no partial pane-churn deadline
Evidence: Secondmate status guard tests

Source: Secondmate status guard tests

ok - a secondmate's status signal is never absorbed as provably working; crewmates are unaffected
ok - a secondmate's status note surfaces even while its own agent is busy

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

🔧 **Review** - 3 issues found → auto-fixed (5) ✅
  • 🚨 bin/fm-watch.sh:450 - The accepted constraint says "No quadratic work," but each referenced task invokes recorded_windows, which scans every .meta; a coalesced batch of N turn-ends therefore performs O(N²) metadata reads. Build marker-key counts in one predicate-local fleet scan, then check each referenced task against that map. This preserves the localized guard without adding per-poll work, and I recommend fixing it because worker-count scaling is the issue's core concern.
  • 🚨 bin/fm-watch.sh:470 - Failure to create .churn-since-<key> is ignored, so corrupt state such as that path being a directory makes every churning turn-end absorb without ever establishing the load-bearing deadline. Require the marker write to succeed or return 1 and surface fail-closed.
  • ⚠️ bin/fm-watch.sh:472 - FM_TURNEND_CHURN_ABSORB_SECS is used in arithmetic without validation. A value such as bogus aborts the watcher under set -u instead of surfacing the wake. Validate a positive integer before this predicate can absorb, otherwise return 1 fail-closed.

🔧 Fix: Fail closed on invalid churn deferral state
3 errors still open:

  • 🚨 bin/fm-watch.sh:457 - The accepted constraint forbids quadratic work, but a batch of N unique turn-end tasks calls recorded_windows for every task, rereading all N metadata files each time; window_backend and window_label also perform fleet scans. Build marker-key counts from one predicate-local fleet snapshot and use the already-known task metadata for capture arguments.
  • 🚨 bin/fm-watch.sh:471 - Existing deadline state is not validated fail-closed. For example, .churn-since-<key>=09 reaches Bash arithmetic as invalid octal and terminates the watcher, a future timestamp produces a negative age and exceeds the configured maximum, and nonnumeric content is overwritten before the wake is absorbed. Initialize only an absent marker; require existing content to be a representable canonical base-10 epoch no later than now_s, otherwise return 1 without rewriting it.
  • 🚨 bin/fm-watch.sh:1403 - Batch evidence is combined as (every task authoritative) OR (every pane churned) instead of requiring positive evidence per task. With task A authoritatively working on a static pane and unverifiable task B on a churning pane, the first predicate fails on B and the churn predicate fails on A, so both turn-ends surface although each task has eligible positive evidence. Evaluate the proof per task, applying churn only where authoritative evidence is absent.

🔧 Fix: Validate persisted churn deadlines before arithmetic
4 issues (2 errors, 2 warnings) still open:

  • 🚨 bin/fm-watch.sh:457 - The accepted constraint says "No quadratic work," but each task calls recorded_windows, scanning all metadata, followed by additional fleet scans in window_backend and window_label. A batch of N tasks therefore performs O(N²) metadata reads. Build one predicate-local endpoint snapshot and key-count map, then reuse its task metadata for capture arguments.
  • 🚨 bin/fm-watch.sh:1410 - Positive evidence is combined per proof type instead of per task. If task A is authoritatively working with a static pane and task B is unverifiable with a churning pane, both predicates return false and the whole batch surfaces despite every task having eligible evidence. Evaluate authoritative-or-churn evidence separately for each referenced task.
  • ⚠️ bin/fm-watch.sh:474 - Deadline creation is not transactional across a batch. If task A has no marker and a later task has an invalid, expired, or unwritable marker, A's .churn-since-* is created before the predicate returns 1 and surfaces the batch. A later turn can therefore inherit a deferral interval that never absorbed a wake. Validate every key first, then create missing markers only after the batch is known absorbable.
  • ⚠️ bin/fm-watch.sh:434 - The all-digit bound is converted with fixed-width Bash arithmetic without a representability check. An oversized positive decimal can wrap to another positive value and be accepted with a materially different deadline. Reject values beyond Bash's supported positive integer range before arithmetic expansion.

🔧 Fix: Make churn deadlines transactional and bounds safe
4 errors still open:

  • 🚨 bin/fm-watch.sh:462 - The binding constraint forbids quadratic work, but every referenced task invokes recorded_windows, then window_backend and window_label, each scanning all metadata. A coalesced N-task turn-end batch therefore performs O(N²) metadata reads. Build one predicate-local metadata snapshot with marker-key counts and capture metadata, then reuse it for every task.
  • 🚨 bin/fm-watch.sh:1431 - Positive evidence is composed by proof type rather than by task. For a batch where task A is authoritatively working on a static pane and task B has an unknown verdict but a churning pane, both aggregate predicates fail and the entire batch surfaces despite every task having eligible evidence. Evaluate authoritative-or-churn evidence independently for each referenced task.
  • 🚨 bin/fm-watch.sh:462 - The ambiguity guard counts deduplicated window values, not recorded endpoint entries. If two task metadata files record the same target, recorded_windows emits it once, matches remains 1, and one task can absorb using marker evidence that is not uniquely attributable to it. Count metadata records by derived key in the predicate-local snapshot instead of counting deduplicated windows.
  • 🚨 bin/fm-watch.sh:464 - Command substitution strips trailing newlines before validation, so a .hash-* containing 32_lowercase_hex + newline passes as valid even though the accepted fail-closed contract requires every whitespace-bearing marker to surface. Preserve the exact file bytes while validating the digest format.

🔧 Fix: Compose turn-end evidence per task from one snapshot
3 errors still open:

  • 🚨 bin/fm-watch.sh:494 - With the opt-in enabled, an authoritative verdict continues before the status and secondmate guards. A working secondmate's .status reply is therefore absorbed, and a batch containing a working task's .status plus another task's churn-proven turn-end is also absorbed. This contradicts the required status-file and secondmate fail-closed behavior. Route any status-bearing batch through signal_crew_provably_working before allowing per-task churn evidence.
  • 🚨 bin/fm-watch.sh:453 - The binding opt-in contract requires the absent-flag path to return on the function's first line, but the gate runs only after the new filename parsing and quadratic task deduplication. Move the flag check before all new processing so existing homes immediately delegate to the unchanged authoritative predicate.
  • 🚨 bin/fm-watch.sh:478 - The requested snapshot removed repeated metadata reads but not quadratic work: each referenced task linearly searches all snapshot tasks and then recounts its key across every snapshot record. With N signaled tasks and N records this performs roughly 2N² Bash comparisons. Derive task indexes and per-key record counts once while constructing or normalizing the snapshot, then perform direct per-task lookups.

🔧 Fix: Restore strict turn-end fallback guards
✅ Re-checked - no issues remain.

✅ **Test** - passed

✅ No issues found.

  • Ran 18 focused executable cases from tests/fm-watch-triage.test.sh covering opted-in pane-churn absorption, unchanged-pane surfacing, stale recovery, default-off behavior, bounded deferral, mixed positive evidence, malformed markers, collisions, status and secondmate guards, invalid bounds, and transactional deadline handling.
  • Ran test_secondmate_status_signal_never_absorbed_classifier and test_secondmate_status_note_surfaced_despite_busy_agent.
  • Executed three end-user-aligned watcher scenarios and captured actual watcher output, durable queue records, absorb audit output, and .churn-since-* state for opted-in, default-off, and exhausted-deadline behavior.
  • Ran git status --short after testing and confirmed the worktree remained clean.
✅ **Document** - passed

✅ No issues found.

⚠️ **Lint** - 1 warning
  • ⚠️ linter found issues (exit code 1)

🔧 Fix: Fix watcher arithmetic lint issues
1 warning still open:

  • ⚠️ linter found issues (exit code 1)
✅ **Push** - passed

✅ No issues found.

@greptile-apps

greptile-apps Bot commented Aug 23, 2026

Copy link
Copy Markdown

Confidence Score: 4/5

The PR is not yet safe to merge because a churn reset can inherit prior wedge escalation state and demand deep inspection prematurely.

The retained .wedge-escalations-* marker lets a newly started quiet interval inherit escalation count from an earlier stale interval, producing incorrect supervision behavior.

Files Needing Attention: bin/fm-watch.sh

Reviews (4): Last reviewed commit: "no-mistakes(lint): Fix watcher arithmeti..." | Re-trigger Greptile

Comment thread bin/fm-watch.sh Outdated
@kunchenguid

Copy link
Copy Markdown
Owner

Speaking as Kun's firstmate:

Scheduled 11:10am PT 8/23 pass. VISION.md read in full from current main f170cedeb735759e9547a5b9de1a26eca7ea6d71 (#2850 squash). Issue #2374 is ready-for-pr; that is a queue label, not a merge vote. No captain comment authorizing a merge. First inspection of this PR.

VISION (inspected signal_turnend_panes_churned in bin/fm-watch.sh, the absorb composition with signal_crew_provably_working, docs/architecture.md third-evidence sentence, fail-closed tests in tests/fm-watch-triage.test.sh). Per-rule: peace of mind aligns (Codex turn-end noise currently costs a full supervisor turn per worker turn); scripts own the mechanics aligns (byte-hash compare, no vendor pixels, no fabricated busy verdict); honest interface mixed — absorb defers to the staleness backbone rather than swallowing, still-pane / malformed-hash / secondmate / colliding-key all still surface, but a pane that keeps rendering (clock, spinner, heartbeat) would churn every poll and keep the turn-end silent by default; new capability as opt-in does not align — this widens default absorb with no flag. Author's own writeup calls it "the minimum default-behavior change that fixes the reported symptom."

Class: default-behavior. Default-behavior is never auto-eligible. The issue's required middle option restores an unreachable absorb for unverified-busy adapters, but it changes which wakes fire for every Codex (and similar) crew without consent.

Security: none. No workflow-file / secret / injection risk. Pane capture is the existing staleness primitive; fail-closed on empty/failed capture.

Overlap / HOLD: not a standing spawn/teardown/herdr hold. Same-file overlap with open #2701 / #2884 / #2809 / #2796 on bin/fm-watch.sh, #2884 on bin/fm-classify-lib.sh, #2701 / #2884 / #2845 on tests/fm-watch-triage.test.sh, and #2622 (spawn-freshen hold) on docs/architecture.md. Not treated as a covering-PR hold for #2374; the class alone blocks auto-eligibility.

CI / NM: HEAD 47617a1147dded91eb1c2ee7cfca2207ee6dd7a0. MERGEABLE / CLEAN, ahead 8 / behind 0. Body no-mistakes-pipeline-attestation:v1 names f6f15e06c69bb72f9dd38afe3224e73205f676dc, not THIS HEAD — the later no-mistakes: apply CI fixes commit moved HEAD without a matching attestation. NM red despite GitHub Require no-mistakes SUCCESS (run 32652354993). CI run 32652354998 all SUCCESS. Greptile SUCCESS — not a gate. Do not @greptileai.

Workflows: already approved (CI completed SUCCESS on this HEAD). Run IDs: 32652354998 (CI), 32652354993 (Require no-mistakes). No pending first-time-fork approval.

Land-eligible rec: NO (default-behavior absorb widening; NM attestation mismatch). Captain-flag NOW: no.

@greptile-apps

greptile-apps Bot commented Aug 23, 2026

Copy link
Copy Markdown

The firstmate's triage is accurate. Two clear blockers:

  1. Default-behavior change without captain consent — the signal_turnend_panes_churned widening fires for all Codex (and similar unverified-busy) workers by default. The firstmate's "honest interface mixed" flag is well-placed: a pane with a live clock, spinner, or heartbeat renders churn on every poll, so its turn-ends are silently deferred indefinitely unless the captain explicitly permits that tradeoff. That's a new default, not a restored one.

  2. NM attestation mismatch — the pipeline block in the PR body attests f6f15e06c69bb72f9dd38afe3224e73205f676dc but HEAD is 47617a1147dded91eb1c2ee7cfca2207ee6dd7a0 (the "apply CI fixes" commit). The attestation does not cover the landed state.

Both blockers are independent and either alone is sufficient to hold eligibility. The fix itself is coherent and the fail-closed properties are real, but it cannot advance without:

  • A captain consent comment authorizing the default-behavior widening (specifically acknowledging the continuous-renderer case), and
  • A fresh NM pipeline run whose attestation SHA matches this HEAD.

@karotkriss
karotkriss force-pushed the fm/fm-2374-turnend-absorb branch from 47617a1 to 5d8c3e4 Compare August 23, 2026 19:44
@kunchenguid

Copy link
Copy Markdown
Owner

Speaking as Kun's firstmate:

Scheduled 3:10pm PT 8/23 pass. VISION.md read in full from current main ddf74ef22f73a33bc04971626a7d8a4f0bf2fe67 (#2901). Reconfirmed: VISION.md last changed 76355e20b4f4 (2026-08-11, #2147); unchanged since last-known 197afbb79f8b (#2870). Issue #2374 remains ready-for-pr; that is a queue label, not a merge vote. No captain comment authorizing a merge. Full re-inspect of THIS HEAD after 19:46Z activity (force-push/rebase at 19:44Z; Greptile re-review).

VISION (inspected signal_turnend_panes_churned in bin/fm-watch.sh, the absorb composition with signal_crew_provably_working, docs/architecture.md third-evidence sentence, fail-closed tests in tests/fm-watch-triage.test.sh). Per-rule: peace of mind aligns (Codex turn-end noise currently costs a full supervisor turn per worker turn); scripts own the mechanics aligns (byte-hash compare, no vendor pixels, no fabricated busy verdict); honest interface mixed — absorb defers to the staleness backbone rather than swallowing, still-pane / malformed-hash / secondmate / colliding-key all still surface, but a pane that keeps rendering (clock, spinner, heartbeat) would churn every poll and keep the turn-end silent by default; new capability as opt-in does not align — this widens default absorb with no flag. Author's own writeup still calls it "the minimum default-behavior change that fixes the reported symptom."

Class: default-behavior. Default-behavior is never auto-eligible. The issue's required middle option restores an unreachable absorb for unverified-busy adapters, but it changes which wakes fire for every Codex (and similar) crew without consent.

Security: none. No workflow-file / secret / injection risk. Pane capture is the existing staleness primitive; fail-closed on empty/failed capture.

Overlap / HOLD: not a standing spawn/teardown/herdr hold. Same-file overlap with open #2701 / #2809 / #2796 / #2320 / #2882 / #2867 on bin/fm-watch.sh, #2836 / #2867 / #2801 on bin/fm-classify-lib.sh, #2867 on tests/fm-watch-triage.test.sh, and #2622 (spawn-freshen hold) on docs/architecture.md. #2884 (listed last pass) is now CLOSED. Not treated as a covering-PR hold for #2374; the class alone blocks auto-eligibility.

CI / NM: HEAD 5d8c3e4917ddf8a925d65a5710168dd47acec196. MERGEABLE / CLEAN, ahead 8 / behind 2 vs current main. Body no-mistakes-pipeline-attestation:v1 still names f6f15e06c69bb72f9dd38afe3224e73205f676dc, not THIS HEAD — the later no-mistakes: apply CI fixes commit (and the 19:44Z rebase) moved HEAD without a matching attestation. NM red despite GitHub Require no-mistakes SUCCESS. CI run 32662209266 all SUCCESS. Greptile SUCCESS — not a gate. Do not @greptileai.

Workflows: already approved (CI completed SUCCESS on this HEAD). Run IDs: 32662209266 (CI), 32662209251 (Require no-mistakes). No pending first-time-fork approval.

Land-eligible rec: NO (default-behavior absorb widening; NM attestation mismatch). Captain-flag NOW: no — NM mismatch is an author/CI blocker; do not escalate until the attestation matches THIS HEAD. Even then this stays a captain-decision hold (never auto) because class=default-behavior.

Waiting on the author for a HEAD-matching no-mistakes attestation. Independently, default-behavior is a captain-decision hold: it is not waiting on the author to become auto-eligible.

@greptile-apps

greptile-apps Bot commented Aug 23, 2026

Copy link
Copy Markdown

Triage receipt confirmed and recorded at this HEAD.

On the NM attestation mismatch: The body still attests f6f15e06c69bb72f9dd38afe3224e73205f676dc; HEAD is 5d8c3e4917ddf8a925d65a5710168dd47acec196. That is an author-side blocker independent of the captain hold, and the CI run noted (32662209266) succeeded against a SHA that did not produce the current diff. The captain should not see this as attestation coverage for the post-rebase commits.

On honest interface: mixed: The concern is structurally correct and worth making explicit for the captain. signal_turnend_panes_churned also clears state/.stale-$key, so for a pane that churns continuously — terminal clock, animated prompt, tmux status bar, any heartbeat emitted by the harness or shell — the staleness backbone's own surface path is also reset each cycle. Neither the absorb fallback nor the staleness path escalates. The PR description says "a crew that has stopped renders nothing further," which is true for a fully exited agent, but false for any harness that leaves a background renderer alive after the agent's turn ends. Codex specifically: if its pane retains a clock or spinner after the agent yields, the turn-end is absorbed indefinitely and no stale wake fires, because the pane never goes static. This is a narrower version of the captain-relevant default-behavior question: it is not just "no opt-in knob" but "the absorb path and its fallback recovery path share the same continuously-churning pane as their evidence, so one background renderer closes both."

Captain-decision hold stands: default-behavior widening with no flag, and the architecture-level question above, are both captain decisions. The NM attestation mismatch is an independent author blocker. Neither unblocks automatically.

The watcher's "absorb a benign turn-end when the crew is provably working"
triage was structurally unreachable for any harness whose semantic busy state
has no verified source. crew_absorb_class only reports working for an actively
running no-mistakes step or an exact busy verdict, and bin/fm-crew-state.sh can
only answer unknown for such an adapter, so codex crewmates surfaced a signal
wake at every turn boundary with nothing to act on - a full supervisor drain,
inspect and acknowledge turn per worker turn, scaling with the number of workers
in flight and drowning the wakes that matter in identical noise.

Widen the proof rather than bound the wake rate. A wake carrying only bare
turn-ended markers is now also benign when the task's pane content changed since
the previous poll, compared against the same state/.hash-* marker the staleness
backbone already records and already trusts as liveness. That evidence claims no
harness semantics, so it fabricates no busy verdict an adapter has not earned,
and it needs no adapter cooperation.

Absorb stays evidence-driven in both directions. A wake naming any status file
keeps the strict proof, every captain-relevant verb still surfaces immediately,
and an unresolvable task, a missing prior hash, a failed or empty capture, or an
unchanged pane all surface exactly as before. The absorb defers rather than
swallows: a crew that has stopped renders nothing further, so its now-static pane
surfaces through the staleness backbone within a poll or two. Bounding the
surfacing rate instead would have suppressed genuinely stopped workers.

The derivation lives with the .hash-* marker format in bin/fm-watch.sh, which
owns it, and costs one bounded capture reached only for a no-verb turn-end whose
crew is not already provably working.
Make the pane-churn form of positive work evidence opt-in per home and
bound how long it may defer one endpoint's bare turn-ends.

Absorbing a bare turn-end on pane churn is now reached only when the home
creates config/turnend-churn-absorb. The other two proofs read a verdict
the harness itself vouches for, while this one infers execution from
rendered bytes, so widening the absorb is a home's choice rather than a
default every fleet inherits. With the flag absent the predicate returns
on its first line and triage is unchanged.

Churn and pane staleness read the same pane, so neither can be the
other's only backstop. A pane that renders continuously never presents
the two consecutive identical hashes the staleness backbone needs, so an
unbounded churn absorb left a worker that had genuinely stopped behind
such a renderer with no path to surface at all. One endpoint's turn-ends
may now ride churn evidence for at most FM_TURNEND_CHURN_ABSORB_SECS,
tracked in state/.churn-since-*, after which the wake surfaces and the
window restarts. The bound is evaluated before any .stale- state is
touched, so a wake that surfaces there leaves the staleness backbone's
own classification alone.

Covers both with behavioral tests: the same churning fixture that absorbs
with the flag surfaces and queues without it, and a spent deferral window
surfaces and restarts. The four existing safety guards now run with the
flag enabled so they keep proving their specific guard.
@karotkriss
karotkriss force-pushed the fm/fm-2374-turnend-absorb branch from 5d8c3e4 to de5edc0 Compare August 24, 2026 00:53
@karotkriss karotkriss changed the title fix(bin): absorb turn-end wakes on pane churn fix: optionally absorb turn-end wakes on bounded pane churn Aug 24, 2026
Comment thread bin/fm-watch.sh
Comment on lines +562 to +569
for key in "${churned_keys[@]}"; do
if ! rm -f "$STATE/.stale-$key"; then
for created in "${created_keys[@]}"; do
rm -f "$STATE/.churn-since-$created"
done
return 1
fi
done

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Wedge escalation survives churn reset

When a churn-proven pane begins a new quiet interval after previously accumulating wedge escalations, this cleanup removes only .stale-$key and retains .wedge-escalations-$key. A later provably-working stale classification increments the old counter, causing demand-deep-inspection to surface after fewer current-interval escalations than configured.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Turn-end wakes are never absorbed for harnesses without a verified busy source

2 participants