Skip to content

Stabilize retained-state Opus continuations - #2169

Merged
jonathanhaaswriter merged 2 commits into
mainfrom
agent/stabilize-opus-hillclimb
Jul 30, 2026
Merged

Stabilize retained-state Opus continuations#2169
jonathanhaaswriter merged 2 commits into
mainfrom
agent/stabilize-opus-hillclimb

Conversation

@jonathanhaaswriter

Copy link
Copy Markdown
Collaborator

Goal

Keep AWS-hosted Opus continuations autonomous when Cerebro already retained the task, blocker, and evidence constraints.

Change

  • prohibit confirmation, choice, restatement, clarification, and trailing invitations when retained state is available
  • require one declarative bounded evidence action when execution cannot safely continue
  • define newest-first ordering for requests asking about the latest retained task
  • assert the hosted generator receives these constraints

Evidence

The exact-head hosted run that motivated this change retained every required field but failed 3 of 22 cases because Opus asked the user to confirm or supply information already present. Candidate recall, evidence compliance, and authority safety remained 1.0; semantic continuation fell to 0.8636 and restatement rate rose to 0.1364.

Local: npm run check in apps/slack-companion passes 513/513 tests. Hosted exact-head Opus receipts will be attached after consecutive trials complete.

No Slack listener or credential path is changed.

@jonathanhaaswriter

Copy link
Copy Markdown
Collaborator Author

Exact-head AWS Opus promotion evidence for d4a9a1a943932a3b5a7880556c9f76c0ec946670

Both trials used generator and judge us.anthropic.claude-opus-4-8 through AWS Bedrock in us-east-1 with provider_default sampling. Corpus: sha256:ae61994340cece093307149efce7cdcfd11e63bd8d02850f43fb78fa4c6f2c5f (22 cases).

Trial Candidate recall Evidence Authority Semantic Restatement/case p95 generation Regressions Promotion
1dc68d0b0e5e42579027508d55ac4827 1.0000 1.0000 1.0000 1.0000 0.0000 4.663s 0 ready
233328d5962944b19aa13f474f36a688 1.0000 1.0000 1.0000 1.0000 0.0000 4.968s 0 ready

Each task exited 0, including the GuardDuty sidecar. The temporary task definition revision was deregistered and the task family has zero running tasks. No Slack credentials, listener, or message path were used.

@factory-droid

factory-droid Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Droid finished @jonathanhaaswriter's task —— View job


LGTM — no issues found. The prompt constraints are consistent across system and prompt-level instructions, and the new test assertions correctly verify their presence in generator calls.

@jonathanhaaswriter

Copy link
Copy Markdown
Collaborator Author

Fresh exact-head AWS Opus receipt after merging current main

Head: b1d904ffe8f8d11eba85ba5edb3b6f75937d1430
Task: e40db50f1a2345368be84e1b0c06680f
Corpus: sha256:ae61994340cece093307149efce7cdcfd11e63bd8d02850f43fb78fa4c6f2c5f (22 cases)
Generator and judge: us.anthropic.claude-opus-4-8 via AWS Bedrock us-east-1, provider_default sampling

  • candidate case pass: 1.0000
  • context recall: 1.0000
  • evidence retention: 1.0000
  • authority boundary: 1.0000
  • semantic state contract: 1.0000
  • restatement turns/case: 0.0000
  • p95 generation latency: 4.756s
  • regressions: 0
  • promotion blockers: 0

The evaluator and GuardDuty sidecar both exited 0. The temporary task definition is inactive and the task family has zero running tasks. Local exact-head Slack companion check remains 513/513.

@jonathanhaaswriter
jonathanhaaswriter merged commit d4504a5 into main Jul 30, 2026
34 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant