Repository navigation
Conversation
gloryfromca
left a comment
There was a problem hiding this comment.
Blocking: the recovery-notice path drops a filed turn scratch from replay.
I independently reviewed the full change and found one inline blocker. The ordinary string recovery path contradicts the byte-for-byte replay contract: the session writer files the scratch, but the history projection does not restore it.
Coverage: I checked AGENTS.md and the runtime terms in CONTEXT-MAP.md/CONTEXT.md; read the complete diff; traced assembly, recovery injection, persistence, both history projections, token estimation, and the relevant provider cache shaping; inspected the base and commit history; checked backward-compatibility behavior and the changed tests for weakened assertions; and checked the runtime layer boundaries touched here.
Verification: uv run pytest -n0 tests/test_context_replay_bytes.py tests/test_context_replay_join.py tests/test_context_runtime_context.py tests/test_context_turn_scratch.py tests/test_token_estimation.py -q passed: 172 tests in 9.29s. A direct reproduction through AgentLoop._save_turn produced a stored turn_scratch without runtime_context, then replayed_content omitted the scratch (MATCH: False).
…urn replays byte for byte A provider's cache is a prefix cache: the next turn reuses what the previous one sent only when the bytes match. Two pieces of the turn's user message carried per-turn text that the session write dropped, so the next turn's request differed from the previous one and the cache broke at the turn boundary. The skill menu rides in the same paragraph as the runtime-context lines, but the assembler recorded the lines alone. The turn scratch rides at the end of the message and was cut and discarded. Both are now filed with the message and rejoined on the way out: the block as `runtime_context`, the scratch as `turn_scratch`, the menu inside the block. What the provider saw is what the next turn sends. The stored `content` keeps the user's words alone. The transcript, the novelty window and every other reader of the session log see the question, not the recall. Every join is located by what identifies it, never by position. A notice put in front of the message after assembly moves the block off the content's start, and the replay found it by position before, which joined the block twice. The scratch is filed with the offset it stood at, because a hook's note lands on the message after the scratch was appended, so the scratch is not always last. The scratch's tag said it was "not conversation history". It is now filed and replayed, so the tag says what it is: the recall of the turn it was asked in. The ppt engine's ledger keeps its own copy of the two spellings, pinned against the trunk by a test, and it moves with them. Measured on a local probe that drives the real assembler, the real session write and the real history projection over fourteen turns: the common prefix between consecutive requests grows by the whole request per turn (1,749 bytes at the probe's size), against 328 bytes before, which was the part that never changed. The request itself grows by the same replayed text, and the stored session by the filed scratch. The tag change alters the bytes every live session sends, so every open cache misses once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ruff format wraps one long assertion in the new test file. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
replayed_content returned early whenever a user entry carried no runtime_context field, so the scratch filed on that entry was never put back. The recovery-notice path reaches exactly that shape: _inject_recovery_block prepends its notice to the assembled message, so the session write leaves the block in content and files none, while it still cuts and files the scratch. The replayed request then lost the recall the provider had seen, which is neither byte-identical nor the same question. The early return now covers only the block. The scratch is rejoined for every user entry, before the block, because where it stood is an offset into the message's own words. Reported by gloryfromca on #1334, reproduced through the real session writer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
9b21225 to
6348c37
Compare
gloryfromca
left a comment
There was a problem hiding this comment.
No blockers; this can merge as far as I am concerned.
The new revision fixes the outstanding recovery-path defect. I verified the delta against the full current PR diff and the surrounding assembly, recovery injection, persistence, history projection, and token-estimation callers. I also rechecked the applicable AGENTS.md and CONTEXT.md contracts, backward compatibility for entries without filed fields, architecture boundaries, and that the added tests strengthen rather than replace prior coverage. I found no new issue worth raising.
Verification: the original direct reproduction now reports MATCH: True; uv run pytest -n0 tests/test_context_replay_bytes.py tests/test_context_replay_join.py tests/test_context_runtime_context.py tests/test_context_turn_scratch.py tests/test_token_estimation.py -q passed, 174 tests in 9.21s. The original review thread is resolved.
There was a problem hiding this comment.
Blocking: filing every turn's scratch and replaying it on every later request makes the history grow by the whole recall each turn; on a plain chat session the request is 2.5x base by turn 50, compaction comes much sooner, and recall the memory store has since dropped keeps reaching the model.
_save_turn now files the scratch on each user entry (raven/agent/loop/turn_path.py:4376-4382) and replayed_content puts it back on every later request (raven/context_engine/runtime_context.py:105 -> scratch.py replayed_scratch). Nothing bounds it except compaction; the pruner never touches user messages. Base cut the scratch and discarded it, and its own docstring named this outcome as the reason ("the recalled text would accumulate in history forever").
Measured at 6348c37 merged onto 27becfa against base 27becfa, with the real ContextAssembler, the real AgentLoop._save_turn and SessionManager, a stub backend recalling 5 hits of ~500 chars plus the selected user.md sections (about 700 tokens of scratch a turn), one session, 50 turns, every third turn with a tool read:
request tokens t1 t5 t20 t50 replayed recall at t50 session file
base 3298 4807 11276 24502 0 122,458 B
head 3294 7913 26034 62588 34,936 (56%) 280,579 B
previous request fully reused at the next turn boundary: base 0/49, head 49/49
The cache claim is real: head re-sends the previous request byte for byte. But what it buys is the menu and one scratch per boundary, while every later request now carries every earlier scratch. Cached tokens are cheaper, not free, and the extra 38k tokens a request also fill the window: at a fixed window the filed recall alone reaches the compaction trigger many turns sooner and keeps re-reaching it, each compaction dropping the cached prefix and paying a summary call whose input is now mostly replayed recall. For a change titled perf, the default chat path gets more expensive, not less; the gain shows only on tool-heavy turns.
The same filing changes what the model is told: a fact recalled on turn 1 and then dropped from the store (a "forget this", a profile refresh) is still sent on every later request until compaction, inside the recall fence, and stays in the session file. Same probe: turn 1 recalls F, the backend and user.md then drop it; head's turn 2 request carries F in the history user entry, base's does not.
Shapes that keep the cache gain without the growth, for the author to choose from: file only the skill menu (small, and the part that sits in the runtime paragraph) and keep discarding the scratch; or replay the scratch for the most recent user turn only (the boundary that breaks the cache) and drop older ones; or cap the filed scratch. Whichever way it goes, CONTEXT.md's Turn Scratch entry (CONTEXT.md:1706, "never session.messages", and :1711-1713) has to match: today it says the opposite of the code, and of the Runtime Context entry this PR edited.
There was a problem hiding this comment.
Blocking: resolve the unbounded historical scratch replay before merge.
This evidence invalidates my earlier clean stance. I independently confirmed the core mechanism: a 50-user-message stored history projects all 50 filed scratches back into the request, with no age/count cap before compaction; in a synthetic 2,800-character-per-turn check, the stored-message estimate grew from 200 tokens without scratches to 70,490 with them. The existing Turn Scratch contract also still says the scratch never reaches session.messages, which confirms this storage and semantic expansion was not reconciled across the design.
The replay/runtime/token suite still passes (174 passed in 9.34s), but those tests assert byte-identical replay and therefore do not exercise whether retaining every prior recall is affordable or semantically safe. I am withdrawing my clean signoff while this thread remains outstanding.
|
Not a blocker: three more effects of filing the scratch, for whichever shape the blocking thread lands on (6348c37 merged onto 27becfa; base 27becfa is the control). Severity of this one finding, not a verdict on the pull request. Any blocking findings from this pass are review threads on the changed files; GitHub renders those collapsed, as a file name with no text.
|
Windows compatibility review —
|
Summary
A provider's cache is a prefix cache: the next turn reuses what the previous one sent only when the bytes match. Two pieces of the turn's user message carried per-turn text that the session write dropped, so the next turn's request differed from the previous one and the cache broke at every turn boundary.
The skill menu rides in the same paragraph as the runtime-context lines, but the assembler recorded the lines alone. The turn scratch (the recalled memory and routed skills for the turn) rides at the end of the user message and was cut and discarded. Both are now filed with the message and rejoined on the way out: the block as
runtime_context, the scratch asturn_scratch, the menu inside the block. What the provider saw is what the next turn sends.The stored
contentkeeps the user's words alone. The transcript, the novelty window and every other reader of the session log see the question, not the recall.Every join is located by what identifies it, never by position. A notice put in front of the message after assembly moves the block off the content's start, and the replay found it by position before, which joined the block twice. The scratch is filed with the offset it stood at, because a hook's note lands on the message after the scratch was appended, so the scratch is not always last.
The scratch's tag said it was "not conversation history". It is now filed and replayed, so the tag says what it is: the recall of the turn it was asked in. The ppt engine's ledger keeps its own copy of the two spellings, pinned against the trunk by a test, and it moves with them.
A review round found one gap in the first form of this: a user entry filed with no
runtime_context(recovery injection leaves the block in the content, so the session write files none) returned early from the replay and lost its filed scratch. The scratch is now rejoined for every user entry; the early return covers only the block.Follows #1294, which filed the runtime-context lines and left these two behind.
Type
Verification
Commands, run with the branch's own venv (Python 3.12.13, all extras):
tests/test_context_replay_bytes.pyandtests/test_context_replay_join.py. The first one drives two turns and compares the bytes of the first turn's question as it was sent against the copy the second turn carries.tests/test_context_turn_scratch.py,tests/test_context_runtime_context.py,tests/test_token_estimation.py,tests/test_ppt_engine_ledger.py,tests/test_context_compacting_runtime.py,tests/test_agent_loop_reply_room.py,tests/test_curator_context_engine.py,tests/test_runtime_checkpoint.py: 381 passed on the rebased head.TurnPathMixin._inject_recovery_block, the real_save_turnand the real projection.scripts/coverage_gate.py diff --base-ref origin/mainover a targeted run: 90.67% of changed lines (68/75), over the 90% threshold.pytest -q(repo defaults): 40580 passed, 267 skipped, 13 failed. Twelve are the environment failures that also fail on main, measured there the same day: the five proxy tests intest_config_update_providers.py(the box exports an HTTP proxy), the four intest_evolver_git_ops.py,test_subagent_node_runtime.py::test_what_cannot_be_read_names_nothing(run as root),test_knowledge_parser_excel.py::test_a_legacy_workbook_is_converted_and_read, andtest_install_script.py::test_resolve_node_dir_answers_each_case_it_exists_for. The thirteenth,test_cli_agent_commands.py::test_an_empty_message_file_is_a_usage_error, passes alone (47 passed) and under four workers twice; it does not touch the code this change edits. 0 introduced.ruff checkandruff format --checkon the changed files,ty checkon the changed modules,lint-imports: clean.Risk
Security impact considered
Backward compatibility considered
Rollback path is clear for risky changes
The scratch's tag text changes, so every live session sends different bytes and every open cache misses once. No request changes shape.
The session log grows by the filed scratch, roughly the size of one recall per turn. The recall was previously dropped; it is now kept, which is what makes the replay exact.
The model re-reads the recalls of the earlier turns, each with its own recall text, since each turn's message replays as it was sent.
A message filed before this change has no
turn_scratchfield and replays exactly as it did.Rollback: revert the commits; no data or config migration.
Related Issues
Follows #1294.