Skip to content

Commit 90699c1

Browse files
authored
fix(*): keep the context window breakdown across a resume (#1340)
## Summary The context card's breakdown (the system prompt, tools, session, files, memory and skills shares of the window) was measured on every model call and sent on the live `message.complete`, but never stored. A page that resumed a conversation therefore drew the fill and said the breakdown would appear after the next turn. A gateway restart forces exactly that reload, so the analysis disappeared until another turn finished. The measured parts now ride with the window reading on the reply that produced it: `_context_parts` is stamped on the same row as `_context_used`, renamed to `context_parts` on save, and read back by `session.resume` from the newest reading-carrying row. `SessionUsage` and `SessionUsageResult` declare the field as optional, and both generated clients were regenerated. The web client needed no change: it already passed `usage.context_parts` through to the card. Key decisions: - The breakdown is only taken from the row whose reading was accepted; an older breakdown is never paired with a newer fill. - Stored counts are sanitized on read: only the six known parts, only non-negative integers a context window can hold. The session file is editable, and a bad value must not reach the card. - The field is optional end to end. Conversations written before this change have no breakdown to return; they keep the existing hint until one new turn records one. There is no honest way to reconstruct it from the stored log, which is not the prompt the model was sent. - A failed or cancelled turn files its newest breakdown on the closing marker, next to the reading it already carried. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed ``` uv run --frozen --all-extras pytest -q tests/test_rpc_session.py tests/test_agent_loop_usage_sink.py -> 281 passed uv run --frozen --all-extras pytest -q tests/test_agent_loop_*.py tests/test_session_*.py -> 1812 passed uv run --frozen --all-extras pytest -q tests/test_rpc_schema_match.py tests/test_rpc_registration.py tests/test_rpc_session.py -> 789 passed uv run --frozen --all-extras pytest -q tests/test_agent_loop_empty_recovery.py tests/test_agent_loop_stream.py tests/test_agent_loop_usage_sink.py tests/test_rpc_session.py -> 811 passed ruff check, ruff format --check and ty check on the changed Python modules -> pass npm run gen:check --prefix ui-web and npm run lint:rpc --prefix ui-tui -> both generated clients match the contract npm run type-check in ui-web and ui-tui -> pass npm run lint in ui-web and ui-tui -> 0 errors (pre-existing warnings only) scripts/check_source_language.py and scripts/check_large_files.py over the change -> pass npm test -- scripts/gates/fixture-shape.test.mjs (ui-web) -> 5 passed ``` No user-facing docs or screenshots change: the card's labels and its empty-state sentence are untouched. The full ui-web suite was also run under concurrent load; 31-40 tests in unrelated files timed out, none referencing this field, and the same files pass 62/62 when rerun in isolation. ## Risk - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes Stored breakdowns come from an editable session file, so the resume reader keeps only the six known parts with non-negative integer values that fit in a window. Old transcripts have no breakdown and show the existing hint until a new turn records one. Reverting the change leaves the extra JSONL key inert to the prior reader and the optional wire field unused. ## Related Issues N/A
1 parent 74517c4 commit 90699c1

11 files changed

Lines changed: 334 additions & 28 deletions

File tree

‎CONTEXT.md‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1796,7 +1796,10 @@ forwarding through `tool_call`, whose message is named for the tool that
17961796
produced it. A `SKILL.md` read through `read_file` or `exec` is not counted:
17971797
the split follows what the skill subsystem wrote, not what a general-purpose
17981798
tool happened to read. Carried on the wire as `context_parts` on a turn's
1799-
usage, and absent until a turn has measured one.
1799+
usage, and absent until a turn has measured one. A turn files its breakdown on
1800+
the reply that carries its window reading, and a resumed session reads the two
1801+
off that same reply -- an older breakdown describes a prompt the newer reading
1802+
did not measure.
18001803
_Avoid_: "segments" for these — a Segment is a piece the context engine assembles
18011804
the prompt from, not a share of the window.
18021805

‎raven/agent/loop/_shared.py‎

Lines changed: 16 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -175,6 +175,12 @@ def _stamp_reasoning_ms(messages: list[dict[str, Any]], response: Any) -> None:
175175
# log whole, so the log is not that.
176176
_CONTEXT_USED_KEY = "_context_used"
177177

178+
# The window divided, kept beside the reading it belongs to and on the same
179+
# terms: measured on the request that reading came from, so a resume can draw
180+
# the card's parts without re-measuring a stored log that is not what the model
181+
# was sent.
182+
_CONTEXT_PARTS_KEY = "_context_parts"
183+
178184

179185
def _window_reading(usage: dict[str, Any] | None) -> int:
180186
"""How full a call left the window: its prompt plus its reply, in tokens.
@@ -214,15 +220,22 @@ def _call_window(response: Any) -> int:
214220
return 0 if _reads_no_window(response) else _window_reading(getattr(response, "usage", None))
215221

216222

217-
def _stamp_context_used(messages: list[dict[str, Any]], response: Any) -> None:
218-
"""Put a model call's window reading on the assistant entry that stands for it.
223+
def _stamp_context_used(
224+
messages: list[dict[str, Any]],
225+
response: Any,
226+
parts: dict[str, int] | None = None,
227+
) -> None:
228+
"""Put a model call's window reading, and its breakdown, on the assistant entry.
219229
220230
Absent when the provider reported no usage, and absent means unknown: a
221-
resume reads the newest entry that has one.
231+
resume reads the newest entry that has one. ``parts`` rides only with a
232+
reading -- a breakdown of a prompt no reading describes is not filed.
222233
"""
223234
used = _call_window(response)
224235
if used > 0 and messages:
225236
messages[-1][_CONTEXT_USED_KEY] = used
237+
if parts:
238+
messages[-1][_CONTEXT_PARTS_KEY] = dict(parts)
226239

227240

228241
def _first_line(text: str) -> str:

‎raven/agent/loop/turn_path.py‎

Lines changed: 45 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@
1111
from raven.agent.context.silence import is_silence, reads_loose_silence, teaches_silence
1212
from raven.agent.loop._shared import (
1313
_ABORTED_ACTION_REPLY,
14+
_CONTEXT_PARTS_KEY,
1415
_CONTEXT_USED_KEY,
1516
_DELEGATED_KEY,
1617
_HOOK_INJECTED_KEY,
@@ -372,7 +373,11 @@ def _stamp_turn_observers(messages: list[dict[str, Any]], metadata: dict[str, An
372373
_TURN_TRANSIENT_KEYS = ("_recovery_synthetic", ATTACHED_IMAGE_KEY)
373374

374375

375-
def _stamp_context_used_on_kept(messages: list[dict[str, Any]], response: Any) -> None:
376+
def _stamp_context_used_on_kept(
377+
messages: list[dict[str, Any]],
378+
response: Any,
379+
parts: dict[str, int] | None = None,
380+
) -> None:
376381
"""Put the reading of a call whose reply is not filed on the newest row that is.
377382
378383
An empty reply never becomes a row of its own in the session: recovery
@@ -383,14 +388,16 @@ def _stamp_context_used_on_kept(messages: list[dict[str, Any]], response: Any) -
383388
is the turn's question it is already on disk and the stamp never reaches
384389
it: the turn either files a row of its own later, which takes a newer
385390
reading, or ends on the marker ``_save_broken_turn`` files, which carries
386-
it.
391+
it. ``parts`` rides with the reading, for the reason ``_shared`` states.
387392
"""
388393
used = _call_window(response)
389394
if used <= 0:
390395
return
391396
for message in reversed(messages):
392397
if not any(message.get(key) for key in _TURN_TRANSIENT_KEYS):
393398
message[_CONTEXT_USED_KEY] = used
399+
if parts:
400+
message[_CONTEXT_PARTS_KEY] = dict(parts)
394401
return
395402

396403

@@ -1006,7 +1013,7 @@ async def _run_agent_loop( # noqa: C901 (cc 100: pre-existing, above the ceilin
10061013
turn_log: TurnLog | None = None,
10071014
rungs_used: set[str] | None = None,
10081015
reply_awaited: bool = False,
1009-
window_reading: dict[str, int] | None = None,
1016+
window_reading: dict[str, Any] | None = None,
10101017
silence_taught: bool = False,
10111018
turn_start_idx: int = 0,
10121019
session_effort: str | None = None,
@@ -1418,9 +1425,14 @@ async def _end_on_the_partial(last: LLMResponse) -> str:
14181425
messages = self.context.add_assistant_message(messages, final)
14191426
if silent:
14201427
messages[-1][_SILENT_KEY] = True
1421-
_stamp_context_used(messages, last)
1428+
_stamp_context_used(messages, last, measured_parts)
14221429
return final
14231430

1431+
# The breakdown measured for the response in hand, carried to every
1432+
# stamp of it. Declared before the loop because the partial-reply path
1433+
# closes over it: a continuation that raises ends the turn on the cut
1434+
# answer from the iteration before, whose measurement this still holds.
1435+
measured_parts: dict[str, int] | None = None
14241436
while iteration < iteration_cap:
14251437
# Checked here and not mid-generation: cancelling a call in flight
14261438
# discards a finished generation and leaves no answer, and a deadline
@@ -1771,10 +1783,18 @@ async def _end_on_the_partial(last: LLMResponse) -> str:
17711783
# the prompt was billed as (see ``_window_reading``), and a
17721784
# call can give none (see ``_call_window``).
17731785
context_used = _call_window(response)
1786+
# The window divided, measured on the request this reading came
1787+
# from: the stored reply carries it so a resume can draw the
1788+
# card's parts without a live report.
1789+
measured_parts = (
1790+
measure_parts(call_messages, call_tools, self._last_window_parts) if context_used > 0 else None
1791+
)
17741792
if context_used > 0:
17751793
window.last_context_used = context_used
17761794
if window_reading is not None:
17771795
window_reading["context_used"] = context_used
1796+
if measured_parts is not None:
1797+
window_reading["context_parts"] = measured_parts
17781798
if usage_sink is not None and response.usage and context_used <= 0 and usage_sink.get("context_used"):
17791799
# No reading -- a block of zeros, or an account ``_call_window``
17801800
# does not take as one -- so the gauge stays on the call before
@@ -1818,7 +1838,11 @@ async def _end_on_the_partial(last: LLMResponse) -> str:
18181838
# The window divided, so a reader can be told what is filling it
18191839
# and not only how full it is. Measured on the request that was
18201840
# sent, which is the one the count above belongs to.
1821-
usage_sink["context_parts"] = measure_parts(call_messages, call_tools, self._last_window_parts)
1841+
usage_sink["context_parts"] = (
1842+
measured_parts
1843+
if measured_parts is not None
1844+
else measure_parts(call_messages, call_tools, self._last_window_parts)
1845+
)
18221846

18231847
# Recoveries that retry this iteration. The classifier on the response
18241848
# says what kind of refusal this was; the Memory role says whether it
@@ -1879,7 +1903,7 @@ async def _end_on_the_partial(last: LLMResponse) -> str:
18791903
# leaves dangling calls that strict providers reject.
18801904
final_content = str(decision.short_circuit_result)
18811905
messages = self.context.add_assistant_message(messages, final_content)
1882-
_stamp_context_used(messages, response)
1906+
_stamp_context_used(messages, response, measured_parts)
18831907
break
18841908
# Blocking a call is handled where the call is: the
18851909
# branch below refuses it and cancels the siblings the
@@ -1920,7 +1944,7 @@ async def _end_on_the_partial(last: LLMResponse) -> str:
19201944
thinking_blocks=response.thinking_blocks,
19211945
)
19221946
_stamp_reasoning_ms(messages, response)
1923-
_stamp_context_used(messages, response)
1947+
_stamp_context_used(messages, response, measured_parts)
19241948

19251949
for tool_call_index, tool_call in enumerate(response.tool_calls):
19261950
tools_used.append(tool_call.name)
@@ -2403,7 +2427,7 @@ async def _end_on_the_partial(last: LLMResponse) -> str:
24032427
status = "error"
24042428
error_detail = _llm_failure_detail(clean, verdict)
24052429
error_full = response_failure_text(clean, response.call_record)
2406-
_stamp_context_used_on_kept(messages, response)
2430+
_stamp_context_used_on_kept(messages, response, measured_parts)
24072431
break
24082432

24092433
if recovery.parts:
@@ -2420,7 +2444,7 @@ async def _end_on_the_partial(last: LLMResponse) -> str:
24202444
carried_gate = None
24212445

24222446
if not (clean or "").strip():
2423-
_stamp_context_used_on_kept(messages, response)
2447+
_stamp_context_used_on_kept(messages, response, measured_parts)
24242448

24252449
# Empty-response recovery. A reply with no tool call is read for
24262450
# what it is (``reply_kind``) and ``decide`` names the move; an
@@ -2750,7 +2774,7 @@ async def _end_on_the_partial(last: LLMResponse) -> str:
27502774
if decision.short_circuit_result is not None:
27512775
final_content = str(decision.short_circuit_result)
27522776
messages = self.context.add_assistant_message(messages, final_content)
2753-
_stamp_context_used(messages, response)
2777+
_stamp_context_used(messages, response, measured_parts)
27542778
break
27552779
else:
27562780
await _settle_silence(silence_gate, silent=ends_silent, dropped_marker=dropped_marker)
@@ -2774,7 +2798,7 @@ async def _end_on_the_partial(last: LLMResponse) -> str:
27742798
thinking_blocks=response.thinking_blocks,
27752799
)
27762800
_stamp_reasoning_ms(messages, response)
2777-
_stamp_context_used(messages, response)
2801+
_stamp_context_used(messages, response, measured_parts)
27782802
# Kept in the history, as the row it is, and filed as a marker so a
27792803
# reload does not read it as an answer.
27802804
silent = ends_silent
@@ -3421,7 +3445,7 @@ async def _tap_episode(_index: int) -> None:
34213445
# A turn that fails or is stopped after calls that each came back empty
34223446
# or as an error has filed no row of its own, and the question it would
34233447
# otherwise end on is already on disk, so its marker carries the reading.
3424-
reading: dict[str, int] = {}
3448+
reading: dict[str, Any] = {}
34253449

34263450
async def _attempt(seed: list[dict], attempt: int):
34273451
pending.update(rerun=False, reasons=[])
@@ -3625,6 +3649,7 @@ def _rerun_pending(final_content: str | None, msgs: list[dict], status: str) ->
36253649
streamed,
36263650
status="cancelled",
36273651
context_used=reading.get("context_used"),
3652+
context_parts=reading.get("context_parts"),
36283653
)
36293654
self._park_task_board(key, live["log"].added(), turn_received_at, req.turn_id)
36303655
raise
@@ -3642,6 +3667,7 @@ def _rerun_pending(final_content: str | None, msgs: list[dict], status: str) ->
36423667
reason=str(exc),
36433668
full_detail=attach_full_detail(exc),
36443669
context_used=reading.get("context_used"),
3670+
context_parts=reading.get("context_parts"),
36453671
)
36463672
self._park_task_board(key, live["log"].added(), turn_received_at, req.turn_id)
36473673
raise
@@ -3659,6 +3685,7 @@ def _rerun_pending(final_content: str | None, msgs: list[dict], status: str) ->
36593685
reason=describe_failure(exc),
36603686
full_detail=attach_full_detail(exc),
36613687
context_used=reading.get("context_used"),
3688+
context_parts=reading.get("context_parts"),
36623689
)
36633690
self._park_task_board(key, live["log"].added(), turn_received_at, req.turn_id)
36643691
raise
@@ -3960,6 +3987,7 @@ def _save_broken_turn(
39603987
full_detail: str | None = None,
39613988
inbound_original: str | None = None,
39623989
context_used: int | None = None,
3990+
context_parts: dict[str, int] | None = None,
39633991
) -> None:
39643992
"""Persist what a cancelled or failed turn got as far as producing.
39653993
@@ -3983,6 +4011,8 @@ def _save_broken_turn(
39834011
``context_used`` is the newest window the turn measured, filed on the
39844012
marker because it is the newest row: a resume reads the window from
39854013
there, and a turn whose calls left no row of their own has no other.
4014+
``context_parts`` rides beside it, measured on the same request, so
4015+
the card's breakdown survives the same resume.
39864016
39874017
``received_at`` and ``inbound_original`` describe the turn's question,
39884018
which ``_process_message`` files before the attempt starts and hands
@@ -4049,6 +4079,8 @@ def _save_broken_turn(
40494079
}
40504080
if context_used:
40514081
marker[_CONTEXT_USED_KEY] = context_used
4082+
if context_parts:
4083+
marker[_CONTEXT_PARTS_KEY] = dict(context_parts)
40524084
tail.append(marker)
40534085
self._save_turn(session, tail, 0, received_at=received_at, inbound_original=inbound_original)
40544086
self.sessions.save(session)
@@ -4326,6 +4358,7 @@ def _save_turn(
43264358
(_REASONING_MS_KEY, "reasoning_ms"),
43274359
(_TOOL_DURATION_MS_KEY, "duration_ms"),
43284360
(_CONTEXT_USED_KEY, "context_used"),
4361+
(_CONTEXT_PARTS_KEY, "context_parts"),
43294362
):
43304363
# Renamed for the same reason as the diff above. A duration or a
43314364
# window reading is a property of the part it measures, so it is

‎raven/rpc/methods/session.py‎

Lines changed: 35 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -759,6 +759,11 @@ def _fill_resumed_context(usage: dict[str, Any], session: Any) -> None:
759759
times the window its last call measured, and a short one at a few tokens,
760760
the system prompt and the tool list being no part of it.
761761
762+
``usage.context_parts`` is taken off that same row and left absent when the
763+
row carries none: an older breakdown describes a prompt the newer reading
764+
did not measure, so pairing the two would draw a composition the fill never
765+
had.
766+
762767
A session with no reading -- written before rows carried one, or whose
763768
calls never reported their usage -- falls back to a tiktoken estimate of
764769
``get_history(max_messages=0)``, marked ``context_estimated``. Failures
@@ -768,10 +773,13 @@ def _fill_resumed_context(usage: dict[str, Any], session: Any) -> None:
768773
if session is None:
769774
return
770775
context_max = usage.get("context_max") or 0
771-
if measured := _last_window_reading(session.messages):
776+
measured, parts = _last_window_reading(session.messages)
777+
if measured:
772778
usage["context_used"] = measured
773779
usage["context_percent"] = round(100 * measured / context_max) if context_max else 0
774780
usage["context_estimated"] = False
781+
if parts is not None:
782+
usage["context_parts"] = parts
775783
return
776784
try:
777785
# The estimate counts what an engine that leaves compaction to the
@@ -802,14 +810,36 @@ def _fill_resumed_context(usage: dict[str, Any], session: Any) -> None:
802810
# percentage. Either is read as no reading.
803811
_READING_CEILING = 1_000_000_000
804812

813+
# The parts a breakdown may carry, in the order the card draws them. A stored
814+
# key outside this set is not part of the contract and is not passed on.
815+
_PART_KEYS = ("system", "tools", "conversation", "files", "memory", "skills")
816+
805817

806-
def _last_window_reading(messages: list[Any]) -> int:
807-
"""The window the newest reading-carrying row records, or 0 for none."""
818+
def _last_window_reading(messages: list[Any]) -> tuple[int, dict[str, int] | None]:
819+
"""The newest reading-carrying row's window and breakdown, or ``(0, None)``."""
808820
for message in reversed(messages):
809821
used = message.get("context_used") if isinstance(message, dict) else None
810822
if isinstance(used, int) and not isinstance(used, bool) and 0 < used <= _READING_CEILING:
811-
return used
812-
return 0
823+
return used, _stored_parts(message.get("context_parts"))
824+
return 0, None
825+
826+
827+
def _stored_parts(value: Any) -> dict[str, int] | None:
828+
"""The breakdown's token counts, or None when none is one.
829+
830+
The record is drawn by a page and the session file is editable: a string, a
831+
bool (an ``int`` to Python), a negative count or one past any window is
832+
dropped rather than reaching the card. A partly valid record keeps its
833+
valid counts.
834+
"""
835+
if not isinstance(value, dict):
836+
return None
837+
parts = {
838+
key: count
839+
for key in _PART_KEYS
840+
if isinstance(count := value.get(key), int) and not isinstance(count, bool) and 0 <= count <= _READING_CEILING
841+
}
842+
return parts or None
813843

814844

815845
def _session_to_list_item(info: dict[str, Any]) -> dict[str, Any]:

‎raven/rpc/models.py‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2223,6 +2223,14 @@ class SessionUsageResult(_Strict):
22232223
)
22242224
context_percent: int = Field(..., description="context_used as a percentage of context_max.")
22252225
context_estimated: bool = Field(..., description="True when context_used is a tiktoken estimate.")
2226+
context_parts: dict[str, int] | None = Field(
2227+
default=None,
2228+
description=(
2229+
"The window divided: tokens by part, keyed system / tools / conversation / "
2230+
"files / memory / skills, as the last measured turn recorded them. Absent when "
2231+
"that turn measured none, or the conversation predates the record."
2232+
),
2233+
)
22262234

22272235

22282236
class SessionHistoryParams(_Strict):
@@ -4364,6 +4372,14 @@ class SessionUsage(_Strict):
43644372
default=None,
43654373
description="True when context_used is a tiktoken estimate of a resumed transcript, not a measurement.",
43664374
)
4375+
context_parts: dict[str, int] | None = Field(
4376+
default=None,
4377+
description=(
4378+
"The window divided: tokens by part, keyed system / tools / conversation / "
4379+
"files / memory / skills, as the last measured turn recorded them. Absent when "
4380+
"that turn measured none, or the conversation predates the record."
4381+
),
4382+
)
43674383

43684384

43694385
class SessionInitInfo(_Strict):

‎rpc-schema/openrpc.json‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8576,6 +8576,16 @@
85768576
"context_estimated": {
85778577
"type": "boolean",
85788578
"description": "True when context_used is a tiktoken estimate."
8579+
},
8580+
"context_parts": {
8581+
"type": [
8582+
"object",
8583+
"null"
8584+
],
8585+
"additionalProperties": {
8586+
"type": "integer"
8587+
},
8588+
"description": "The window divided: tokens by part, keyed system / tools / conversation / files / memory / skills, as the last measured turn recorded them. Absent when that turn measured none, or the conversation predates the record."
85798589
}
85808590
}
85818591
}
@@ -13248,6 +13258,16 @@
1324813258
"context_estimated": {
1324913259
"type": "boolean",
1325013260
"description": "True when context_used is a tiktoken estimate of a resumed transcript, not a measurement."
13261+
},
13262+
"context_parts": {
13263+
"type": [
13264+
"object",
13265+
"null"
13266+
],
13267+
"additionalProperties": {
13268+
"type": "integer"
13269+
},
13270+
"description": "The window divided: tokens by part, keyed system / tools / conversation / files / memory / skills, as the last measured turn recorded them. Absent when that turn measured none, or the conversation predates the record."
1325113271
}
1325213272
},
1325313273
"required": [

0 commit comments

Comments
 (0)