Skip to content

fix: skip Claude's <synthetic> placeholder when reading transcript models - #1056

Draft
MetalZealot wants to merge 2 commits into
siteboon:mainfrom
MetalZealot:fix/synthetic-model-guard
Draft

fix: skip Claude's <synthetic> placeholder when reading transcript models#1056
MetalZealot wants to merge 2 commits into
siteboon:mainfrom
MetalZealot:fix/synthetic-model-guard

Conversation

@MetalZealot

@MetalZealot MetalZealot commented Jul 23, 2026

Copy link
Copy Markdown

Problem

After an API error (e.g. 529 Overloaded), Claude Code appends locally-fabricated assistant rows to the session transcript stamped with the placeholder model "<synthetic>" — the API-error notice, "No response requested.", and session-limit messages. extractClaudeEventModel accepts whatever model the last assistant row carries, so the backward transcript scan in getCurrentActiveModel adopts the placeholder and reports <synthetic> as the session's active model until a genuine turn lands.

Repro

  1. Have a Claude session hit an API error (529 Overloaded, session limit, etc.) — the CLI writes assistant rows with "model": "<synthetic>" to the session JSONL.
  2. Switch to / reopen that session → the active model resolves to <synthetic> instead of the model the session was actually running.

Fix

extractClaudeEventModel now rejects angle-bracketed placeholder values for both event.model and message.model, so the scan keeps walking back to the last genuine turn and recovers the session's real model. A transcript containing only synthetic rows falls back to the catalog default, same as before when no model was found.

Tests

New server/modules/providers/tests/claude-models.test.ts (node:test, same isolated-database harness as opencode-sessions.test.ts):

  • control: the last assistant turn's model is read from the transcript
  • synthetic error rows are skipped and the real model is recovered
  • an all-synthetic transcript falls back to the catalog default

Run with npx tsx --tsconfig server/tsconfig.json --test server/modules/providers/tests/claude-models.test.ts — 3/3 pass with the fix; the two guard tests fail on current main. tsc --noEmit -p server/tsconfig.json and eslint are clean.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Bug Fixes
    • Improved Claude model detection by ignoring synthetic placeholder model values.
    • Active model selection now correctly retains the last genuine model after errors or synthetic transcript entries.
    • Added/verified fallback behavior when no valid model is available.
  • Tests
    • Added coverage to validate active model extraction across transcripts containing synthetic placeholder entries.

…cript models

After an API error (e.g. 529 Overloaded), Claude Code appends assistant
rows stamped model "<synthetic>" (the error notice, "No response
requested.", session-limit messages). The transcript scanner adopted
that placeholder as the session's model, and since transcript-sourced
models feed the send path, the next turn was sent with model
"<synthetic>" and failed with "There's an issue with the selected
model (<synthetic>)" until the user switched surfaces.

extractClaudeEventModel now rejects angle-bracketed placeholder values
for both event.model and message.model, so the backward scan continues
to the last genuine turn and recovers the session's real model.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 454abaee-1aba-4daa-beb7-69a4b1b3eed7

📥 Commits

Reviewing files that changed from the base of the PR and between ce7b3bc and 351f987.

📒 Files selected for processing (1)
  • server/modules/providers/tests/claude-models.test.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • server/modules/providers/tests/claude-models.test.ts

📝 Walkthrough

Walkthrough

Claude model extraction now ignores synthetic <...> placeholders from session events. New isolated-database tests verify selection of the latest real model, skipping synthetic rows, and fallback behavior when no real model exists.

Changes

Claude model filtering

Layer / File(s) Summary
Placeholder model extraction
server/modules/providers/list/claude/claude-models.provider.ts
Recognizes <...> placeholder values and excludes them from direct and nested Claude event model fields.
Model selection validation
server/modules/providers/tests/claude-models.test.ts
Sets up isolated transcript data and tests real-model selection, skipping synthetic rows, and returning the default model when all rows are synthetic.

Possibly related PRs

Poem

I nibbled placeholders away,
So real Claude models could stay.
Through transcript rows I hop and roam,
And find the last true model home.
If none appear, the default takes flight.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the main change: ignoring Claude synthetic placeholder models when reading transcript models.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

MetalZealot added a commit to MetalZealot/CLIde that referenced this pull request Jul 23, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
server/modules/providers/tests/claude-models.test.ts (1)

90-105: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Cover placeholders in event.model too.

The test currently validates only message.model; add model: '<synthetic>' to one synthetic row so both changed extraction paths are protected.

Proposed test update
         {
           type: 'assistant',
           sessionId: PROVIDER_SESSION_ID,
+          model: '<synthetic>',
           message: {
             model: '<synthetic>',
             content: [{ type: 'text', text: 'API Error: 529 Overloaded.' }],
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@server/modules/providers/tests/claude-models.test.ts` around lines 90 - 105,
Update the synthetic assistant event fixtures in the test to include model:
'<synthetic>' on one row, while retaining message.model coverage on another row,
so extraction from both event.model and message.model is validated.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@server/modules/providers/tests/claude-models.test.ts`:
- Around line 90-105: Update the synthetic assistant event fixtures in the test
to include model: '<synthetic>' on one row, while retaining message.model
coverage on another row, so extraction from both event.model and message.model
is validated.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 37ebf5f8-de5e-45c0-a5de-72cd3a8fd748

📥 Commits

Reviewing files that changed from the base of the PR and between 27eaf01 and ce7b3bc.

📒 Files selected for processing (2)
  • server/modules/providers/list/claude/claude-models.provider.ts
  • server/modules/providers/tests/claude-models.test.ts

CodeRabbit review on siteboon#1056: the guard checks both event.model and
message.model, but the fixtures only set message.model. Stamp the
529 row with a top-level placeholder too so both paths are exercised.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
MetalZealot added a commit to MetalZealot/CLIde that referenced this pull request Jul 23, 2026
Mirrors the CodeRabbit-suggested coverage pushed to PR siteboon#1056
(351f987 on fix/synthetic-model-guard): the guard checks both
event.model and message.model, but fixtures only set message.model.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@blackmammoth

Copy link
Copy Markdown
Member

Hello @MetalZealot, thanks for the PR. I tried replicating the issue but it doesn't seem to occur. Can you check and verify again? If so, can you share a more clear reproduction step?

@blackmammoth
blackmammoth marked this pull request as draft July 31, 2026 09:50
@MetalZealot

Copy link
Copy Markdown
Author

Thanks for checking. I re-verified this against current main (c2408f0) and the issue is still present.

The trigger is narrower than “any 529”: it occurs only when Claude Code appends a locally generated assistant row whose message.model is "<synthetic>". If the failed request does not write one of those rows, the bug will not reproduce.

A deterministic reproduction is:

  1. Create/open a Claude session and let one genuine assistant turn land.
  2. In that session's JSONL, append a synthetic assistant row after the real turn (replace <session-id> with the transcript's session id):
{"type":"assistant","sessionId":"<session-id>","message":{"model":"claude-sonnet-5","content":[]}}
{"type":"assistant","sessionId":"<session-id>","message":{"model":"<synthetic>","content":[{"type":"text","text":"You've hit your session limit"}]}}
  1. Switch away and reopen the session, then open /model.

Actual on current main: the active model resolves to <synthetic>.

Expected: the scan skips that placeholder row and keeps walking back to claude-sonnet-5.

The same rows are produced naturally by session-limit notices, some API/auth errors, and "No response requested.". As a real-world check, I found 173 synthetic rows across 75 local Claude transcript files, including recent rows from July 30. One sanitized adjacent pair is:

2026-07-17T14:21:54.526Z  claude-fable-5
2026-07-17T14:21:55.603Z  <synthetic>  You've hit your session limit

I also merged this PR locally into current upstream main and reran:

npx tsx --tsconfig server/tsconfig.json --test server/modules/providers/tests/claude-models.test.ts

All 3 tests pass on the current merge result.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants