ai-gateway: split session_id from conversation_id (schema v6) (#104) by philcunliffe · Pull Request #117 · hyparam/hypaware

philcunliffe · 2026-06-16T18:00:29Z

Problem

conversation_id was overloaded and confused both humans and LLMs analyzing ai_gateway_messages. Codex puts a real thread there; Claude has no conversation id, so we shoved the session id in — but a Claude session holds the main loop plus N subagents, side chats, title/recap calls, etc. So conversation_id was really a container of many threads, and anyone scoping a conversation silently mixed them.

New model

column	Claude main	Claude subagent	Codex
`session_id` (partition key, non-null)	session	session	`metadata.session_id`
`conversation_id` (nullable)	null	null	thread
`agent_id` (nullable)	null	agent	null
`parent_thread_id` (nullable)	null	null	parent thread

Touches

Schema (message_projector.js): add session_id (non-null), make conversation_id nullable, SCHEMA_VERSION 5→6.
Scoping: the prior-message chain + fallback hash now scope on (conversation_id ?? session_id, agent_id) — Claude scopes by session (conversation_id null), Codex by its thread. Behaviour is unchanged from before the split, since Claude's old conversation_id was the session id.
Claude (projector + backfill): set session_id (with a non-null fallback so the partition key is never null), conversation_id = null.
Codex (exchange-projector + backfill): session_id = metadata.session_id (fallback to thread), keep conversation_id = thread and parent_thread_id.
settle.js: group/load transcripts by session_id (Claude conversation_id is now null).
dataset.js: partition by session_id (required identity field); conversation_id rides along as a non-required identity field; partition label bumped; legacy partitions still discovered.
Context-graph (graph contract + enrich config + tests): Session nodes key on session_id.

BREAKING

Moves the Iceberg partition key, so it needs a cache recreate + backfill. Must not be bundled with additive schema changes (issue #102 — those evolve in place; this can't).

LLP

New LLP 0030 documents the model and the breaking boundary; LLP 0026 updated (conversation_id is no longer "the session id for Claude"); LLP 0022 updated for the new partition key. All @refs verified to resolve.

Tests

Migrated the affected suites (ai-gateway dataset/materializer/projector, claude/codex backfill, settlement, graph contract, context-graph project/propose) to the new model. npm test: 1148 pass / 0 fail / 1 skip.

Fixes #104

🤖 Generated with Claude Code

conversation_id was overloaded: Codex put a real thread there, while Claude (no per-thread id) shoved the session id in — but a Claude session holds the main loop, subagents, and side chats, so conversation_id was really a container of many threads, confusing anyone analyzing the data. Introduce session_id as the always-present partition key and make conversation_id the (nullable) thread within it: - session_id (partition key, non-null): Claude session / Codex metadata.session_id (fallback to a hash / the thread so it's never null) - conversation_id (nullable): Codex thread; null for Claude - agent_id (nullable): Claude subagent; unchanged - parent_thread_id (nullable): Codex parent thread; unchanged Touches: schema columns + SCHEMA_VERSION 5->6; message_projector scopes the prior-message chain + fallback hash on (conversation_id ?? session_id, agent_id); claude/codex projectors and backfills set the columns per the model; settle.js groups/loads transcripts by session_id; dataset partitions by session_id (conversation_id rides along as a non-required identity field). Context-graph consumers (graph contract, enrich config) key Session nodes on session_id. Tests migrated to the new model. BREAKING: moves the Iceberg partition key, so it needs a cache recreate + backfill. Must NOT be bundled with additive schema changes (issue #102). Documents the model in new LLP 0030; updates LLP 0026 (conversation_id is no longer 'the session id for Claude') and LLP 0022 (partition key). Fixes #104 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

philcunliffe · 2026-06-16T18:35:35Z

Dual-agent review — `request_changes`

Verdict: request_changes
Risk class: medium
Auto-merge advisory: 👎 thumbs down — verdict is request_changes; needs human-gated follow-up

Advisory only: no merge was attempted.

Risk capstone

Cross-reference: reviewer findings vs high-risk surfaces

Source	Finding (severity, evidence)	Intersects
Claude	major — smoke flows query `conversation_id` as session key (backfill_claude_fixture.js:164, walkthrough_backfill_client_history.js:188,209, context_graph_projects_rows.js:209)	Direct callers (smoke-flow consumers of `ai_gateway_messages`), Risks
Codex	major — `conversationStartedAt`/`toolCallLookupByConversation` keyed on `session_id` cross Codex threads (message_projector.js:248,256; exchange-projector.js:70)	Concurrency surface, Risks
Codex	minor — packaged agent/skill guidance still uses `conversation_id` (hypaware-analyst.md, hypaware-graph SKILL.md)	Risks

Codex review

Fix Validations

Overloaded `conversation_id` in row schema/provider projection

Status: correct
Evidence: hypaware-core/plugins-workspace/ai-gateway/src/message_projector.js:26, hypaware-core/plugins-workspace/claude/src/projector.js:240, hypaware-core/plugins-workspace/codex/src/exchange-projector.js:70
Assessment: The core projection shape now has required session_id, nullable/omitted Claude conversation_id, and Codex emits both session and thread identities.

Fallback identity and previous-message chain scope

Status: correct
Evidence: hypaware-core/plugins-workspace/ai-gateway/src/message_projector.js:279
Assessment: The fallback chain uses conversation_id ?? session_id, preserving Claude’s old session scope and Codex’s thread scope.

Findings

1) Behavioral Correctness

Severity: major
Confidence: high
Evidence: hypaware-core/plugins-workspace/codex/src/exchange-projector.js:70, hypaware-core/plugins-workspace/codex/src/exchange-projector.js:89, hypaware-core/plugins-workspace/ai-gateway/src/message_projector.js:248, hypaware-core/plugins-workspace/ai-gateway/src/message_projector.js:256, hypaware-core/plugins-workspace/ai-gateway/src/message_projector.js:494
Why it matters: Codex can now emit one session_id with distinct thread conversation_ids, but conversation_started_at and toolCallLookupByConversation are memoized by session_id, so separate Codex threads in one session can inherit the first thread’s start time and cross-resolve tool-result names if tool call ids collide.
Suggested fix: Key those two maps with the same threadScope = conversationId ?? sessionId used for fallback identity/chain, and add a regression with two Codex projections sharing session_id but using different conversation_ids, timestamps, and overlapping tool call ids.

11) Debuggability & Operability

Severity: minor
Confidence: high
Evidence: hypaware-core/plugins-workspace/claude/agents/hypaware-analyst.md:39, hypaware-core/plugins-workspace/claude/agents/hypaware-analyst.md:40, hypaware-core/plugins-workspace/context-graph/skills/hypaware-graph/SKILL.md:26, hypaware-core/plugins-workspace/context-graph/skills/hypaware-graph/SKILL.md:28, hypaware-core/plugins-workspace/claude/src/projector.js:240
Why it matters: Packaged agent/skill instructions still tell automated analysis to group/order graph sessions by conversation_id, which is now null/omitted for Claude and no longer the graph Session key.
Suggested fix: Update bundled query/analyst/graph skill guidance to use session_id as the session/grouping/natural key, with conversation_id described only as nullable thread identity.

No Finding

Contract & Interface Fidelity
Change Impact / Blast Radius
Concurrency, Ordering & State Safety
Error Handling & Resilience
Security Surface
Resource Lifecycle & Cleanup
Release Safety
Test Evidence Quality
Architectural Consistency

Evidence Bundle

Changed hot paths: ai-gateway row projection; Codex exchange projection; Claude live/backfill projection; Claude settlement; ai-gateway dataset partitioning; graph contract/session anchoring.
Impacted callers: createAiGatewayMessageProjector persistent live state at hypaware-core/plugins-workspace/ai-gateway/src/message_projector.js:151; Codex projector at hypaware-core/plugins-workspace/codex/src/exchange-projector.js:89; Claude projector at hypaware-core/plugins-workspace/claude/src/projector.js:240.
Impacted tests: test/plugins/ai-gateway-message-projector.test.js:406; test/plugins/claude-settlement.test.js:31; test/plugins/context-graph-enrich-propose.test.js:30; test/plugins/context-graph-project-e2e.test.js:25.
Unresolved uncertainty: I did not run npm test; review is based on the provided diff plus targeted caller/contract traces.

Claude review

Claude review

Hermetic smoke flows still query/seed `conversation_id` as the Claude session key — three will fail

Severity: major
Confidence: 88
Evidence: hypaware-core/smoke/flows/backfill_claude_fixture.js:164, hypaware-core/smoke/flows/walkthrough_backfill_client_history.js:188, hypaware-core/smoke/flows/walkthrough_backfill_client_history.js:209-211, hypaware-core/smoke/flows/context_graph_projects_rows.js:209
Why it matters: The PR moves the Claude session id out of conversation_id (now null for Claude) into the new session_id column, and repoints the context-graph Session anchor to session_id. The traditional test/** suite was migrated (and passes 1148/0), but the hermetic smoke tier — a distinct release gate per CLAUDE.md "Smoke Test Model", not run by npm test — was not. Concretely: backfill_claude_fixture filters where conversation_id = '${sessionId}', which now matches zero Claude rows, failing its "two rows for the backfilled session" assertion; walkthrough_backfill_client_history filters where conversation_id in ('${claudeSession}', ...) and rows.filter(r => r.conversation_id === claudeSession), so its "claude session produced two rows" assertion fails; and context_graph_projects_rows seeds a fixture row with conversation_id: 'conv-1' and no session_id, so the real ai-gateway-graph Session node/edges (now keyed on session_id) get a null key and the asserted "7 nodes / 6 edges, Session === 1" projection collapses. gateway_claude_capture.js:261 (order by conversation_id) is stale but non-fatal since its row assertions key on message_id/role.
Suggested fix: Migrate the three Claude-consuming smoke flows to the new model — query/filter on session_id for Claude (backfill_claude_fixture, walkthrough_backfill_client_history), and add session_id to the context_graph_projects_rows fixture row (line 209) so the Session anchor resolves. Run npm run smoke -- backfill_claude_fixture, walkthrough_backfill_client_history, and context_graph_projects_rows to confirm green, since npm test does not cover this tier.

Reports: /Users/phil/workspace/hypaware/.git/worktrees/dual-review-pr-117/dual-review/pr-117

…umers Review fixes for the schema v6 session_id split: - Key conversationStartedAt and toolCallLookupByConversation by threadScope (conversation_id ?? session_id), the same scope used for the prior-message chain and fallback identity. A Codex session_id can carry several thread conversation_ids; keying these maps by session_id let a later thread inherit the first thread's start time and cross-resolve tool-result names on colliding tool_call ids. Adds a regression with two Codex threads sharing a session_id. - Migrate the hermetic smoke flows off conversation_id as the Claude session key: query/filter on session_id (null conversation_id for Claude), seed session_id in the context-graph fixture, and update the stale order-by. (Two of these smokes also hit a pre-existing icebird partition-pruning limitation for >16-char string partition values, present on master and unrelated to this change.) - Update bundled analyst/graph/query skill guidance to treat session_id as the session/grouping/natural key and conversation_id as a nullable thread identity. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

# Conflicts: # test/plugins/ai-gateway-message-projector.test.js

Re-scope the restart-replay seen-set seeding (issue #108) onto session_id (the LLP 0030 partition key) — Claude conversation_id is null, so the prior conversation-scoped seed never deduped. Update the re-settle (#105), aux graph-contract / context-graph (#106), and restart-replay (#108) test fixtures to the v6 schema (non-null session_id partition key).

philcunliffe and others added 3 commits June 16, 2026 12:46

Merge remote-tracking branch 'origin/master' into merge-117

53e153d

# Conflicts: # test/plugins/ai-gateway-message-projector.test.js

philcunliffe merged commit 30aa737 into master Jun 16, 2026
6 checks passed

philcunliffe deleted the fix/issue-104-split-session-id branch June 16, 2026 20:32

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ai-gateway: split session_id from conversation_id (schema v6) (#104)#117

ai-gateway: split session_id from conversation_id (schema v6) (#104)#117
philcunliffe merged 4 commits into
masterfrom
fix/issue-104-split-session-id

philcunliffe commented Jun 16, 2026

Uh oh!

philcunliffe commented Jun 16, 2026

Fix Validations

Overloaded `conversation_id` in row schema/provider projection

Fallback identity and previous-message chain scope

Findings

1) Behavioral Correctness

11) Debuggability & Operability

No Finding

Evidence Bundle

Claude review

Hermetic smoke flows still query/seed `conversation_id` as the Claude session key — three will fail

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

philcunliffe commented Jun 16, 2026

Problem

New model

Touches

BREAKING

LLP

Tests

Uh oh!

philcunliffe commented Jun 16, 2026

Dual-agent review — request_changes

Risk capstone

Cross-reference: reviewer findings vs high-risk surfaces

Fix Validations

Overloaded conversation_id in row schema/provider projection

Fallback identity and previous-message chain scope

Findings

1) Behavioral Correctness

11) Debuggability & Operability

No Finding

Evidence Bundle

Claude review

Hermetic smoke flows still query/seed conversation_id as the Claude session key — three will fail

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Dual-agent review — `request_changes`

Overloaded `conversation_id` in row schema/provider projection

Hermetic smoke flows still query/seed `conversation_id` as the Claude session key — three will fail