feat: chat runtime - pause/resume, SSE transport, React bindings#5
feat: chat runtime - pause/resume, SSE transport, React bindings#5marslavish wants to merge 23 commits into
Conversation
marslavish
changed the title
marslavish
changed the title
marslavish
changed the title
| return new Response(sse, responseInit); | ||
| } | ||
|
|
||
| then<TResult1 = void, TResult2 = never>( |
There was a problem hiding this comment.
I might be too paranoid here but this is likely a design flaw
DefaultAgentRunHandle opts into Promise assimilation by extending PromiseLike<void>, which lets a caller write await handle as shorthand for
"run to completion and discard events." That ergonomic sugar is the source of every problem below.
The relevant code
// packages/agent/src/run-handle.ts:10-14
export interface AgentRunHandle extends PromiseLike<void> {
events(): AsyncIterable<AgentEvent>;
toReadableStream(): ReadableStream<AgentEvent>;
toResponse(init?: ResponseInit): Response;
}// packages/agent/src/run-handle.ts:62-70
then(onfulfilled, onrejected) {
if (!this.startedAs) {
this.startSink(); // ← side effect on observation
}
return this.completion!.then(onfulfilled, onrejected);
}// packages/agent/src/run-handle.ts:164-171
private startSink(): void {
this.ensureNotStarted('sink');
this.startedAs = 'sink';
this.completion = this.bind(null, abortController.signal); // push = null → events dropped
}The key insight: any thenable in JavaScript can be silently consumed by the Promise machinery — await, Promise.resolve, returning from an
async function, Promise.all, even some logger middleware. The runtime calls .then() on it without asking. And in this class, .then() is
not an observation — it mutates state (startedAs = 'sink') and dispatches the bind.
Hazard A — Any assimilator silently starts the run as a sink
const handle = agent.prompt('hi');
await handle; // ← starts run, events dropped
Promise.resolve(handle); // ← same
async function factory() { return handle; }
await factory(); // ← sameAll three call handle.then() under the hood. The run begins with push = null, so every event the agent emits goes nowhere.
Hazard B — After assimilation, the documented API throws
const handle = agent.prompt('hi');
someLogger.info({ handle }); // looks harmless
await Promise.resolve(handle); // ← caller didn't realize this consumed it
handle.events(); // throws "already consumed via sink"
handle.toReadableStream(); // throws
handle.toResponse(); // throwsThe ensureNotStarted guard at run-handle.ts:72-81 enforces single-use: once startedAs is set, every other consumption mode is dead. The
handle is now a paperweight that has already burned its run.
Hazard C — TypeScript erases the handle reference
async function makeHandle() {
return agent.prompt('hi'); // returns AgentRunHandle
}
const result = await makeHandle();
// typeof result is `void`, not AgentRunHandle — verified by tsc
result.events(); // compile error: events does not exist on voidBecause AgentRunHandle extends PromiseLike<void>, the await unwraps it to void. The IDE will complain when you try to use the handle, but
only after you've already lost the reference and triggered the bind. The type system reflects the hazard but doesn't prevent it.
Hazard D — The assimilator deadlocks on the full agent run
This is the part that goes beyond "the run starts as a sink." Look at line 69:
return this.completion!.then(onfulfilled, onrejected);this.completion is the promise returned by bind(null, …), which resolves only when the entire agent run finishes — through all LLM
streaming, all tool calls, all turns, until agent_end. Anything that assimilated the handle is now waiting on that.
async function getHandle() { return agent.prompt('hi'); }
const handle = await getHandle();
// ← blocks for 30+ seconds while the LLM streams, AND no events
// are observable because the run is already in sink mode.
// In a Next.js route, the response is held open with nothing to send.2 participants
Builds on
feat/features-complete. Adds the chat-runtime layer on top of the redesigned core: pausable tool execution, an SSE-serializable run handle, a headless React hook, a Next.js reference demo, and shared test infrastructure.Summary
@agentic-kit/agent— pausable tools,AgentRunHandle(events /ReadableStream/ SSEResponse),maxSteps, decision lookup bytoolCallId.@agentic-kit/react(new package) —useChathook that POSTs to an SSE endpoint and folds events into messages, streaming snapshot, pending decisions, and executing tools.apps/nextjs-chat-demo(new) — Next.js App Router demo wiringagent.prompt(...).toResponse()touseChat, with a tool-approval UI.agentic-kit—injectDeferralResultshelper for the "user types instead of approving" flow;cross-fetchdropped in the OpenAI adapter in favor of nativefetch.tools/test/(scripted provider, SSE stub, fixtures), SSE parser tests, run-handle tests (443 LOC),useChattests (1011 LOC).What's New
@agentic-kit/agent— pause/resume + SSEdecisionJSON Schema. When the agent reaches a call with no attached decision, it emitstool_decision_pendingand stops. Attach the decision to the matchingtoolCallblock and callcontinue()to resume.AgentRunHandlereturned byprompt()/continue(), consumable exactly once as:await handle— run to completionhandle.events()— async iterator ofAgentEventshandle.toReadableStream()—ReadableStream<AgentEvent>handle.toResponse()— SSEResponseready to return from a Next.js / Hono / Express handlerparseSSEStream()exported from the package for clients consumingtoResponse().maxStepscap on model invocations per run (resets inprompt(), persists acrosscontinue());stopReason: 'completed' | 'max_steps'onagent_end.continue()and the underlying loop walk the message log backwards to find the most recent un-decidedtoolCallmatching a giventoolCallId, so callers may append unrelated messages between the pause and the response.@agentic-kit/react— new packageuseChat({ api, body?, initialMessages?, fetch?, on* }).messages,streamingMessage,isStreaming,pendingDecisions: ReadonlyMap<string, ToolDecisionPendingEvent>,executingToolCallIds: ReadonlySet<string>,error.send,sendMessages,setMessages(array or updater),respondWithDecision(toolCallId, value),abort().abort()finalizes any visible streamed text as an assistant message and drops orphantoolCallblocks so the next call doesn't re-pause.onMessage,onFinish,onDecisionPending,onToolExecutionStart/End,onError.runId. State lives in the message log.agentic-kit—injectDeferralResultsFor the case where the user types a new message while a tool is paused: synthesizes a stand-in
toolResultfor everytoolCallthat lacks both a decision and a paired result, so the server picks up a well-formed transcript.apps/nextjs-chat-demo/api/chat/route.tsconstructs anAgent, applies prior messages, and returnsagent.prompt(...).toResponse().useChatwithchat-input,chat-messages,tool-call-card,tool-approval-cardcomponents.Test infrastructure
tools/test/— repo-internal helpers (nopackage.json, imported via tsconfigpaths). Scripted provider, SSE stub, fixtures, shared index.pnpm teststays deterministic and offline.sse.test.ts(parser),run-handle.test.ts(443 LOC),use-chat.test.ts(1011 LOC underjsdom),inject-deferral-results.test.ts.@agentic-kit/reactis the only package onjsdom; everything else stays onnode.Cleanup
cross-fetchremoved from the OpenAI adapter — runtimes are expected to providefetch.sourceexport condition so workspace consumers can resolve TypeScript directly.Test Plan
pnpm install && pnpm build && pnpm testis green across packagesapps/nextjs-chat-demoboots, streams a chat turn, and a paused tool can be approved/denied viarespondWithDecisionsend()does not re-pauseinjectDeferralResultsflow: pause a tool, send a fresh user message instead of deciding, verify the next request carries synthesized stand-in results