feat(assemblyai): add streaming inactivity timeout option#1888
Conversation
🦋 Changeset detectedLatest commit: 18ecdfa The changes in this PR will be included in the next version bump. This PR includes changesets to release 35 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| * Session inactivity timeout in seconds. AssemblyAI accepts integer values | ||
| * from 5 to 3600; when unset, no inactivity timeout is applied. | ||
| */ | ||
| inactivityTimeout?: number; |
There was a problem hiding this comment.
🟡 Inactivity timeout uses seconds but the plain-form name implies milliseconds per repo convention
The timeout field is named without the InS suffix (inactivityTimeout at plugins/assemblyai/src/stt.ts:72) yet documented as accepting seconds, so users familiar with the codebase convention will misinterpret the expected unit.
Impact: Users may pass a value like 45000 (thinking milliseconds) when the API actually expects 45 (seconds), causing the session to never time out or to receive an API validation error.
Repository time-unit convention and within-file inconsistency
CLAUDE.md line 155 states: "Time units: Use milliseconds for all time-based values by default. Only use seconds when the name explicitly ends with InS."
Line 182 reiterates: "Only use seconds as the unit if the variable name explicitly ends with InS (e.g. delayInS)"
Within the same STTOptions interface, other time fields follow the convention correctly:
bufferSizeMsatplugins/assemblyai/src/stt.ts:64— explicitMssuffixminTurnSilenceatplugins/assemblyai/src/stt.ts:75— plain name, documented as "(ms)"maxTurnSilenceatplugins/assemblyai/src/stt.ts:77— plain name, documented as "(ms)"#speechDurationInSatplugins/assemblyai/src/stt.ts:209— explicitInSsuffix for seconds
The new inactivityTimeout at line 72 uses a plain name (implying ms) but the JSDoc at lines 69–71 says "in seconds" and the value is passed directly to the API at line 330 without conversion. To comply with the convention, the field should either be renamed to inactivityTimeoutInS or accept milliseconds and convert before sending.
Prompt for agents
The field `inactivityTimeout` in `STTOptions` (plugins/assemblyai/src/stt.ts:72) is documented as accepting seconds but uses a plain name which, per CLAUDE.md time-unit convention, implies milliseconds. Two options to fix:
1. Rename the field to `inactivityTimeoutInS` throughout the codebase (stt.ts and stt.test.ts), keeping the value in seconds as-is.
2. Keep the plain name `inactivityTimeout`, change the JSDoc to say milliseconds, and divide by 1000 at the point it's sent to the API (line 330: `inactivity_timeout: this.#opts.inactivityTimeout !== undefined ? this.#opts.inactivityTimeout / 1000 : undefined`). Update the test value from 45 to 45000.
Option 1 is simpler and less error-prone since the AssemblyAI API natively accepts seconds. Files affected: plugins/assemblyai/src/stt.ts (interface + usage at line 330) and plugins/assemblyai/src/stt.test.ts (line 87, 95).
Was this helpful? React with 👍 or 👎 to provide feedback.
| * Session inactivity timeout in seconds. AssemblyAI accepts integer values | ||
| * from 5 to 3600; when unset, no inactivity timeout is applied. | ||
| */ | ||
| inactivityTimeout?: number; |
There was a problem hiding this comment.
🚩 ElevenLabs uses same naming pattern, suggesting an established (if incorrect) precedent
The ElevenLabs plugin (plugins/elevenlabs/src/tts.ts:76) also uses inactivityTimeout in plain form for a seconds value (default 180 at line 30). This suggests the convention violation is an established pattern in the codebase for API-passthrough timeout values. A reviewer may want to consider whether to fix both at once, or accept this as an intentional exception for external API parameters that natively use seconds.
Was this helpful? React with 👍 or 👎 to provide feedback.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 18ecdfa154
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| ? JSON.stringify(this.#opts.keytermsPrompt) | ||
| : undefined, | ||
| language_detection: languageDetection, | ||
| inactivity_timeout: this.#opts.inactivityTimeout, |
There was a problem hiding this comment.
When inactivityTimeout is set and the stream has no audio/messages for that period, AssemblyAI closes the socket for the configured inactivity timeout; however run() treats any server-initiated close while the input is still open as a retryable error, so this parameter makes an idle long-lived stream immediately open another idle session with the same timeout instead of actually cleaning up. Please special-case the inactivity close (or defer reconnect until new audio / send KeepAlive, depending on the intended behavior).
Useful? React with 👍 / 👎.
1 participant
Summary
inactivityTimeoutto AssemblyAISTTOptions.inactivity_timeout.Why
AssemblyAI's streaming API supports an inactivity timeout for closing idle sessions. The JS plugin already exposes other connection-time Universal Streaming parameters in
STTOptions, but this timeout was not available, forcing applications to carry a patch or accept the provider default.This option is useful for voice agents that need explicit idle-session behavior in production, especially when calls can pause or when deployments need predictable cleanup of provider-side streaming sessions.
The option is documented as seconds to match the AssemblyAI API field semantics and to avoid mixing it with the framework's millisecond turn-timing options (
minTurnSilence,maxTurnSilence, etc.).Validation
pnpm exec vitest run plugins/assemblyai/src/stt.test.tspnpm --filter @livekit/agents-plugin-assemblyai lintpnpm exec prettier --check plugins/assemblyai/src/stt.ts plugins/assemblyai/src/stt.test.ts .changeset/assemblyai-inactivity-timeout.mdpnpm --filter @livekit/agents-plugin-assemblyai buildI also ran
pnpm --filter @livekit/agents-plugin-assemblyai api:check; it reached API Extractor but exited nonzero on the existing missing API report/release-tag/unresolved-link warnings for this plugin rather than a new type error.