Repository navigation
test(ai-sdk-provider): pin array content normalization for Gemini - #1157
Merged
Merged
Conversation
Member
Author
|
Part of one change with: |
andrelandgraf
commented
Nov 17, 2026
andrelandgraf
left a comment
Member
Author
There was a problem hiding this comment.
Aileen review: approve (test, low risk, b1ba6e9)
Verdict
Approve, low risk. The diff changes comments and tests in @neon/ai-sdk-provider and no runtime code. It replaces the TODO(#308) that said to delete the array content normalizer with the condition that keeps it, adds a regression test for the Gemini array shape, and points the live e2e normalization block at gemini-4-argon.
Evidence
- The new tests guard the wrapper. With
wrapFetchWithHarmonyNormalizationremoved fromprovider.ts, both newsdk-compatibility.test.tscases fail withInvalid input: expected string, received arrayatchoices[0].message.content. With the wrapper in place, they pass. The tree was restored after the check. - Routing matches the claim:
gemini-4-argonresolves to themlflowroute (createChatModel), which is the only route wrapped by the normalizer. Claude goes through the Anthropic route, so the four Claude models in the description's live check never reach this code. - Local build of affected packages passes. In
packages/ai-sdk-provider,vitest run --testTimeout=60000gives 13 files passed and 1 skipped (neon-catalog-drift, env-gated, pre-existing), with 132 tests passed. The rootpnpm --filter '...[HEAD]'form matched no projects on a clean checkout, so the package suite was run directly. - PR checks are green on head
b1ba6e9, includingLive AI Gateway e2e, which runs the renamedgemini-4-argonblock against a real gateway.
Regressions
There are none, because runtime behavior is unchanged. The only loss is the live streaming check for gpt-oss, which the description says now returns string content.
Non-blocking notes
- The comments mention history ("gpt-oss, which it was written for (#308), is compliant now") and repeat the three Gemini ids in the module header and in the test fixture. Those lists will drift.
- Nothing fails when the gateway is fixed: the e2e
generateTextcase passes whether or not the wrapper rewrote anything, so deleting the module still depends on someone remembering to do it. - The
streamTextcase in the normalization block is a plain smoke test, as its comment says. maxOutputTokens: 512ongemini-4-argoncould flake if thinking uses up the budget. CI passed with it on this head.- The
harmonynaming is now a misnomer. Renaming it belongs in a separate PR.
Member
Author
|
Waiting for Andre: category test needs Andre. Merge it when it is ready, or comment to have Aileen change it. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Found by a coding-agent eval of AI Gateway onboarding (Claude Code with Claude Opus 5.5): the chat completions
contentshape differs by model.Problem
@neon/ai-sdk-providerwraps the unified chat route's fetch with a normalizer that flattens an arraymessage.contentinto a string. Without it, the AI SDK rejects the response. The module and its wiring inprovider.tswere markedTODO(#308): a temporary gpt-oss workaround, to be deleted once gpt-oss returns a compliant shape. That condition has now been met: gpt-oss models return a string on/v1/chat/completions.But the normalizer still does real work. On a live gateway branch,
gemini-3-7-flash,gemini-3-8-flash, andgemini-4-argonreturnmessage.contentas an array of{ type: 'text', text, thoughtSignature }on every non-streaming call. Following the TODO would breakgenerateTextfor those models for every AI SDK user. The provider is also the route the AI Gateway docs now recommend for getting plain text from those models with no extra code.No unit test covered that shape. The only end-to-end check exercised gpt-oss, so it passed with or without the wrapper.
Solution
neon-harmony-normalize.tsandprovider.ts: replace the TODO with what is true now. The module normalizes any arraycontenton the chat route, and it stays until the gateway returns a string for every model. The comments now describe the Gemini shape next to the gpt-oss one. No runtime code changed.sdk-compatibility.test.ts:generateTextwith AI SDK 6 and 7 returns the text from the Gemini array shape through a mocked fetch. It lives in the provider-level test file rather than the normalizer's own, so deleting the module along with its tests still fails CI.e2e/gateway-matrix.e2e.test.ts: the normalization block now callsgemini-4-argoninstead ofgpt-oss-120b. Its stream case is renamed as a plain sanity check, since Gemini streams string deltas.This belongs in the provider because the provider is the layer that absorbs the gateway's array shape for AI SDK consumers. The docs that point users to it are updated in the matching neondatabase/website change.
No changeset: only comments and tests change.
Verification
provider.ts, the two new tests fail withInvalid input: expected string, received arrayatchoices[0].message.content. With the wrapper in place, they pass.gemini-4-argon, for bothgenerateTextandstreamText. A separate run ofgenerateTextandstreamTextthrough the provider returned plain text for all seven models that return array content on the raw route: three Gemini and four Claude.pnpm --filter '...[HEAD]...' --filter '!.' build, thennpm_config_workspace_concurrency=1 VITEST_MIN_FORKS=1 VITEST_MAX_FORKS=4 pnpm --filter '...[HEAD]' --filter '!.' test:ci --testTimeout=60000(13 files passed, 1 skipped; 132 tests passed).pnpm lint:cipasses; its one info notice is in an untouched CLI file.Left open
/v1/chat/completionsreturns non-spec arraycontentfor these Gemini models. Once it returns a string, this module can be deleted, and the new tests should then be rewritten against a compliant body.Opened by Aileen (factory
neon-dx).