Repository navigation
convexLoad: one subscription per query, SSR payload for hydration - #60
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
mikecann
left a comment
There was a problem hiding this comment.
Thanks again @mmailaender, this one is a big one! Same deal, had my agent go through it and it's looking really good. The escaping matches SvelteKit's own and server load results stay out of the payload, which was my main worry. Two things I think need fixing before we merge tho:
-
Remote functions. When SvelteKit runs a remote function it copies the request state and just adds
is_in_remote_function: true(derive_remote_function_eventinruntime/app/server/remote/shared.js), sois_in_universal_loadis still true inside it. So if a+page.tscalls a remotequery()that usesconvexLoadand only returns part of the result, the full result still ends up in the HTML. My agent reproduced it with a fake secret and it came out in the<head>. I think also checkingis_in_remote_function !== trueinhydration-server.ts:35should sort it. -
If the scanner can't find
</head>or<body, the payload gets put before the<!doctype>(hydration.ts:169), which puts the page into quirks mode. Seems safer to just embed nothing in that case since the client already falls back fine.
A few smaller ones if you fancy it: the empty payload on ssr = false routes means every convexLoad there gets the dev warning, a malformed payload throws at jsonToConvex (hydration.ts:202) instead of falling back, and the first Fixes heading in the changeset says hydration no longer queries without auth, but that's only with the handle.
Cheers!
3a79a83 to
c784afb
Compare
- On client-side navigation, open the live subscription and await its first value instead of a one-shot client.query() followed by a second subscription (Add → Remove → Add). convexLoadPaginated no longer fetches the first page twice. - New convexLoadHydration handle (convex-svelte/sveltekit/server) embeds convexLoad results of universal loads in the SSR HTML. Loads re-running during hydration reuse them instead of querying before setupAuth, and the live subscription is deferred until auth is set up. Server-load results (transport) are not embedded twice. hydrate: false opts out. - A load awaiting its first value holds its query (no unclaimed timeout while e.g. offline).
…ginated first page - Record convexLoad results only while a universal load runs (SvelteKit's is_in_universal_load request-store flag, as used by its remote functions). Server loads never contribute, so results they only use internally can't leak into the HTML, and a transported server-load result no longer suppresses a universal one with the same key. Fail-safe: without the flag nothing is embedded (warned once). - Inject the payload in <head>, before SvelteKit's start script. - Paginated: settle the first value only once a page arrived (Convex reports LoadingFirstPage immediately). - Dev warning when a hydration load misses the payload (e.g. streamed convexLoad); hydrate: false skips lookup and warning.
- Move convexLoadHydration to convex-svelte/sveltekit/server/hydration. It needs @sveltejs/kit/internal/server (2.31+) and the universal-load marker (2.56+); keeping it out of convex-svelte/sveltekit/server means withServerConvexToken keeps working on older SvelteKit versions. - Warn once (and embed nothing) whenever the marker can't be read: missing accessor, store without the marker, or a throwing accessor. - Find the real </head> with a scanner that skips comments and raw-text elements, so '</head>' inside a head script can't be split. - Document that everything convexLoad fetches while a universal load runs is embedded (hydrate: false for private helper queries), and that a streamed result is only embedded if it finished before the HTML. - e2e: universal + server layouts with parent(), exact payload contents.
- findTag: ASCII-only case folding (toLowerCase() can change the length, e.g. 'İ', shifting the insertion point), skip whole tags respecting quoted attribute values, and require a tag boundary for raw-text end tags (</script-not-a-tag> doesn't close a script). - Paginated state machine: SSR initialData is current data, not previous data — the initial args no longer mark it as loading, and the hydration guard runs before keepPreviousData. Fixes a loading flash right after hydration for convexLoadPaginated and usePaginatedQuery with initialData. - e2e: no loading state from SSR through hydration and the first live update (regular and paginated).
They are read up to the next '>' (doctype or bogus comment), so a '</head>' inside a processing instruction or CDATA-like declaration no longer counts as the end of the head.
…and safer client - Embed only when SvelteKit's request state says universal load AND not remote function. A remote function called from +page.ts copies the state, so its full result used to end up in the HTML. Unknown or missing flags embed nothing (fail closed). - Replace the </head> scanner with a %convex.hydration% placeholder in app.html, replaced in transformPageChunk (SvelteKit's documented way to inject request-specific markup). A missing placeholder embeds nothing and warns once in dev, instead of inserting before <!doctype>. - Emit no payload when nothing was collected, so ssr = false pages no longer get a dev warning for every convexLoad. - An unreadable payload value is treated as a miss (warn once in dev) instead of failing the load. - Rebased on the updated #58: holdWhile's timeout closes like the unclaimed timeout. - Changeset: the hydration fix applies with the handle.
c784afb to
5afe992
Compare
mikecann
left a comment
There was a problem hiding this comment.
Thanks @mmailaender, looks good! The app.html placeholder is a much nicer way of doing it. Cheers!
commit: |
Stacked on #58. Review the diff against
fix/release-route-scoped-queries.TL;DR:
convexLoadnow opens one subscription instead of two, and during hydration it reuses the server's result instead of querying Convex beforesetupAuth()has run. Also fixes a loading flash over server-rendered paginated data.None of these problems were introduced by #58. They all exist on
main.Why
1. Hydration queries Convex without auth
SvelteKit re-runs universal loads (
+page.ts,+layout.ts) in the browser during hydration. That happens before the root layout'ssetupAuth()has authenticated the client, soconvexLoadqueried Convex anonymously. The WebSocket traffic on/tests/convex-load-authshows it:For signed-in users, this replaced the server-rendered (authenticated) data with an anonymous result, or failed the load for queries that throw without auth. Hydration also had to wait for the WebSocket connection and a query round trip before the page became interactive.
Two obvious fixes don't work:
setupAuth.fetchreplay: its cache key includes request headers, and so the token.2. Two subscriptions per
convexLoadOn client-side navigation,
convexLoadran a one-shotclient.query()(subscribe → first result → unsubscribe) and then subscribed again, so every query ran twice.convexLoadPaginatedcouldn't even reuse a live result, because the paginated subscription adds anidto itspaginationOpts.3. Loading flash over paginated SSR data
convexLoadPaginated, andusePaginatedQuerywithinitialData, briefly rendered their loading state over the server-rendered results right after hydration. The paginated state machine marked SSR data as loading on the initialonArgsChange(since8da778f2), and itskeepPreviousDatabranch ran before the hydration guard (since97e072f). Codex found this during review.How
1. SSR payload, reused during hydration
A new opt-in
handle,convexLoadHydration, embeds the server'sconvexLoadresults in the page. During hydration,convexLoadreuses them and defers the live subscription untilsetupAuth/setupConvex, exactly like the transport path already does. No query and no waiting.AsyncLocalStorage) and embedded in<head>as<script type="application/json" data-convex-load>. Placing it in<head>means it exists before SvelteKit's start script runs the loads. The JSON is serialized withconvexToJsonand escaped like SvelteKit's ownserialize_data.</head>: a small scanner skips comments, tags (including quoted attribute values), and raw-text elements such as<script>. So a"</head>"string inside a head script can't split the document.closeConvex().+page.server.tsload may fetch private data and return only part of it. Universal loads are detected with SvelteKit's internalstate.is_in_universal_loadflag, which SvelteKit's remote functions use for the same purpose. Because it's internal API, it's read defensively: if it's missing (SvelteKit < 2.56) or unreadable, nothing is embedded and a warning is logged once.convex-svelte/sveltekit/server/hydrationis the only module that imports SvelteKit internals.convex-svelte/sveltekit/server(withServerConvexToken) keeps working on SvelteKit < 2.31.2. One subscription
convexLoadnow opens the live subscription immediately and awaits its first value, then returns that same subscription. The immediate subscribe bypasses the deferred queue, which would deadlock inside a load.client.query()did, and the subscription is disposed.3. No loading flash
SSR data is treated as current data: it isn't loading until the args change, and the hydration guard runs first.
What changes
convex-svelte/sveltekit/server/hydrationwithconvexLoadHydration(SvelteKit 2.56+).hydrate: falseonconvexLoad/convexLoadPaginated.convexLoadfetches while a universal load runs ends up in the HTML, including calls in helpers that load invokes.hydrate: falseopts a result out.Tests
hydration.spec.ts: XSS-safe escaping, exact round trips of Convex values (bigint, bytes, nested,null), placement in<head>across tricky markup, payload only during the initial hydration, malformed or missing payload, dev warning.hydration-server.spec.ts: the real handle andAsyncLocalStoragewith the real server path. Server-load results are never embedded; concurrent requests stay apart; fail-safe with one warning when the SvelteKit marker is missing.server-token.spec.ts:convex-svelte/sveltekit/serverstill imports without SvelteKit internals./tests/convex-load-auth,/tests/convex-load-serverand/tests/convex-load-nested.Authenticate, exactly oneAddper query, only universal-load results in the HTML (also with nested server and universal layouts andparent()).</script><script>…doesn't execute.Add.loadMoregeneration guard were removed temporarily to confirm their tests fail without them. The new e2e tests passed 3 repeated runs.pnpm test,pnpm check,pnpm lint,pnpm lint:package, andconvex dev --oncepass.Review
Four rounds with an independent Codex (
gpt-6-astra) reviewer. Final verdict: ready to merge, no blockers.What the review rounds fixed
</head>inside head scripts, missing warning when the SvelteKit marker is absent, separate entry point for SvelteKit < 2.31, docs wording (confidentiality contract, streamed-result timing, 30 s release timing).</head>in attribute values and after</script-not-a-tag>, the paginated SSR loading flash (Why 3).<?…>and<![CDATA[…>, included.Kept as is, and the reviewer agreed:
Possible follow-ups, none blocking:
<template>in<head>and legacy script double-escaping (only affects hand-written markup).main:'skip'withinitialData, andisLoadingduringLoadingMorewhile SSR data applies.Related