fix(retry): bound every server-controlled retry delay - #4
Merged
Conversation
mzner
force-pushed
the
fix/cap-retry-after
branch
2 times, most recently
from
August 9, 2026 14:32
ec01f61 to
d957b5a
Compare
A retried response could dictate how long the CLI paused. Both the WebDAV
and API transports parsed Retry-After and slept for exactly that long with
no ceiling, so `Retry-After: 86400` suspended a single retry for 24 hours.
No request is in flight while waiting, so --timeout could not recover from
it: that value is the per-request http.Client timeout, not a deadline for
the whole operation. A hostile or misconfigured server could therefore hang
any command indefinitely, and the exponential backoff had the same gap for
a large --retries with a long base wait.
Extract the retry policy into internal/retry and apply one ceiling of 30
seconds to every wait, whether it came from Retry-After or from backoff. An
excessive hint is clamped rather than discarded, so throttling is still
respected; the retry just happens no later than the ceiling.
The policy previously existed as three functions duplicated verbatim in
internal/httpapi and internal/webdav, which is why a bound added to one
would not have covered the other. Both packages now call the shared
implementation, and the TUS upload path inherits it.
Retry-After parsing follows RFC 9110's two forms. Previously the header was
parsed by appending "s" and calling time.ParseDuration, which accepted Go
duration syntax the specification does not define ("5m3" became 5m3s) while
rejecting other values outright ("1h"). Delta-seconds now parses as an
integer, and an out-of-range value saturates so that an absurd hint is
capped instead of being read as "no delay requested".
Tests cover the ceiling, both header forms, unusable values, and backoff
growth, plus one test at each transport boundary proving Retry-After is
routed through the bounded policy. The ceiling itself is asserted against
the pure functions, since observing it end to end would mean a test that
sleeps for it.
Clamping a server-requested delay to the ceiling bounded the wait but sent the follow-up request sooner than the server asked for. For a legitimate 429 that is the wrong trade: retrying before the requested delay expires can worsen the throttling or extend a rate-limit ban, and repeating it every attempt makes that likely. A server-requested delay is now honored exactly when it fits within the ceiling and refused with a typed error naming both the requested wait and the local limit when it does not. Waiting out a 24-hour delay is still never an option; the command stops promptly instead, and the user can run it again later. Exponential backoff is the CLI's own choice, so it stays clamped rather than refused. After also stops saturating at the ceiling, since the error has to be able to report what the server actually asked for.
- return an already-canceled context before evaluating retry policy - cover cancellation combined with an excessive Retry-After delay Signed-off-by: Matteo <mzner@pm.me>
mzner
enabled auto-merge (squash)
August 9, 2026 14:39
mzner
force-pushed
the
fix/cap-retry-after
branch
from
August 9, 2026 14:40
d957b5a to
be06f62
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
A retried response could dictate how long the CLI paused.
internal/httpapi/client.goandinternal/webdav/client.goeach parsedRetry-Afterand slept for exactly that duration with no ceiling.Retry-After: 86400suspended a single retry for 24 hours.--timeoutdoes not recover from this. It is the per-requesthttp.Client.Timeout, not a deadline for the whole operation, and no request is in flight while the retry loop waits. A hostile or misconfigured server could hang any command indefinitely, with nothing on screen to distinguish it from a crash. The exponential backoff had the same gap: a large--retrieswith a long base wait grew without bound.Confirmed by measurement — the capped path now waits 30s where the previous code would have slept 86400s.
Why it was in two places
The policy existed as three functions (
retryableStatus,retryAfter,waitRetry) duplicated verbatim acrossinternal/httpapiandinternal/webdav. A ceiling added to one copy would silently have left the other unbounded.Fix
internal/retrypackage owns the policy: which statuses may be retried, and how long to wait.MaxDelay = 30sbounds every wait, fromRetry-Afteror from backoff.Retry-After parsing
Parsing now follows RFC 9110's two defined forms. The previous implementation appended
"s"and calledtime.ParseDuration, which:5m3became 5m3s), and1h).Delta-seconds now parses as an integer. An out-of-range value saturates rather than failing, so an absurd hint is capped instead of being read as "no delay requested" — failing toward a bounded retry, not toward none.
Tests
internal/retrycovers the ceiling, both header forms, unusable values, whitespace, and backoff growth (96% statement coverage). One test at each transport boundary provesRetry-Afteris routed through the bounded policy.The ceiling itself is asserted against the pure functions rather than end to end, because observing a 30-second wait through an HTTP round trip means a test that sleeps for 30 seconds. That tradeoff is noted in both boundary tests.
Verification
go test ./...,go test -race,go vet,gofmt, andgolangci-lintall clean.make coveragepasses on all 12 gated packages, including the new one.Docs
ARCHITECTURE.md gains the package entry and a design rule ("Retry policy lives only in
internal/retry"), since this is a cross-cutting invariant new adapters must preserve. README documents the 30-second bound as user-visible behavior.Follow-up from review
A second commit (
27ce6aa) changes what happens when a server asks for a delay beyond the ceiling.Clamping respects throttling in the wrong direction. A server that asks for a long wait is usually rate-limiting, and a request sent before the requested delay expires can worsen the throttling or extend a ban — so clamping
Retry-After: 86400to 30s means hammering the exact server that asked to be left alone. Neither a 24-hour sleep nor an early retry is acceptable, so the operation now stops with an actionable error:A delay the CLI chose for itself is still simply clamped — the ceiling only refuses server-requested delays.
retry.Delayandretry.Waitgained an error return carrying a typed*retry.DelayTooLongError, so the refusal lives in the shared policy and all three call sites (httpapi,webdav, TUS) inherit it with no changes.Afterno longer saturates atMaxDelay, which would have hidden an excessive hint from the new check; it saturates atmath.MaxInt64and the ceiling decides. Exit-code behaviour is unchanged — both the old 429 error and the new typed error map to general failure.A test at the webdav boundary asserts the refusal is prompt: one attempt,
errors.Asmatches the typed error, and elapsed time stays underMaxDelay.