fix(http): apply per-read socket timeout to bound body-stall hangs#590
Open
nathantournant wants to merge 1 commit into
Open
fix(http): apply per-read socket timeout to bound body-stall hangs#590nathantournant wants to merge 1 commit into
nathantournant wants to merge 1 commit into
Conversation
Previously every HTTP verb passed an integer timeout to aiohttp, which wraps it as ClientTimeout(total=<int>). Once response headers arrive, aiohttp's total timer stops protecting the body read, so a server that streams a large response slowly can hold the connection open for any length of time with no error surfaced. Add _client_timeout() returning ClientTimeout(total=None, sock_read=self.timeout). - total=None preserves the current behaviour: long-but-progressing body reads are not hard-capped, matching what production callers already rely on for large resource listings. - sock_read=self.timeout bounds the inter-chunk gap: if the server stops sending bytes for more than self.timeout seconds, asyncio.TimeoutError is raised, propagates through request_with_retry uncaught, and is caught by _import_get_resources_cb's existing handler where it is classified as failure_class=http_timeout. Also tighten the catch chains in resources_handler.py: - except (asyncio.TimeoutError, TimeoutError) at the _import_get_resources_cb call site (asyncio.TimeoutError is a subclass of TimeoutError in Python 3.11+ but a distinct class in 3.9/3.10; the dual catch is safe on all supported versions). - isinstance check in _sanitize_reason updated to match.
|
michael-richey
approved these changes
Jun 4, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a
sock_readtimeout to the aiohttpClientTimeoutused by sync-cli'sHTTP client. Previously, only a bare integer was passed as the
timeoutargument; aiohttp wraps this as
ClientTimeout(total=<int>). Once responseheaders arrive, aiohttp's
totaltimer is effectively disarmed — theresponse-body read is unprotected. If the upstream API stalls mid-body for
any length of time, sync-cli waits indefinitely with no surfaceable signal.
This change adds a per-read socket-level deadline so a body-read stall of
more than
--http-client-timeoutseconds firesasyncio.TimeoutError,which is caught by the existing per-resource-type handler and surfaced as
a classified failure (
failure_class=http_timeout) instead of a silent hang.Why
Observed: an unbounded
GETagainst a large org's resource list (largeresponse body with slow server-side delivery) appears to sync-cli as a
silent multi-minute hang. The subprocess produces no stdout during the
body read, so any process running sync-cli as a subprocess (e.g. via
subprocess.Popen+cmd.Wait) cannot observe progress until the bodyread either completes or fails.
The fix is strictly additive:
total=Noneis preserved (we explicitlyallow long-but-progressing body reads), and
sock_readbounds theinter-chunk gap. Long successful body reads are unaffected.
Change
The helper
_client_timeout()is applied to all six HTTP verb call sites(
get,post,put,patch,delete,_post_raw) for consistency.Two catch chains in
resources_handler.pyare also tightened fromexcept TimeoutErrortoexcept (asyncio.TimeoutError, TimeoutError).In Python 3.11+,
asyncio.TimeoutErroris an alias forTimeoutErrorso there is no functional change there; in Python 3.9/3.10 (both in the
supported range
>=3.9) the two are distinct classes and the dual catchensures the
asyncio.TimeoutErrorraised by aiohttp is handled correctly.Tests
tests/unit/test_custom_client_timeout.py(new, 12 tests):TestClientTimeoutShape— verifies_client_timeout()returnsaiohttp.ClientTimeoutwithtotal=Noneandsock_readequal toself.timeoutfor several representative timeout values.TestVerbsUseClientTimeout— mocks the aiohttp session and assertsthat every HTTP verb (
get,post,put,patch,delete) forwardsa
ClientTimeoutrather than a bare integer.TestTimeoutPropagation— verifies that bothasyncio.TimeoutErrorand
TimeoutErrorraised during a body read propagate throughrequest_with_retryto the caller.tests/unit/test_failure_class.py(one test added):test_asyncio_timeout_error— confirms_sanitize_reasonclassifiesasyncio.TimeoutErrorasfailure_class=http_timeout, the same as theexisting
TimeoutErrorbranch.All 687 unit tests pass. Lint (
ruff) clean.Backwards compatibility
total=None(no wall-clock cap on the overall request) — matches thepreviously-observed behaviour for slow-streaming responses, so no regression.
sock_read=self.timeout— new bound on per-read gaps. Existing callerswho expect responses with no data gap longer than
--http-client-timeoutseconds see no behavioural change.
Out of scope
very large response bodies — separate consideration).