fix(event/stream): scale the close drain bound with PullTimeout - #10
Open
ttradesman wants to merge 2 commits into
Open
fix(event/stream): scale the close drain bound with PullTimeout#10ttradesman wants to merge 2 commits into
ttradesman wants to merge 2 commits into
Conversation
closeDrainTimeout was a fixed 5s while the pull loop can be parked inside a non-ctx-aware SendSoap for as long as the caller's HTTP client allows. That was fine when PullTimeout was also 5s, but a caller raising the poll made the drain the shorter of the two: Close then times out on every shutdown landing mid-poll, and the Unsubscribe below it is skipped, orphaning the pull-point at the camera until its termination expires. Derive the default as PullTimeout + closeDrainSlack so it outlasts a poll, and expose CloseDrainTimeout for callers whose client ceiling is higher still — that ceiling, not PullTimeout, is the real worst case for a stalled camera, and only the caller knows it. The hung-HTTP test now sets it explicitly; it was relying on the constant being small.
The drain default was PullTimeout + slack, which bounds the poll but not the call. PullTimeout is how long the camera holds a poll; http.Client.Timeout is what actually caps the request, and callers set it higher — it has to cover dial, TLS and the response transfer on top of the poll it outlasts. So a caller with a 30s poll and a 40s ceiling got a 40s drain against a call that can also run 40s: a dead heat, and Close still times out and skips Unsubscribe on the shutdowns that land mid-poll. The Options escape hatch covered it, but a default that every caller has to override is not much of a default. Take whichever of the two is longer. A zero ceiling means unbounded, where no finite drain helps, so the poll stays the best bound available. Derived in NewStream because only that entry point sees the device; newStream keeps the PullTimeout-only bound for callers wiring their own caller implementation.
Member
|
@ttradesman we have some merge conflicts now, due to merging your previous PR's. All are currently deployed in v3.9.0. Many thanks! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Scale the close drain bound with PullTimeout
Summary
closeDrainTimeoutwas a fixed 5 seconds.Closeuses it to wait for the pull and renew goroutines to exit, but those loops block insidecaller.SendSoap, which is not ctx-aware — as thecallerinterface documents, only the HTTP client's own timeout can unblock them. So the drain bound has to outlast an in-flight poll, and a fixed 5s only did that whilePullTimeoutwas also 5s.Why it matters
Any caller raising
PullTimeout— which is the natural thing to do, since a long-poll is meant to be long and a longer poll means fewer requests — makes the drain the shorter of the two.Closethen times out on every shutdown that lands mid-poll, which for a 30-second poll is most of them.The consequence is not just a spurious error. When the drain times out,
Closereturns early and theUnsubscribebelow it is skipped:So the pull-point is left registered at the camera until
InitialTerminationexpires on its own. On a consumer that rebuilds its stream on reconfiguration or reconnect, that orphans a subscription per cycle, and the goroutine and its socket stay live until the HTTP call eventually returns.Changes
The default is now derived rather than fixed, from whichever of the two relevant timeouts is longer:
Both matter, and taking only the first is not enough.
PullTimeoutis how long the camera holds a poll, buthttp.Client.Timeoutis what actually caps the request — it has to cover dial, TLS and the response transfer on top of the poll it outlasts, so callers set it higher. A 30s poll behind a 40s ceiling would otherwise get a 40s drain against a call that can itself run 40s: a dead heat, andClosestill loses the shutdowns that land mid-poll.A zero client ceiling means unbounded, where no finite drain helps, so
PullTimeoutremains the best bound available in that case.This is derived in
NewStream, because that is the only entry point with the device in hand. Callers wiring their owncallerimplementation keep thePullTimeout-only bound.Options.CloseDrainTimeoutoverrides both, following the package's existing zero-means-default convention.closeDrainTimeoutthe constant is replaced bycloseDrainSlack, which is the part that was genuinely a constant; the bound itself was never one.Tests
TestCloseDrainTimeout_ScalesWithPullTimeoutcovers the derivation from the default poll, derivation from a long poll, and an explicit value overriding both. It also asserts the invariant directly — the resolved drain must exceed the resolvedPullTimeout— so a future change to either default cannot silently reintroduce the inversion.TestDrainFor_BoundedByTheLongestPossibleCallcovers the client-ceiling half: a ceiling above the poll becomes the bound, a ceiling below it does not, equal values leave the poll as the bound, and an unbounded ceiling falls back to the poll. Each case also asserts the resulting drain outlasts both inputs.TestClose_BoundedWhenLoopsStuckOnHungHTTPnow setsCloseDrainTimeoutexplicitly. It had been relying on the constant being small, and would otherwise sit for the derived default on every run.Compatibility
Behavioural change for callers that set a
PullTimeoutabove 5 seconds:Closenow waits longer before giving up, which is the point — it waits long enough to drain cleanly and reachUnsubscribe. Callers who preferred the old fixed bound can setCloseDrainTimeout: 5 * time.Secondand get exactly the previous behaviour. No signature changes;Optionsgains one optional field.