docs(osep): add OSEP-0018 Auto Idle-Pause and Resume-on-Revisit - #1449
docs(osep): add OSEP-0018 Auto Idle-Pause and Resume-on-Revisit#1449marunrun wants to merge 1 commit into
Conversation
github-actions
Bot
added
documentation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ea7ee3c0c5
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
ea7ee3c to
eca3138
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: eca3138f42
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| async def stop(self) -> None: ... # mirrors RenewIntentConsumer.stop | ||
| ``` | ||
|
|
||
| The coordinator owns its **own** `OrderedDict[str, _IdleSandboxState]` with the same LRU-evict pattern and 8192 cap as `consumer.py`. It does **not** share `_MemSandboxState` with `renew_intent`: the concerns differ (cooldown vs. idleness), as do their lifetimes and eviction semantics. |
There was a problem hiding this comment.
In the Helm values I checked, the server defaults to two replicas (kubernetes/charts/opensandbox-server/values.yaml:30-31), but this design keeps last_activity in a process-local OrderedDict while every server process starts its own scanner. When proxy traffic for a sandbox is load-balanced to replica A, replica B never sees the touch and can independently list that Running sandbox as idle and pause it during active use. The coordinator needs a shared activity store/leader scanner, or the feature must be explicitly guarded to single-replica server deployments.
Useful? React with 👍 / 👎.
| scan_interval_seconds = 60 # scanner period | ||
| idle_threshold_seconds = 600 # idle duration before pause; MUST be >= scan_interval | ||
| min_sandbox_age_seconds = 120 # don't pause a sandbox younger than this (warmup) | ||
| paused_retention_seconds = 86400 # TTL pushed before pause; protects files while Paused |
There was a problem hiding this comment.
This finite retention window contradicts the hard requirement that paused sandboxes must never be deleted by the TTL clock: after the scanner pauses a sandbox it only lists Running sandboxes, so nothing renews the now-paused object again, and the controller expiration check described above will still delete it once now + paused_retention_seconds passes. To satisfy the no-data-loss guarantee, the design needs periodic renewal of Paused sandboxes, a controller-side TTL freeze/no-expiry state, or it must explicitly weaken the requirement to finite retention.
Useful? React with 👍 / 👎.
| idle_threshold_seconds = 600 # idle duration before pause; MUST be >= scan_interval | ||
| min_sandbox_age_seconds = 120 # don't pause a sandbox younger than this (warmup) | ||
| paused_retention_seconds = 86400 # TTL pushed before pause; protects files while Paused | ||
| resume_ttl_seconds = 3600 # TTL reset after resume; shorter than paused_retention by design — post-resume, TTL returns to normal active semantics |
There was a problem hiding this comment.
Resetting to a fixed resume_ttl_seconds after resume runs while the current expireTime is still the pre-pause now + paused_retention_seconds, so the proposed renew_expiration(id, now + 3600) is usually earlier than the current expiresAt. The public contract for renew-expiration says the new value must be after the current expiresAt, so implementing this either violates the API contract by shortening TTLs or starts failing when the provider enforces the spec; preserve the original active TTL or only call renew when the target is later than the current expiration.
AGENTS.md reference: AGENTS.md:L43-L44
Useful? React with 👍 / 👎.
Introduce OSEP-0018 proposing a server-side IdlePauseCoordinator that automatically pauses sandboxes idle beyond a configurable threshold and resumes them on the next proxy access — the symmetric counterpart of renew_intent (OSEP-0009). The goal is to convert a hard memory constraint into a cheap storage constraint for long-lived, bursty Agent workloads that must not lose filesystem state. Design: - Reuses the proxy activity signal (the only unified signal; no /exec or /files endpoints exist) at api/proxy.py:265 (HTTP) and :431 (WS). - Background scanner lists Running sandboxes; idle > threshold -> renew_expiration(paused_retention) -> pause_sandbox. - Auto-resume: proxy ConnectError (Paused -> Pod scaled to 0) -> ensure_resumed -> resume -> poll to Running -> reset TTL -> retry. - Core safety invariant: renew-before-pause, because pause_sandbox only patches spec.pause and the Go controller's expiration check is unconditional, so a paused sandbox with a short TTL would be deleted. - Global [idle_pause] config; server-only; no Go/controller or client changes; default off. Mirrors integrations/renew_intent/ patterns. Builds on OSEP-0008 (pause/resume) and OSEP-0009 (auto-renew). Updates oseps/README.md index. Refs opensandbox-group#1448.
eca3138 to
c7f41b0
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c7f41b0f2c
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| 3. Poll `get_sandbox` every `IDLE_RESUME_POLL_INTERVAL_SECONDS` (1 s) until `state == "Running"` or `IDLE_RESUME_TIMEOUT_SECONDS` (60 s). Transition: Paused → Resuming → Running. | ||
| 4. Clear `resuming`, release lock, return True. | ||
|
|
||
| TTL is **not** reset here. The proxy access that triggered this resume flows on to `_schedule_proxy_renew` (`proxy.py:265`) once `get_endpoint` succeeds, so `renew_intent` extends the now-active sandbox's TTL on access — the same path every other active sandbox uses. This removes the earlier `resume_ttl_seconds` renew and its "shortening the TTL" contract problem. |
There was a problem hiding this comment.
The draft now relies on renew_intent to extend a sandbox after resume, but the current renew path is not just a server switch: AccessRenewController._try_renew_sync returns without renewing when ExtensionService.get_access_renew_extend_seconds() is absent, and OSEP-0009 makes that value a per-sandbox extensions["access.renew.extend.seconds"] opt-in. In the proposed global/no-client-change rollout, existing sandboxes usually will not have that extension, so a resumed-but-active sandbox keeps the original expireTime and can still be reclaimed while the agent is using it. Please add a server default/creation-time extension path or keep an idle_pause-owned renew before depending on this behavior.
AGENTS.md reference: AGENTS.md:L43-L44
Useful? React with 👍 / 👎.
| try: | ||
| endpoint = lifecycle.sandbox_service.get_endpoint(sandbox_id, port, resolve_internal=True) | ||
| except HTTPException as e: | ||
| if e.status_code != 404 or e.detail_code != SandboxErrorCodes.K8S_POD_IP_NOT_AVAILABLE: |
There was a problem hiding this comment.
If this proxy hook is implemented as shown, a K8S_POD_IP_NOT_AVAILABLE 404 never reaches ensure_resumed: FastAPI's HTTPException exposes status_code and detail, while get_endpoint stores the service code inside detail["code"], not detail_code. For paused or warming-up sandboxes, this line raises AttributeError while handling the 404, turning the endpoint miss into a proxy failure instead of deciding whether to resume. Use a checked e.detail.get("code") helper here.
Useful? React with 👍 / 👎.
| idle = now - (st.last_activity or created) | ||
| if idle < idle_threshold_seconds: continue | ||
| await _pause_one(sb.id) # pause only (see Pause) |
There was a problem hiding this comment.
The final re-read only covers activity that arrives before the read; because touch() remains unlocked, a proxy hit can update last_activity immediately after the idle calculation and before or during _pause_one, so the scanner can still pause a sandbox with an active request in flight. That violates the "no proxy access for the threshold" rule in the exact race the exception table says is avoided; guard the final check and pause with a per-sandbox lock, or make the pause conditional on an unchanged activity generation.
Useful? React with 👍 / 👎.
1 participant
Summary
Adds OSEP-0018: Auto Idle-Pause and Resume-on-Revisit — the design proposal for a server-side
IdlePauseCoordinatorthat automatically pauses sandboxes idle beyond a configurable threshold and resumes them on the next proxy access.It is the symmetric counterpart of
renew_intent(OSEP-0009):renew_intent: on access → extend TTLidle_pause: no access beyond a threshold → pause; on next access → resumeMotivation and full discussion: #1448.
Why
Long-lived Agent workloads are bursty — real I/O in short windows, idle for the majority of their lifetime (LLM think time). On a memory-constrained namespace the resident concurrency ceiling is fixed by physics, yet these sandboxes must not lose filesystem state (short TTL destroys intermediate files → Agent hallucination). OSEP-0008 gives manual pause/resume (release memory, keep state); OSEP-0009 renews TTL on access but never releases memory. Nothing today automatically releases idle memory while preserving state, or wakes a paused sandbox on return. This proposal fills that gap, turning the memory constraint into a cheap storage constraint.
What's in this PR
oseps/0018-auto-idle-pause-and-resume-on-revisit.md— full proposal (status:draft).oseps/README.md— index entry for OSEP-0018.Documentation only. No code, no schema, no behavior change. Implementation will follow in separate PRs once the design is reviewed.
Key design points (for reviewers)
api/proxy.py:265(HTTP) and:431(WS). Verified there are no/execor/filesendpoints; all Agent data-plane I/O flows through the proxy.pause_sandboxonly patchesspec.pauseand the Go controller's expiration check is unconditional (runs before pause scheduling), so a paused sandbox with a short TTL is still deleted. The proposal pushesrenew_expirationto a retention window before pausing — server-only, no Go change.ConnectError(Paused → Pod scaled to 0) →ensure_resumed→ resume → poll to Running → reset TTL → retry.[idle_pause]config (default off); mirrors[renew_intent]. Server-only, no client/fork changes, no new dependencies.Checklist
oseps/osep-template.md.templatestructureoseps/README.mdindex updateddraftLooking for feedback on: the renew-before-pause invariant as the TTL-safety approach (vs. a Go controller change), the global-switch vs per-sandbox opt-in trade-off, and the resume-latency visibility story.