feat(api): /events accepts since and limit query params with X-EXO-Last-Idx header#1979
Closed
5F0jd2vLq54RerYW wants to merge 2 commits into
Closed
feat(api): /events accepts since and limit query params with X-EXO-Last-Idx header#19795F0jd2vLq54RerYW wants to merge 2 commits into
5F0jd2vLq54RerYW wants to merge 2 commits into
Conversation
…st-Idx header Extends GET /events stream_events handler to accept optional since (default 0) and limit (default None) query parameters. Reuses the existing DiskEventLog.read_range(start, end) primitive, so cost is ~10 LOC of API surface plus a test. The response includes an X-EXO-Last-Idx header set to the upper bound consumed, allowing clients to chain reads without a separate /state round-trip for lastEventAppliedIdx. Backward compatible: no params (since=0, limit=None) takes a fast path that calls read_all() and matches pre-patch behavior exactly. Use case: enables event-cursor tailing for downstream tools like the control-plane EXO swarm dispatcher, which previously had to poll the full ~150KB /state snapshot to detect runner state changes. Test: src/exo/api/tests/test_stream_events.py — 7 cases covering full dump backward compat, since+limit, since-only, since-beyond-count (empty), limit-larger-than-remaining (clamped), negative since rejected (FastAPI ge=0 validator), and chained cursor reads with no overlap. Patch is intended for upstream PR to exo-explore/exo. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Author
|
Superseded by #2133, which includes the exo_rs conftest stub and ruff formatting on top of the original feature commit. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Extends
GET /eventsto accept optionalsince: int = 0andlimit: int | None = Nonequery parameters, returning events in[since, end)whereend = log_count if limit is None else min(since + limit, log_count). The response sets anX-EXO-Last-Idxheader equal to the upper bound consumed, so clients can chain reads without a separate/stateround-trip forlastEventAppliedIdx.Reuses the existing
DiskEventLog.read_range(start, end)primitive — the underlying support is already there atsrc/exo/utils/disk_event_log.py:120. Patch is ~10 LOC of API surface plus tests.Why
Today
/eventsalways dumps the full ledger viaread_all(), which is fine for ad-hoc inspection but expensive for any client that wants to tail event transitions in real-time (e.g., monitoringRunnerStatusUpdatedfor runner saturation telemetry). On a busy cluster the response can grow to several hundred KB and take seconds to stream. Cursor-based reads make incremental polling cheap.Backward Compatibility
When called with no query parameters (existing behavior), the handler takes a fast path that calls
read_all()exactly as before. No change to existing clients. The newX-EXO-Last-Idxheader is also emitted in the no-params case, so callers that want to start using the cursor pattern can read it from the first response.Test plan
New file
src/exo/api/tests/test_stream_events.pyadds 7 cases:X-EXO-Last-Idxreflects log lengthsince=N&limit=Mreturns events in[N, N+M); header reflectsN+Msince=Nonly returns[N, end); header equals log lengthsincepast end returns[]; header equals log length (clamped)limitlarger than remaining is clamped to log length; no errorsincerejected with 422 (via FastAPIQuery(ge=0))All 7 pass with
uv run pytest src/exo/api/tests/test_stream_events.py.uv run basedpyrightanduv run ruff checkare clean on both modified files.Out of Scope
This patch only adds cursor support. SSE-style streaming on
/eventsis not added — the existingapplication/jsonarray response is preserved. Real-time push semantics can be layered later if the polling pattern proves insufficient.