Skip to content

feat(api): /events accepts since and limit query params with X-EXO-Last-Idx header#1979

Closed
5F0jd2vLq54RerYW wants to merge 2 commits into
exo-explore:mainfrom
5F0jd2vLq54RerYW:feat/events-cursor
Closed

feat(api): /events accepts since and limit query params with X-EXO-Last-Idx header#1979
5F0jd2vLq54RerYW wants to merge 2 commits into
exo-explore:mainfrom
5F0jd2vLq54RerYW:feat/events-cursor

Conversation

@5F0jd2vLq54RerYW
Copy link
Copy Markdown

Summary

Extends GET /events to accept optional since: int = 0 and limit: int | None = None query parameters, returning events in [since, end) where end = log_count if limit is None else min(since + limit, log_count). The response sets an X-EXO-Last-Idx header equal to the upper bound consumed, so clients can chain reads without a separate /state round-trip for lastEventAppliedIdx.

Reuses the existing DiskEventLog.read_range(start, end) primitive — the underlying support is already there at src/exo/utils/disk_event_log.py:120. Patch is ~10 LOC of API surface plus tests.

Why

Today /events always dumps the full ledger via read_all(), which is fine for ad-hoc inspection but expensive for any client that wants to tail event transitions in real-time (e.g., monitoring RunnerStatusUpdated for runner saturation telemetry). On a busy cluster the response can grow to several hundred KB and take seconds to stream. Cursor-based reads make incremental polling cheap.

Backward Compatibility

When called with no query parameters (existing behavior), the handler takes a fast path that calls read_all() exactly as before. No change to existing clients. The new X-EXO-Last-Idx header is also emitted in the no-params case, so callers that want to start using the cursor pattern can read it from the first response.

Test plan

New file src/exo/api/tests/test_stream_events.py adds 7 cases:

  • Full ledger dump with no params returns all events; X-EXO-Last-Idx reflects log length
  • since=N&limit=M returns events in [N, N+M); header reflects N+M
  • since=N only returns [N, end); header equals log length
  • since past end returns []; header equals log length (clamped)
  • limit larger than remaining is clamped to log length; no error
  • Negative since rejected with 422 (via FastAPI Query(ge=0))
  • Two chained reads using returned cursor cover full ledger with no overlap

All 7 pass with uv run pytest src/exo/api/tests/test_stream_events.py. uv run basedpyright and uv run ruff check are clean on both modified files.

Out of Scope

This patch only adds cursor support. SSE-style streaming on /events is not added — the existing application/json array response is preserved. Real-time push semantics can be layered later if the polling pattern proves insufficient.

Michael Mei and others added 2 commits April 24, 2026 22:14
…st-Idx header

Extends GET /events stream_events handler to accept optional since (default 0)
and limit (default None) query parameters. Reuses the existing
DiskEventLog.read_range(start, end) primitive, so cost is ~10 LOC of API
surface plus a test.

The response includes an X-EXO-Last-Idx header set to the upper bound
consumed, allowing clients to chain reads without a separate /state
round-trip for lastEventAppliedIdx. Backward compatible: no params (since=0,
limit=None) takes a fast path that calls read_all() and matches pre-patch
behavior exactly.

Use case: enables event-cursor tailing for downstream tools like the
control-plane EXO swarm dispatcher, which previously had to poll the full
~150KB /state snapshot to detect runner state changes.

Test: src/exo/api/tests/test_stream_events.py — 7 cases covering full dump
backward compat, since+limit, since-only, since-beyond-count (empty),
limit-larger-than-remaining (clamped), negative since rejected (FastAPI
ge=0 validator), and chained cursor reads with no overlap.

Patch is intended for upstream PR to exo-explore/exo.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@5F0jd2vLq54RerYW 5F0jd2vLq54RerYW marked this pull request as draft June 1, 2026 00:38
@5F0jd2vLq54RerYW 5F0jd2vLq54RerYW marked this pull request as ready for review June 1, 2026 00:41
@5F0jd2vLq54RerYW 5F0jd2vLq54RerYW marked this pull request as draft June 1, 2026 00:41
@5F0jd2vLq54RerYW
Copy link
Copy Markdown
Author

Superseded by #2133, which includes the exo_rs conftest stub and ruff formatting on top of the original feature commit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant