Skip to content

[fix] Enforce trajectory start rate after waiting for concurrency - #2169

Closed
jxucoder wants to merge 1 commit into
NovaSky-AI:mainfrom
jxucoder:codex/fix-rate-limiter-start-bursts
Closed

[fix] Enforce trajectory start rate after waiting for concurrency#2169
jxucoder wants to merge 1 commit into
NovaSky-AI:mainfrom
jxucoder:codex/fix-rate-limiter-start-bursts

Conversation

@jxucoder

@jxucoder jxucoder commented Sep 5, 2026

Copy link
Copy Markdown

When rate and concurrency limits are enabled together, queued trials consume rate tokens before waiting for a concurrency slot. After a long-running trial finishes, they can start in a burst beyond the token bucket's allowed start rate.

For example, with rate=1 and max_concurrency=1, two trials queue at t=1 and t=2 behind an occupied slot. When the slot is released at t=2, both can start at t=2 instead of the second waiting for another token.

Reserve a concurrency slot before acquiring a rate token, and release that slot if the rate-token wait fails or is cancelled. Add a simulated-clock regression for queued start bursts and tests for cancellation at both waiting stages. Rename the existing combined-limit test to match the behavior it checks.

Validation:

  • Before the fix: the new burst regression fails; 38 tests pass.
  • After the fix: all 39 rate-limiter tests pass.
  • Ruff 0.11.9, Black 24.10.0, and git diff --check pass for the changed files.
  • Gitleaks 8.24.2 passes for the staged changes.

The tests ran on macOS/Python 3.12 using an isolated harness that loads the actual limiter module at its canonical import name and disables parent conftest loading, bypassing unrelated training imports and the Ray fixture. Standard project-environment validation and Harbor end-to-end validation are still pending.

@jxucoder jxucoder closed this Sep 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant