fix(archiver): add per-attempt timeout to redirect resolve, fix spaceManager loop leak

[dylanjeffers]

What

Two bugs causing archiver stem jobs to get stuck in waiting or active and never complete — the same symptom as the Summer Cypher Vol 2 remix contest stall.

---

Bug 1 (root cause): resolveCanonicalUrl hangs forever under load

File: apps/archiver/src/workers/createStemsArchive/utils.ts

resolveCanonicalUrl fetches api.audius.co to resolve the download redirect with no per-attempt timeout. node-fetch has no built-in timeout. When the API is slow or rate-limited under contest traffic spikes, the fetch blocks indefinitely and holds a BullMQ worker slot open.

With concurrentJobs = 5, five such hangs fill every slot. New jobs queue in waiting and never get picked up. The loading wheel spins forever.

The code already documented the rate-limiting risk and had retry logic for 429/5xx responses — neither helped when the API simply didn't respond at all.

Fix: Wrap each fetch attempt in linkAbort(signal, REDIRECT_RESOLVE_TIMEOUT_MS) (30 s, matching the existing mirror download timeout). A network timeout is treated as a retryable error — same path as 429/5xx — so a momentary API stall retries up to MAX_REDIRECT_RESOLVE_ATTEMPTS times before failing the job. The job-level abort signal propagates through so a cancelled job still surfaces its own AbortError.

---

Bug 2 (secondary): spaceManager.waitForSpace leaks allocated space after timeout

File: apps/archiver/src/workers/spaceManager.ts

waitForSpace used const shouldContinue = true, making the background claimSpacePromise loop run forever. When Promise.race settled on the timeout, the outer IIFE threw — but the background loop kept polling every 100 ms. If it later claimed space, no one would ever call releaseSpace: state.usedSpace inflated permanently. Over time this caused spurious space exhaustion, making subsequent jobs timeout at the waitForSpace stage.

Fix: Add a loopController = new AbortController(). timeoutPromise calls loopController.abort() before rejecting, stopping the loop on its next iteration. A post-claim double-check + releaseSpace handles the narrow race window where the mutex was held at the moment of abort.

---

Contest-specific finding

No special gating or URL differences for Summer Cypher Vol 2 stems vs regular track downloads — the archiver uses identical SDK paths. The contest caused a traffic spike that saturated the 5 concurrent worker slots via Bug 1.

k8s

GCloud auth expired locally — pod logs unavailable. Deployment config shows autoUpgradeSchedule: "*/5 * * * *" with imageTag: audius/archiver:latest, so the fix auto-rolls within 5 min of merge.