Skip to content

fix(import): back off on refused IMAP connections instead of shrinking the batch - #317

Merged
hieunc229 merged 2 commits into
hieunc229:mainfrom
Qadri-uk:fix/imap-connection-backoff
Nov 18, 2026
Merged

hieunc229 merged 2 commits into
hieunc229:mainfrom
Qadri-uk:fix/imap-connection-backoff

Conversation

@Qadri-uk

Copy link
Copy Markdown
Contributor

What is the issue?

Long IMAP imports from cPanel hosts die with Stream was cancelled. Each chunk opens a new connection and logs in. When the host throttles repeated logins (cPHulk/firewall), the chunk fails at the greeting; the job then halves the batch (making logins more frequent), retries immediately while still blocked, and fails after 4 attempts.

How does this PR resolve it?

retryImapChunkSmaller now recognises connection-level failures (isImapConnectionError: stream cancelled, connection closed, no greeting, reset/refused, timeout). For those it keeps the batch size, re-queues with a growing delay (60s × failures, capped at 10 minutes) and allows up to 10 attempts. Batch failures keep the existing halve-and-retry behaviour. enqueue gains an optional delaySeconds. No schema change.

Steps to reproduce (optional)

  1. On main, run a long IMAP import from a host that rate-limits IMAP logins.
  2. After a few thousand messages the chunk size collapses to 1-2 and the job fails with Stream was cancelled.
  3. On this branch the job pauses and resumes until the host accepts connections again.

Demo

N/A (backend). Worker log before: four immediate retries at greeting, then failed. After: retries spaced 1, 2, 3… minutes apart, batch size kept, job continues.

How was it tested?

  • node --test tests/imap-*.test.mjs tests/jmap-*.test.mjs passes, including a new case for isImapConnectionError
  • By hand on a deployed Worker: a 12,000-message cPanel import that failed on main resumed and kept running

🤖 Generated with Claude Code

https://claude.ai/code/session_01LRRzcVEYpPMKt9dL8AN2jq

claude and others added 2 commits November 15, 2026 19:30
…g the batch

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LRRzcVEYpPMKt9dL8AN2jq
Keep the batch size for the first three connection failures, since a
throttled host only needs time, then halve it as well: a batch that is
too heavy can also drop the connection. The backoff delay keeps growing.
@hieunc229
hieunc229 merged commit 4e572de into hieunc229:main Nov 18, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants