feat: job_backfill_banned (one-time historical ban archival)#156
Open
extreme4all-ai wants to merge 2 commits into
Open
feat: job_backfill_banned (one-time historical ban archival)#156extreme4all-ai wants to merge 2 commits into
extreme4all-ai wants to merge 2 commits into
Conversation
Implements PR 2 of the report-data-retention spec — the ban-archival leg's shared infrastructure (schema, event, migration logic, worker). - schema: report_archive table (denormalized, PK (reported_id, reported_at)) - grants: new ban-migration DB user (SELECT on report*; INSERT on report_archive) - event infra: PlayerBannedStruct + re-exports; players.banned kafka topic - component: database/report/migration.py (migrate_banned_player_reports — INSERT IGNORE ... SELECT join into report_archive; copy semantics, idempotent) - base: worker_ban_migration (mirrors worker_report; consumes players.banned) - project + Dockerfile + docker-compose (prod & dev) - tests: migration idempotency/copy-semantics + worker adapter/handle The worker is idle until PR 3 (runemetrics emit) lands — safe to deploy ahead of the producer. Historical banned players are covered by PR 4 (backfill).
Implements PR 4 of the report-data-retention spec — a one-shot job that archives location history for players already banned (label_jagex = 2) before deployment, since PR 3 only emits reactively on new ban transitions. - base: job_backfill_banned/core.py — paginates Players WHERE label_jagex = 2, calls migrate_banned_player_reports per player, skip-on-error & continue - project: pyproject + Dockerfile + uv.lock (NOT wired into docker-compose; invoked manually e.g. kubectl run) - tests: pagination terminates, migrates each banned player, skips on error Idempotent via INSERT IGNORE — safe to re-run if the one-time execution is interrupted. Stacked on PR 2 (migration component).
extreme4all
force-pushed
the
feat/worker-ban-migration
branch
from
3d74c21 to
09e5aad
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements PR 4 of the Report Data Retention & Ban Archival spec — a one-shot job that archives location history for players already banned (
label_jagex = 2) before deployment.PR #155 (runemetrics emit) only fires reactively on new ban transitions, so the pre-existing banned population needs a one-time scan. This covers them.
Changes
job_backfill_banned/core.py— paginatesPlayers WHERE label_jagex = 2(id > :player_id … LIMIT), callsmigrate_banned_player_reportsper player, skip-on-error & continue so one bad player can't abort the run.projects/job_backfill_banned/(pyproject.toml,Dockerfile,uv.lock).docker-compose.yml— invoked manually as a one-off (e.g.kubectl run/ a one-shot Job).Why one-time / manual
Per the spec's locked decision: this is a throwaway one-shot executed once post-deploy, not a recurring schedule or admin endpoint.
INSERT IGNOREmakes it idempotent, so a re-run after an interrupted execution is safe.Test plan
uv run ruff check— cleanuv run pytest test/bases/bot_detector/job_backfill_banned/— 3 passed