feat: worker_ban_migration + players.banned event infra (ban archival)#154
Open
extreme4all-ai wants to merge 1 commit into
Open
feat: worker_ban_migration + players.banned event infra (ban archival)#154extreme4all-ai wants to merge 1 commit into
extreme4all-ai wants to merge 1 commit into
Conversation
Implements PR 2 of the report-data-retention spec — the ban-archival leg's shared infrastructure (schema, event, migration logic, worker). - schema: report_archive table (denormalized, PK (reported_id, reported_at)) - grants: new ban-migration DB user (SELECT on report*; INSERT on report_archive) - event infra: PlayerBannedStruct + re-exports; players.banned kafka topic - component: database/report/migration.py (migrate_banned_player_reports — INSERT IGNORE ... SELECT join into report_archive; copy semantics, idempotent) - base: worker_ban_migration (mirrors worker_report; consumes players.banned) - project + Dockerfile + docker-compose (prod & dev) - tests: migration idempotency/copy-semantics + worker adapter/handle The worker is idle until PR 3 (runemetrics emit) lands — safe to deploy ahead of the producer. Historical banned players are covered by PR 4 (backfill).
This was referenced Jul 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements PR 2 of the Report Data Retention & Ban Archival spec — the largest PR. Bundles the shared infrastructure the ban-archival pipeline needs: schema, event, migration logic, and the consuming worker.
When Jagex bans a player (
label_jagex = 2), that reported player's location history is valuable and must be preserved in a query-optimized table before the prune job (PR #153) can delete it. This PR ships the archive target + the worker that populates it. It is independent of PR #153 (either can merge first).Changes
Schema & grants
01_tables.sql—report_archivetable: one row per(banned player, moment), denormalized with location coords + gear ids so it is queryable standalone. PK(reported_id, reported_at)doubles as the idempotency guard;idx_region_idfor the "where are bots seen" query.00_init.sql+02_grants.sql— new least-privilegeban-migrationuser (SELECTonreport/report_sighting/report_gear/report_location/Players;INSERTonreport_archive).Event infra
event_queue/structs/players_banned.py—PlayerBannedStruct(metadata, player_id, name)+ re-exports (structs/__init__.py,event_queue/__init__.py)._infra/_kafka/src/topics.py— registerplayers.bannedtopic (4 partitions).Component
database/report/migration.py→migrate_banned_player_reports()—INSERT IGNORE … SELECTjoin ofreport→report_sighting/report_location/report_gearintoreport_archive. Copy semantics (source untouched; prune reclaims later). Idempotent via PK +INSERT IGNORE— re-delivered events and backfill re-runs are safe.Base + project
worker_ban_migration/(mirrorsworker_report) — consumesplayers.bannedviaWorkerRunner, calls the migration fn per event.projects/worker_ban_migration/(pyproject.toml,Dockerfile,uv.lock).docker-compose.yml+docker-compose-dev.yml.Tests
test_report_migration.py— idempotency (second call inserts 0), correct join predicate, source rows not deleted (copy-only).worker_ban_migration/test_core.py— adapter version guard + per-record migration dispatch + skip on invalid version.Notes
label_jagex = 2before deploy) are covered by PR 4 (backfill).report_archivemust be created manually on existing volumes (ops runbook).Test plan
uv run ruff check— cleanuv run pytest test/components/bot_detector/database/test_report_migration.py test/bases/bot_detector/worker_ban_migration/— 7 passed