Skip to content

feat: worker_ban_migration + players.banned event infra (ban archival)#154

Open
extreme4all-ai wants to merge 1 commit into
developfrom
feat/worker-ban-migration
Open

feat: worker_ban_migration + players.banned event infra (ban archival)#154
extreme4all-ai wants to merge 1 commit into
developfrom
feat/worker-ban-migration

Conversation

@extreme4all-ai

Copy link
Copy Markdown
Contributor

Summary

Implements PR 2 of the Report Data Retention & Ban Archival spec — the largest PR. Bundles the shared infrastructure the ban-archival pipeline needs: schema, event, migration logic, and the consuming worker.

When Jagex bans a player (label_jagex = 2), that reported player's location history is valuable and must be preserved in a query-optimized table before the prune job (PR #153) can delete it. This PR ships the archive target + the worker that populates it. It is independent of PR #153 (either can merge first).

Changes

Schema & grants

  • 01_tables.sqlreport_archive table: one row per (banned player, moment), denormalized with location coords + gear ids so it is queryable standalone. PK (reported_id, reported_at) doubles as the idempotency guard; idx_region_id for the "where are bots seen" query.
  • 00_init.sql + 02_grants.sql — new least-privilege ban-migration user (SELECT on report/report_sighting/report_gear/report_location/Players; INSERT on report_archive).

Event infra

  • event_queue/structs/players_banned.pyPlayerBannedStruct(metadata, player_id, name) + re-exports (structs/__init__.py, event_queue/__init__.py).
  • _infra/_kafka/src/topics.py — register players.banned topic (4 partitions).

Component

  • database/report/migration.pymigrate_banned_player_reports()INSERT IGNORE … SELECT join of reportreport_sighting/report_location/report_gear into report_archive. Copy semantics (source untouched; prune reclaims later). Idempotent via PK + INSERT IGNORE — re-delivered events and backfill re-runs are safe.

Base + project

  • worker_ban_migration/ (mirrors worker_report) — consumes players.banned via WorkerRunner, calls the migration fn per event.
  • projects/worker_ban_migration/ (pyproject.toml, Dockerfile, uv.lock).
  • Compose: new service in docker-compose.yml + docker-compose-dev.yml.

Tests

  • test_report_migration.py — idempotency (second call inserts 0), correct join predicate, source rows not deleted (copy-only).
  • worker_ban_migration/test_core.py — adapter version guard + per-record migration dispatch + skip on invalid version.

Notes

  • The worker is idle until PR 3 (runemetrics emit) lands events — safe to deploy ahead of the producer.
  • Historical banned players (already label_jagex = 2 before deploy) are covered by PR 4 (backfill).
  • ⚠️ Existing volumes: no Alembic in this repo — schema is container-init only. report_archive must be created manually on existing volumes (ops runbook).

Test plan

  • uv run ruff check — clean
  • uv run pytest test/components/bot_detector/database/test_report_migration.py test/bases/bot_detector/worker_ban_migration/ — 7 passed

Implements PR 2 of the report-data-retention spec — the ban-archival leg's
shared infrastructure (schema, event, migration logic, worker).

- schema: report_archive table (denormalized, PK (reported_id, reported_at))
- grants: new ban-migration DB user (SELECT on report*; INSERT on report_archive)
- event infra: PlayerBannedStruct + re-exports; players.banned kafka topic
- component: database/report/migration.py (migrate_banned_player_reports —
  INSERT IGNORE ... SELECT join into report_archive; copy semantics, idempotent)
- base: worker_ban_migration (mirrors worker_report; consumes players.banned)
- project + Dockerfile + docker-compose (prod & dev)
- tests: migration idempotency/copy-semantics + worker adapter/handle

The worker is idle until PR 3 (runemetrics emit) lands — safe to deploy ahead
of the producer. Historical banned players are covered by PR 4 (backfill).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant