Skip to content

Run a top-K aggregation and write the ranked rows (Step 3 of #386) - #494

Open
zipdoki wants to merge 5 commits into
feat/topk-metadatafrom
feat/topk-aggregate
Open

Run a top-K aggregation and write the ranked rows (Step 3 of #386)#494
zipdoki wants to merge 5 commits into
feat/topk-metadatafrom
feat/topk-aggregate

Conversation

@zipdoki

@zipdoki zipdoki commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Summary

A group can declare a top-K, but nothing computes one. The declaration sits in metadata and no ranked row is ever written. This runs the aggregation.

TopkAggregationHandler scans the source group with the declaration's ranges interpolated against the event, sums metric per (entity, dimension key), and writes rank rows keyed by rankSource(...) under the metric_desc index. When a declaration is a sliding window and refreshAfterMillis > 0, it also schedules a refresh
for the instant the bucket leaves the window. Consuming that refresh message is the next PR in the stack.

POST /aggregations/v1/aggregate drives it, taking a batch of edges:

{
  "items": [
    {
      "database": "graph",
      "table": "likes",
      "edge": {
        "source": "...",
        "target": "...",
        "...": "..."
      }
    }
  ]
}
{
  "items": [
    {
      "database": "graph",
      "table": "likes",
      "source": "...",
      "target": "...",
      "status": "OK",
      "error": null
    }
  ]
}

Status and error are reported per item, so one bad edge does not sink the rest of the batch.

Two bucket changes ride along because the window math depends on them. Bucket now has a test pinning how a date bucket rounds both a value and a range bound, which the refresh schedule reads directly, and its format precision is read in one place instead of being re-derived at each call site. Separately, rank keys escape | so
two rankings can never collide on one row.

Stacked on #493 — that needs to land first.

Part of #386.

Test plan

  • ./gradlew :core:test --tests '*BucketTest*' — date bucket rounding for values and for range bounds
  • ./gradlew :core:test --tests '*AggregationConstantsTest*' — rank key composition and | escaping
  • ./gradlew :engine:test --tests '*RankingInputsTest*'ranges interpolation, entity and dimension resolution
  • ./gradlew :engine:test --tests '*TopkAggregationHandlerTest*' — aggregation over the source group, rank row writes, refresh scheduling
  • ./gradlew :server:test --tests '*MetadataAggControllerE2ETest*'POST /aggregations/v1/aggregate over HTTP
  • ./gradlew spotlessKotlinCheck build — formatting and full build

AI Assistance

  • This PR was written largely with AI assistance.
    • Tool / model: Claude Code (Opus 5)

@zipdoki zipdoki self-assigned this Aug 19, 2026
@zipdoki
zipdoki requested a review from em3s as a code owner August 19, 2026 04:31
@dosubot dosubot Bot added size:XL This PR changes 500-999 lines, ignoring generated files. enhancement New feature or request labels Aug 19, 2026
@zipdoki
zipdoki force-pushed the feat/topk-aggregate branch from 59beb1e to 37e1c4b Compare August 19, 2026 08:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request size:XL This PR changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant