Summary
v1.9.0 fetches only workflow runs created in the last 12 hours, but the two per-run GaugeVec collectors retain label sets from every prior refresh. In an active installation, expired run IDs disappear from the API response but remain registered in the exporter, so metric cardinality and process memory grow until the process is restarted or reaches its memory limit.
The current master loop sets workflowRunStatusGauge and workflowRunDurationGauge for each returned run, then sleeps and repeats without resetting either vector or deleting expired label values:
https://github.com/Labbs/github-actions-exporter/blob/master/pkg/metrics/get_workflow_runs_from_github.go#L108-L144
Minimal reproduction
- Start
v1.9.0 with one or more repositories that create workflow runs continuously. Keep the default per-run fields, especially id.
- Record these values after a fresh start:
count(github_workflow_run_status)
count(github_workflow_run_duration_ms)
go_memstats_heap_alloc_bytes
process_resident_memory_bytes
- Let the exporter refresh across several 12-hour windows while new workflow-run IDs are created.
- Repeat the measurements after older runs are outside the query window.
Expected: expired run label sets are removed, and series count / heap / RSS remain bounded by the configured workflow window.
Actual: expired label sets remain in the GaugeVec maps. In one three-repository deployment, RSS grew from roughly 45 MiB after restart to about 253 MiB at a 256 MiB container limit over several weeks, without a corresponding increase in the intended 12-hour window.
Suggested direction
Reset the per-run gauges at the start of each collection cycle, or explicitly delete label values for run IDs no longer present in the current window. A maintained implementation of the reset-and-prune approach is visible here:
https://github.com/amirwollman/github-actions-exporter/blob/v1.0.6/pkg/metrics/get_workflow_runs_from_github.go
This report contains no credentials or private repository data.
Summary
v1.9.0fetches only workflow runs created in the last 12 hours, but the two per-runGaugeVeccollectors retain label sets from every prior refresh. In an active installation, expired run IDs disappear from the API response but remain registered in the exporter, so metric cardinality and process memory grow until the process is restarted or reaches its memory limit.The current
masterloop setsworkflowRunStatusGaugeandworkflowRunDurationGaugefor each returned run, then sleeps and repeats without resetting either vector or deleting expired label values:https://github.com/Labbs/github-actions-exporter/blob/master/pkg/metrics/get_workflow_runs_from_github.go#L108-L144
Minimal reproduction
v1.9.0with one or more repositories that create workflow runs continuously. Keep the default per-run fields, especiallyid.count(github_workflow_run_status)count(github_workflow_run_duration_ms)go_memstats_heap_alloc_bytesprocess_resident_memory_bytesExpected: expired run label sets are removed, and series count / heap / RSS remain bounded by the configured workflow window.
Actual: expired label sets remain in the
GaugeVecmaps. In one three-repository deployment, RSS grew from roughly 45 MiB after restart to about 253 MiB at a 256 MiB container limit over several weeks, without a corresponding increase in the intended 12-hour window.Suggested direction
Reset the per-run gauges at the start of each collection cycle, or explicitly delete label values for run IDs no longer present in the current window. A maintained implementation of the reset-and-prune approach is visible here:
https://github.com/amirwollman/github-actions-exporter/blob/v1.0.6/pkg/metrics/get_workflow_runs_from_github.go
This report contains no credentials or private repository data.