Skip to content

tests: update operator-metrics thresholds and chainsaw apply timeout#1

Closed
IshwarKanse wants to merge 9 commits into
os-observability:rhosdt-3.10from
IshwarKanse:rhosdt-3.10
Closed

tests: update operator-metrics thresholds and chainsaw apply timeout#1
IshwarKanse wants to merge 9 commits into
os-observability:rhosdt-3.10from
IshwarKanse:rhosdt-3.10

Conversation

@IshwarKanse

@IshwarKanse IshwarKanse commented Jun 10, 2026

Copy link
Copy Markdown
Member

Summary

  • Raise TEMPOSTACK_THRESHOLD from 1000 → 2500 and TEMPOMONOLITHIC_THRESHOLD from 1000 → 400 in tests/operator-metrics/max-loops/01-verify-metrics.yaml. The old thresholds predate several new test suites added after this test was written. Observed peaks across the full OpenShift e2e test suite run were 2051 (tempostack) and 276 (tempomonolithic); new values include ~20% headroom.
  • Increase chainsaw apply timeout from 10s → 30s in .chainsaw-openshift.yaml to prevent spurious context deadline exceeded failures when the API server is under load during concurrent test suite runs.

Test plan

  • Run chainsaw test --test-dir ./tests/operator-metrics --config .chainsaw-openshift.yaml after a full e2e suite run and confirm the job exits 0
  • Confirm no regressions in other test suites (apply steps no longer time out prematurely)

nickcaballero and others added 9 commits June 1, 2026 08:33
* feat: add support for metrics-generator

* fix lint

* add changelog entry

* update config/bundle

* move processor defaults to webhook

* clean up variable declaration

* fix configuration template

* add example

* add network policy

* use extraConfig.tempo instead of passing processors via crd

* add test

* regenerate manifests

* fix mtls

* fix storage

* fix network policy

* fix querier to metrics-generator connectivity

* set the enabled metrics-generator processors

* remove default storage path for metrics-aggregator

* add enabled field, set storage paths, update webhooks

- use `.enabled` field (similar to enabling the gateway, jaeger query etc.)
- set storage paths
- utilize default and validating webhooks
- remove metrics-generator deployment and service if `.metricsGenerator.enabled` is updated from `true` to `false`
- add example file to `docs/examples`

Signed-off-by: Andreas Gerstmayr <agerstmayr@redhat.com>

---------

Signed-off-by: Andreas Gerstmayr <agerstmayr@redhat.com>
Co-authored-by: Andreas Gerstmayr <agerstmayr@redhat.com>
Both fields declare a kubebuilder default ("Managed" and "static")
yet were also marked +required. A field that the API server always
populates via defaulting cannot meaningfully be a required input, and
the contradiction causes static schema validators (e.g. kubeconform)
to reject otherwise-valid TempoStack and TempoMonolithic resources
that omit these fields.

Mark both +optional while keeping their defaults, so the generated CRD
no longer lists them under `required`. Behaviour is unchanged: the API
server still defaults them to "Managed" and "static".

Signed-off-by: Timothy Griffiths <griffiths.timothy@gmail.com>
Bumps [actions/checkout](https://github.com/actions/checkout) from 6.0.2 to 6.0.3.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@de0fac2...df4cb1c)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: 6.0.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 4.36.0 to 4.36.1.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](github/codeql-action@7211b7c...87557b9)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-version: 4.36.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: renovate-sh-app[bot] <219655108+renovate-sh-app[bot]@users.noreply.github.com>
Co-authored-by: renovate-sh-app[bot] <219655108+renovate-sh-app[bot]@users.noreply.github.com>
)

| datasource | package                | from   | to      |
| ---------- | ---------------------- | ------ | ------- |
| go         | github.com/spf13/pflag | v1.0.9 | v1.0.10 |

Signed-off-by: renovate-sh-app[bot] <219655108+renovate-sh-app[bot]@users.noreply.github.com>
Co-authored-by: renovate-sh-app[bot] <219655108+renovate-sh-app[bot]@users.noreply.github.com>
Raise TEMPOSTACK_THRESHOLD (1000→2500) and TEMPOMONOLITHIC_THRESHOLD
(1000→400) to reflect the reconcile counts accumulated across the full
OpenShift e2e test suite (observed peaks: 2051 and 276 respectively,
+20% headroom). The old thresholds predate several new test suites that
were added after the operator-metrics test was written.

Also increase the chainsaw apply timeout (10s→30s) to prevent spurious
context-deadline-exceeded failures during resource apply steps on loaded
API servers.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@IshwarKanse

Copy link
Copy Markdown
Member Author

Closing in favour of a clean branch with only the intended commit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants