Skip to content

feat(otlp-receiver): Otel receiver config value defaults and missing env vars - #80

Merged
sudopower merged 6 commits into
mainfrom
etl-1047-1048-otlp-receiver-config
May 7, 2026
Merged

sudopower merged 6 commits into
mainfrom
etl-1047-1048-otlp-receiver-config

Conversation

@sudopower

@sudopower sudopower commented May 7, 2026 •

Copy link
Copy Markdown
Member

Summary

  • ETL-1047 / ETL-1048 — expose maxConcurrentRequests and natsChunkSize as first-class chart values wired to GLASSFLOW_OTLP_MAX_CONCURRENT_REQUESTS and GLASSFLOW_OTLP_NATS_CHUNK_SIZE
  • ETL-1058 (bug fix) — the OTLP receiver was missing GLASSFLOW_OTEL_METRICS_ENABLED and GLASSFLOW_OTEL_LOGS_ENABLED in its configmap, so both defaulted to true at startup and the receiver always tried to connect to the OTel collector — even when it wasn't deployed. This caused noisy connection errors in the logs. Now both flags are driven by global.observability.metrics.enabled / global.observability.logs.enabled, matching what the API configmap already does.

Config changes

Value Default Env var
sources.otlpReceiver.maxConcurrentRequests 50 GLASSFLOW_OTLP_MAX_CONCURRENT_REQUESTS
sources.otlpReceiver.natsChunkSize 1000 GLASSFLOW_OTLP_NATS_CHUNK_SIZE
global.observability.metrics.enabled true GLASSFLOW_OTEL_METRICS_ENABLED
global.observability.logs.enabled false GLASSFLOW_OTEL_LOGS_ENABLED

Related

Adds a run-data-migration init container that runs after run-migration
and before the API starts. It uses the API image with -role=migrate-data
to run the versioned data migration registry, which backfills schema_versions,
transformation_configs, join_configs, and sink_configs for pre-v3 pipelines.
@sudopower
sudopower requested a review from Nuzhnov May 7, 2026 08:02
@sudopower sudopower changed the title feat(otlp-receiver): expose maxConcurrentRequests and natsChunkSize feat(otlp-receiver): expose tuning knobs and fix spurious collector connection errors May 7, 2026
@sudopower
sudopower requested a review from ashish-bagri May 7, 2026 08:08
@sudopower sudopower changed the title feat(otlp-receiver): expose tuning knobs and fix spurious collector connection errors feat(otlp-receiver): Otel receiver config value defaults and missing env vars May 7, 2026
feat: add run-data-migration init container for pipeline v2→v3 promotion (ETL-676)
@sudopower
sudopower merged commit 7a3cd03 into main May 7, 2026
1 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants