Skip to content

[flink] Include lake reader in source metrics and correct max timestamp lag semantics#3356

Open
zuston wants to merge 1 commit into
apache:mainfrom
zuston:lakefetch
Open

[flink] Include lake reader in source metrics and correct max timestamp lag semantics#3356
zuston wants to merge 1 commit into
apache:mainfrom
zuston:lakefetch

Conversation

@zuston
Copy link
Copy Markdown
Member

@zuston zuston commented May 21, 2026

Purpose

Linked issue: close #3304

Brief change log

  1. Add the lake reader into the source metrics, this is necessary for the long-term backfill when using the fluss multi-layer log table as the source
  2. Correct the max timestamp log semantics on subscribing the multi buckets or partitions. that is mentioned in [flink] currentFetchEventTimeLag is misleading when subscribing multiple partitions/buckets with uneven lag #3304
  3. Make these metrics collecting logic into one place in FlinkRecordsWithSplitIds

Tests

API and Format

Documentation

@zuston
Copy link
Copy Markdown
Member Author

zuston commented May 21, 2026

cc @swuferhong @luoyuxia

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates Fluss’s Flink source metrics to (1) include lake-reader-produced records in event-time lag reporting by preserving LogRecord metadata through bounded scanning, and (2) correct currentFetchEventTimeLag semantics when consuming multiple buckets/partitions by tracking lag per table-bucket and aggregating at the reader level. It also centralizes metrics reporting logic in FlinkRecordsWithSplitIds.

Changes:

  • Preserve lake LogRecord metadata (offset/timestamp/change type) through bounded readers via a new LogRecordRowIterator adapter.
  • Move fetch event-time lag reporting into FlinkRecordsWithSplitIds and add per-table-bucket currentFetchEventTimeLag gauges.
  • Add/extend unit tests for per-split metrics reporting and lake metadata preservation.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
fluss-flink/fluss-flink-common/src/test/java/org/apache/fluss/flink/source/reader/FlinkRecordsWithSplitIdsTest.java Adds unit coverage verifying event-time lag reporting from FlinkRecordsWithSplitIds.
fluss-flink/fluss-flink-common/src/test/java/org/apache/fluss/flink/source/reader/BoundedSplitReaderTest.java Adds coverage ensuring lake log record metadata is preserved through bounded scanning.
fluss-flink/fluss-flink-common/src/test/java/org/apache/fluss/flink/source/metrics/FlinkSourceReaderMetricsTest.java Adds coverage for currentFetchEventTimeLag aggregation behavior and per-bucket gauges.
fluss-flink/fluss-flink-common/src/main/java/org/apache/fluss/flink/source/reader/LogRecordRowIterator.java Introduces an iterator contract to expose LogRecord metadata while still iterating InternalRow.
fluss-flink/fluss-flink-common/src/main/java/org/apache/fluss/flink/source/reader/FlinkSourceSplitReader.java Removes fetch-level lag calculation/reporting (now handled downstream).
fluss-flink/fluss-flink-common/src/main/java/org/apache/fluss/flink/source/reader/FlinkRecordsWithSplitIds.java Reports per-record timestamp to metrics on record emission.
fluss-flink/fluss-flink-common/src/main/java/org/apache/fluss/flink/source/reader/BoundedSplitReader.java Detects LogRecordRowIterator to build ScanRecord with metadata when available.
fluss-flink/fluss-flink-common/src/main/java/org/apache/fluss/flink/source/metrics/FlinkSourceReaderMetrics.java Adds per-bucket lag tracking and a reader-level aggregated CURRENT_FETCH_EVENT_TIME_LAG gauge.
fluss-flink/fluss-flink-common/src/main/java/org/apache/fluss/flink/lake/reader/LakeSnapshotScanner.java Makes the lake snapshot row iterator expose LogRecord metadata via LogRecordRowIterator.
fluss-flink/fluss-flink-common/src/main/java/org/apache/fluss/flink/lake/reader/IndexedLakeSplitRecordIterator.java Exposes LogRecord metadata via LogRecordRowIterator instead of row-only iteration.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +85 to 91
if (lag > maxFetchEventTimeLag) {
if (maxFetchEventTimeLag == UNINITIALIZED) {
sourceReaderMetricGroup.gauge(
MetricNames.CURRENT_FETCH_EVENT_TIME_LAG, () -> maxFetchEventTimeLag);
}
maxFetchEventTimeLag = lag;
}
Comment on lines +110 to +120
assertThat((long) readerEventTimeLagGauge.get().getValue())
.isEqualTo(maxReaderEventTimeLag);
assertThat((long) bucket1EventTimeLagGauge.get().getValue())
.isLessThan(maxReaderEventTimeLag);

long updatedBucket0Timestamp = newerTimestamp - 50000L;
flinkSourceReaderMetrics.reportRecordEventTime(tableBucket0, updatedBucket0Timestamp);
assertThat((long) readerEventTimeLagGauge.get().getValue())
.isEqualTo(maxReaderEventTimeLag);
assertThat((long) bucket0EventTimeLagGauge.get().getValue())
.isLessThan(maxReaderEventTimeLag);
Comment on lines +68 to +70
private volatile long maxFetchEventTimeLag = UNINITIALIZED;
// Map for tracking current fetch event time lag by table bucket
private final Map<TableBucket, Long> currentFetchEventTimeLags = new HashMap<>();
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[flink] currentFetchEventTimeLag is misleading when subscribing multiple partitions/buckets with uneven lag

2 participants