Apache Iceberg Rust version
None
Describe the bug
Context
When reading _last_updated_sequence_number, the Arrow reader gates on the data
file's first_row_id: if it is null, the whole column is nulled and any per-row
values the file physically carries are discarded (the physical leaf is never
projected).
Introduced in #2985 alongside the coalesce path.
Behavior
This matches Java (ValueReaders.lastUpdated returns nulls when the base row id
is null), so it is not an interop regression. But the spec does not forbid such a
file, so it is a divergence between what the bytes can express and what we return.
Flagged in review as worth tracking rather than fixing in the coalesce PR.
Options
- Keep the Java-parity behavior (current) ie a file with no row lineage has no
meaningful per-row sequence numbers.
- Pass non-null per-row values through, leaving null rows null when there is no
fallback. Note that this diverges from Java.
To Reproduce
No response
Expected behavior
No response
Willingness to contribute
I can contribute a fix for this bug independently
Apache Iceberg Rust version
None
Describe the bug
Context
When reading
_last_updated_sequence_number, the Arrow reader gates on the datafile's
first_row_id: if it is null, the whole column is nulled and any per-rowvalues the file physically carries are discarded (the physical leaf is never
projected).
Introduced in #2985 alongside the coalesce path.
Behavior
This matches Java (
ValueReaders.lastUpdatedreturns nulls when the base row idis null), so it is not an interop regression. But the spec does not forbid such a
file, so it is a divergence between what the bytes can express and what we return.
Flagged in review as worth tracking rather than fixing in the coalesce PR.
Options
meaningful per-row sequence numbers.
fallback. Note that this diverges from Java.
To Reproduce
No response
Expected behavior
No response
Willingness to contribute
I can contribute a fix for this bug independently