[FLINK-39720][table] SubQueryDecorrelator produces incorrect plans for correlated EXISTS with HAVING on aggregate outputs#28217
Open
lincoln-lil wants to merge 1 commit into
Open
Conversation
…r correlated EXISTS with HAVING on aggregate outputs
Collaborator
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What is the purpose of the change
This pull request fixes a silent wrong-result bug in SubQueryDecorrelator: correlated EXISTS / NOT EXISTS sub-queries that combine a correlated WHERE with a HAVING
clause on aggregate output produced plans where the HAVING predicate pointed at the wrong column.
When decorrelateRel(LogicalAggregate) rewrites the inner aggregate, it injects the correlated columns into the group key, which shifts the position of the original
aggregate-output fields. The Frame records this shift in oldToNewOutputs. But decorrelateRel(LogicalFilter) then reattaches the non-correlated remainder of the
Filter condition (the HAVING clause, for example SUM(r.e) >= 3) to the rewritten input without remapping its RexInputRefs through frame.oldToNewOutputs. Every
other decorrelateRel(...) method in the same file (Project, Aggregate, Join, Correlate) already calls adjustInputRefs on surviving expressions before reattaching
them — the Filter path was the only one missing the call.
The result is a structurally valid but semantically wrong plan: HAVING SUM(r.e) >= 3 ends up applied to the injected correlated column (e.g. r.d) instead of the
SUM(r.e) output. This restores symmetry with Calcite's bottom-up RelDecorrelator.decorrelateRel(Filter), which walks every RexInputRef through oldToNewOutputs in a
single pass.
Brief change log
SubQueryDecorrelator.decorrelateRel(LogicalFilter): re-index remainingCondition (the non-correlated remainder built from nonCorConditions) through
adjustInputRefs(remainingCondition, frame.oldToNewOutputs, frame.r.getRowType()) between RexUtil.composeConjunction(...) and LogicalFilter.create(...). The
defensive null guard mirrors surrounding style; composeConjunction(..., /nullOnEmpty=/false) returns TRUE for an empty list, and adjustInputRefs(TRUE, ...) is a
no-op, so the all-correlated case is unaffected.
Verifying this change
SubQuerySemiJoinTest — five new test cases
Does this pull request potentially affect one of the following parts:
Documentation