Skip to content

feat(rust): support datafusion expressions for merge insert predicates#6934

Merged
wjones127 merged 3 commits into
lance-format:mainfrom
Ar-maan05:feat/merge-insert-expr-support
May 28, 2026
Merged

feat(rust): support datafusion expressions for merge insert predicates#6934
wjones127 merged 3 commits into
lance-format:mainfrom
Ar-maan05:feat/merge-insert-expr-support

Conversation

@Ar-maan05
Copy link
Copy Markdown
Contributor

@Ar-maan05 Ar-maan05 commented May 25, 2026

Description

This PR adds support for passing logical DataFusion expressions (datafusion_expr::Expr) directly to the merge insert query planner for conditional updates (WhenMatched::UpdateIfExpr). Currently, matched update predicates only accept SQL strings which are parsed at plan creation time, introducing serialize-deserialize round-trip overhead when callers build expressions programmatically.

Key Changes

  • WhenMatched Enum: Added the UpdateIfExpr(datafusion_expr::Expr) variant and the associated constructor helper WhenMatched::update_if_expr(expr: Expr).
  • Query Planner (merge_insert.rs): Updated plan generation to optimize the DataFusion logical expression and convert it directly to a physical expression.
  • Physical & Logical Layouts: Handled the new variant exhaustively in assign_action.rs (action determination), exec/write.rs (DisplayAs), and logical_plan.rs (Explain formatting).

Verification

  • Verified compile-safety and workspace formatting checks.
  • Verified via downstream integration tests in lancedb.

Closes #6861

Copy link
Copy Markdown

@claude claude Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@github-actions github-actions Bot added the enhancement New feature or request label May 25, 2026
@Ar-maan05 Ar-maan05 force-pushed the feat/merge-insert-expr-support branch from 84cb398 to 3a6e755 Compare May 26, 2026 17:47
@codecov
Copy link
Copy Markdown

codecov Bot commented May 26, 2026

Copy link
Copy Markdown
Contributor

@wjones127 wjones127 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks reasonable, but I have a minor change requested to deduplicate some logic.

Comment thread rust/lance/src/dataset/write/merge_insert/exec/write.rs
Comment thread rust/lance/src/dataset/write/merge_insert.rs
Comment thread rust/lance/src/dataset/write/merge_insert.rs Outdated
@Ar-maan05 Ar-maan05 requested a review from wjones127 May 27, 2026 03:16
Copy link
Copy Markdown
Contributor

@wjones127 wjones127 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Can merge once tests are passing.

@Ar-maan05
Copy link
Copy Markdown
Contributor Author

CI is green and ready on my end. Let me know if anything else is needed before merge.

@wjones127 wjones127 merged commit 1c06b3c into lance-format:main May 28, 2026
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feature: support datafusion expression for merge insert predicates

2 participants