[Spark] Migrate DeltaSinkSuite from path-based to name-based access#7081
Open
PorridgeSwim wants to merge 1 commit into
Open
[Spark] Migrate DeltaSinkSuite from path-based to name-based access#7081PorridgeSwim wants to merge 1 commit into
PorridgeSwim wants to merge 1 commit into
Conversation
murali-db
approved these changes
Jun 24, 2026
BrooksWalls
reviewed
Jun 24, 2026
| .trigger(org.apache.spark.sql.streaming.Trigger.Once) | ||
| .option("checkpointLocation", chkDir) | ||
| .start(tableDir) | ||
| withTable("test_delta_sink") { |
Collaborator
There was a problem hiding this comment.
why does this test use withTable and not withSinkTarget like the rest?
PorridgeSwim
force-pushed
the
nameBasedDeltaSinkSuite
branch
from
bb9fff5 to
5acab6c
Compare
BrooksWalls
approved these changes
Jun 24, 2026
zikangh
reviewed
Jun 24, 2026
| /** | ||
| * Run a sink test against a name-based (catalog) target table. | ||
| */ | ||
| protected def withSinkTarget(f: (String, File) => Unit): Unit = { |
Collaborator
There was a problem hiding this comment.
I'm concerned we are losing test coverage for path-based tables.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which Delta project/connector is this regarding?
Description
Test-only refactor that migrates
DeltaSinkSuitefrom path-based to name-based table access:writeStream...toTable(name)instead of...start(path)spark.read.table(name)/DeltaLog.forTable(spark, TableIdentifier(name))instead of...load(path)/forTable(spark, path)A new
withSinkTargethelper runs each migrated test against a uniquely-named managed table.Motivation (same as #7058 for the type-widening tests):
A few tests are intentionally left path-based because their behavior is intrinsically path-specific — e.g.
path not specified(assertsstart()with no path errors) andincompatible schema merging ...(relies onsave(path)'s schema-merge-and-reject; catalog writes resolve-and-cast instead, so the merge conflict never occurs).How was this patch tested?
Test-only change. The existing
DeltaSinkSuitetests continue to pass with name-based access.Does this PR introduce any user-facing changes?
No.