Skip to content

[Spark] Only attempt server-side planning for Delta tables in loadTable#7039

Open
chenwang-databricks wants to merge 2 commits into
delta-io:masterfrom
chenwang-databricks:chenwang/ssp-skip-view-loadtable
Open

[Spark] Only attempt server-side planning for Delta tables in loadTable#7039
chenwang-databricks wants to merge 2 commits into
delta-io:masterfrom
chenwang-databricks:chenwang/ssp-skip-view-loadtable

Conversation

@chenwang-databricks

Copy link
Copy Markdown

Description

AbstractDeltaCatalog.loadTable called ServerSidePlannedTable.tryCreate on every table returned by super.loadTable and returned the resulting SSP table early, before the case o => o fallthrough:

val table = ... // super.loadTable(ident)
ServerSidePlannedTable.tryCreate(spark, ident, table, isUnityCatalog).foreach { sspt =>
  return sspt
}
table match {
  case v1: V1Table if DeltaTableUtils.isDeltaTable(v1.catalogTable) =>
    loadCatalogTable(ident, v1.catalogTable)
  case o => o
}

Server-side planning only applies to Delta tables, but running tryCreate ahead of the match means it also sees non-Delta tables and catalog-specific shapes that are meant to pass through unchanged. In particular, when DeltaCatalog delegates to a Unity Catalog catalog, a metric view comes back as a MetadataTable wrapping a ViewInfo; tryCreate can capture that and short-circuit view loading instead of letting it flow through case o => o to the resolver.

This moves the tryCreate attempt inside the Delta-V1Table branch so SSP is only attempted for Delta tables; everything else (non-Delta tables, view-shaped relations) flows through case o => o unchanged:

table match {
  case v1: V1Table if DeltaTableUtils.isDeltaTable(v1.catalogTable) =>
    ServerSidePlannedTable
      .tryCreate(spark, ident, table, isUnityCatalog)
      .getOrElse(loadCatalogTable(ident, v1.catalogTable))
  case o => o
}

Behavior for Delta tables is unchanged (SSP still attempted, falling back to loadCatalogTable); only non-Delta / view shapes are no longer captured by SSP.

How was this patch tested?

build/sbt spark/compile. The change is a localized, type-preserving transformation of existing loadTable logic (tryCreate returns Option[ServerSidePlannedTable]; getOrElse(loadCatalogTable(...): Table) widens to Table). No behavior change for Delta tables; the affected path is the catalog-delegation case where a non-Delta/view relation must pass through unchanged.

Does this PR introduce any user-facing changes?

No.

AbstractDeltaCatalog.loadTable called ServerSidePlannedTable.tryCreate on every
table returned by super.loadTable and returned the SSP table early when one was
produced. That runs before the `case o => o` fallthrough, so a non-Delta table
or a catalog-specific shape that is meant to pass through unchanged -- e.g. Unity
Catalog's MetadataTable wrapping a ViewInfo for a metric view -- can be captured
by SSP and short-circuit view loading.

Move the tryCreate attempt inside the `case v1: V1Table if isDeltaTable` branch
so SSP applies only to Delta tables; everything else flows through `case o => o`
unchanged.

Signed-off-by: Chen Wang <chen.wang@databricks.com>
SSP now only wraps Delta tables in loadTable, but the suite's fixtures
were created USING parquet, so they fell through the non-Delta path and
were never wrapped as ServerSidePlannedTable. Switch both fixtures to
USING delta so they route through the Delta branch and SSP engages as
the suite expects.

Co-authored-by: Isaac
@yili-db yili-db requested review from openinx and yili-db June 23, 2026 21:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants