[Spark] Only attempt server-side planning for Delta tables in loadTable#7039
Open
chenwang-databricks wants to merge 2 commits into
Open
[Spark] Only attempt server-side planning for Delta tables in loadTable#7039chenwang-databricks wants to merge 2 commits into
chenwang-databricks wants to merge 2 commits into
Conversation
AbstractDeltaCatalog.loadTable called ServerSidePlannedTable.tryCreate on every table returned by super.loadTable and returned the SSP table early when one was produced. That runs before the `case o => o` fallthrough, so a non-Delta table or a catalog-specific shape that is meant to pass through unchanged -- e.g. Unity Catalog's MetadataTable wrapping a ViewInfo for a metric view -- can be captured by SSP and short-circuit view loading. Move the tryCreate attempt inside the `case v1: V1Table if isDeltaTable` branch so SSP applies only to Delta tables; everything else flows through `case o => o` unchanged. Signed-off-by: Chen Wang <chen.wang@databricks.com>
9 tasks
murali-db
approved these changes
Jun 18, 2026
SSP now only wraps Delta tables in loadTable, but the suite's fixtures were created USING parquet, so they fell through the non-Delta path and were never wrapped as ServerSidePlannedTable. Switch both fixtures to USING delta so they route through the Delta branch and SSP engages as the suite expects. Co-authored-by: Isaac
yili-db
approved these changes
Jun 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
AbstractDeltaCatalog.loadTablecalledServerSidePlannedTable.tryCreateon every table returned bysuper.loadTableand returned the resulting SSP table early, before thecase o => ofallthrough:Server-side planning only applies to Delta tables, but running
tryCreateahead of the match means it also sees non-Delta tables and catalog-specific shapes that are meant to pass through unchanged. In particular, whenDeltaCatalogdelegates to a Unity Catalog catalog, a metric view comes back as aMetadataTablewrapping aViewInfo;tryCreatecan capture that and short-circuit view loading instead of letting it flow throughcase o => oto the resolver.This moves the
tryCreateattempt inside the Delta-V1Tablebranch so SSP is only attempted for Delta tables; everything else (non-Delta tables, view-shaped relations) flows throughcase o => ounchanged:Behavior for Delta tables is unchanged (SSP still attempted, falling back to
loadCatalogTable); only non-Delta / view shapes are no longer captured by SSP.How was this patch tested?
build/sbt spark/compile. The change is a localized, type-preserving transformation of existingloadTablelogic (tryCreatereturnsOption[ServerSidePlannedTable];getOrElse(loadCatalogTable(...): Table)widens toTable). No behavior change for Delta tables; the affected path is the catalog-delegation case where a non-Delta/view relation must pass through unchanged.Does this PR introduce any user-facing changes?
No.