Retain table and column comments in CREATE OR REPLACE TABLE#7035
Open
geraintcjy wants to merge 1 commit into
Open
Retain table and column comments in CREATE OR REPLACE TABLE#7035geraintcjy wants to merge 1 commit into
geraintcjy wants to merge 1 commit into
Conversation
geraintcjy
force-pushed
the
xta-16662-retain-comments-cort
branch
from
85e9755 to
d6033f3
Compare
If enabled via a config flag, CREATE OR REPLACE TABLE now carries over table-level and per-column comments from the previous table version when the replacement DDL does not explicitly specify them. The carry-over logic recurses into nested struct, array, and map fields.
geraintcjy
force-pushed
the
xta-16662-retain-comments-cort
branch
from
d6033f3 to
a8610a4
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
When a statement does not explicitly specify column or table comments, the existing comments from the table being replaced are now preserved.
Previously, replacing a table always discarded all comments, even when the replacement DDL made no mention of comments. This change introduces carry-over logic that matches column comments by case-insensitive name and recursively handles nested struct types.
A new configuration flag
spark.databricks.delta.retainCommentsDuringReplace(default:true) gates this behavior for safe rollout. If a comment is explicitly specified in the new DDL (even an empty string), that value takes precedence.Changes
replaceMetadataIfNecessary()inCreateDeltaTableCommandretainCommentsDuringReplaceinDeltaSQLConfTests
New and updated tests in
DeltaTableCreationTestscovering:REPLACE TABLE AS SELECT, nested struct commentsNoMapping,NameMapping,IdMapping)