Skip to content

fix(trino): catch up with SQLFluff and stop panicking on common statements - #3046

Open
dkxmercury wants to merge 1 commit into
quarylabs:mainfrom
dkxmercury:fix/trino-catch-up-sqlfluff
Open

fix(trino): catch up with SQLFluff and stop panicking on common statements#3046
dkxmercury wants to merge 1 commit into
quarylabs:mainfrom
dkxmercury:fix/trino-catch-up-sqlfluff

Conversation

@dkxmercury

Copy link
Copy Markdown
Contributor

Fixes #3017.

Summary

Implements the four SQLFluff commits you listed, so the Trino dialect stops panicking on everyday statements and picks up the upstream grammar it had missed.

Before this, parsing any of these panicked instead of producing a tree or a clean error:

create table              -> Grammar refers to 'TRANSIENT' which was not found in the dialect
insert into t values (1)  -> Grammar refers to 'OVERWRITE' which was not found in the dialect
alter table t             -> Grammar refers to 'SEQUENCE' which was not found in the dialect

What was ported

  • c6b39aae TemporaryTransientGrammar becomes Nothing, which removes the inherited TRANSIENT reference. PrimaryKeyGrammar, ForeignKeyGrammar, UniqueKeyGrammar and TemporaryGrammar are silenced the same way, matching upstream.
  • d36f2f06 an explicit Trino StatementSegment listing only the statements Trino supports, plus Trino CREATE TABLE, ColumnDefinitionSegment and TransactionStatementSegment. Also adds the FUNCTION keyword, which that commit added to the Trino keyword list.
  • 74aaf399 Trino INSERT without OVERWRITE, and COMMIT, ROLLBACK and SET SESSION.
  • 07db73d1 Trino ALTER TABLE, covering RENAME, ADD/DROP/RENAME/ALTER COLUMN, SET AUTHORIZATION, SET PROPERTIES and EXECUTE, without unsupported inherited options.

Listing the supported statements explicitly is the part that actually closes off the panics, since unsupported ANSI branches such as sequences are no longer reachable and cannot resolve a keyword Trino does not define.

One addition beyond a direct port: ALTER TABLE ... EXECUTE f(x => 1) needs => to lex as a single token, so the dialect now inserts a fat_right_arrow lexer matcher alongside the existing right_arrow, mirroring StringLexer("fat_right_arrow", "=>", ...) upstream. Without it the argument list is unparsable.

Testing

  • The three panics are gone. create table, create table t (a integer), insert into t values (1), insert into t (a) select 1, alter table t rename to u, alter table t add column c integer, alter table t execute f(x => 1), commit, rollback, start transaction isolation level serializable and set session foo = 1 all parse.
  • Added the upstream fixtures for CREATE TABLE, ALTER TABLE, INSERT, COMMIT, ROLLBACK, START TRANSACTION and SET SESSION, with generated yml snapshots. All parse with no unparsable segments.
  • Full cargo test -p sqruff-lib-dialects --test dialects run is clean, and I ran it twice before and after the change to be sure the result was stable. cargo test -p sqruff-lib-core is green, cargo fmt --check and cargo clippy -p sqruff-lib-dialects --tests -- -Dwarnings are clean.

One local note in case it helps anyone else on Windows: the fixture tests are sensitive to line endings, and with core.autocrlf=true a set of unrelated fixtures with multi line string bodies fails and the failing set varies between runs. With an LF checkout the suite is stable and green, which is what I used for the comparison above.

…ments

Trino is built from the ANSI dialect and then clears its keyword sets, but
the inherited ANSI grammar still referenced keywords Trino no longer defines.
Parsing hit the None arm of Dialect::ref and panicked on everyday SQL:

  create table              -> Grammar refers to 'TRANSIENT'
  insert into t values (1)  -> Grammar refers to 'OVERWRITE'
  alter table t             -> Grammar refers to 'SEQUENCE'

Ports the Trino changes from the SQLFluff commits listed in the issue:

  c6b39aae  TemporaryTransientGrammar becomes Nothing
  d36f2f06  explicit Trino StatementSegment, CREATE TABLE, column and
            transaction grammars, and the FUNCTION keyword
  74aaf399  Trino INSERT without OVERWRITE, plus COMMIT, ROLLBACK and
            SET SESSION
  07db73d1  Trino ALTER TABLE without unsupported inherited options

Listing the supported statements explicitly is what keeps unsupported ANSI
branches such as sequences out of the dialect, so a missing keyword can no
longer be reached. Also adds a lexer matcher for `=>` so ALTER TABLE EXECUTE
arguments lex as one token.

Fixtures for CREATE TABLE, ALTER TABLE, INSERT, COMMIT, ROLLBACK,
START TRANSACTION and SET SESSION come from the same upstream commits.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Parser panics on CREATE TABLE / INSERT in the Trino dialect (grammar refs to keywords Trino removed)

1 participant