Skip to content
Open
11 changes: 11 additions & 0 deletions .markdownlint.jsonc
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{
"MD004": { "style": "dash" },
"MD010": { "code_blocks": false },
"MD013": {
"line_length": 80,
"code_block_line_length": 120,
"tables": false,
"headings": false
},
"MD029": { "style": "ordered" }
}
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# Changelog

## Unreleased

### Breaking changes

- Reject legacy DDlog tokens that previously had inconsistent parser treatment:
`typedef`, `bigint`, `bit`, `double`, `float`, `signed`, bare `#`, and `<=>`.
Use `type`, sized integer types such as `u32`/`i64`, `f32`/`f64`, and
`#[...]` attributes instead.
4 changes: 4 additions & 0 deletions docs/contents.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@

## Core project documents

- [Changelog](../CHANGELOG.md): User-facing release notes and migration
highlights.
- [Repository layout](./repository-layout.md): Map of the repository tree,
path responsibilities, and placement rules for contributors.
- [Users' guide](./users-guide.md): User-visible syntax support and parser
Expand Down Expand Up @@ -82,6 +84,8 @@
Plan for transformer declaration grammar alignment.
- [2.6.6 resolve relation form coverage](./execplans/2-6-6-resolve-relation-form-coverage.md):
Plan for relation form coverage and modelling alignment.
- [2.6.7 finalize legacy token compatibility policy](./execplans/2-6-7-finalize-legacy-token-compatibility-policy.md):
Plan for closing the legacy-token compatibility policy.
- [3.1.1 core rule and CST rule traits](./execplans/3-1-1-core-rule-and-cst-rule-traits.md):
Plan for lint rule trait foundations.
- [3.1.2 rule context struct](./execplans/3-1-2-rule-context-struct.md):
Expand Down
5 changes: 4 additions & 1 deletion docs/ddlint-design.md
Original file line number Diff line number Diff line change
Expand Up @@ -233,7 +233,10 @@ the syntax-layer contract before crate extraction: the parser now accepts only
the canonical spec-form `index Name(field: Type, ...) on Atom`, rejects the
older shorthand `index Name on Relation(columns)` with a targeted diagnostic,
and exposes the typed field list plus normalized `on` target directly from the
CST-backed `Index` wrapper.
CST-backed `Index` wrapper. Legacy compatibility tokens follow the same
parser-contract rule: `src/parser/reserved_tokens.rs` single-sources
diagnostics for tokens that stay lexed for span precision but have no supported
grammar semantics.

Relation declarations use the same CST-backed boundary. The `Relation` wrapper
models the declaration preamble with `RelationRole` (`Input`, `Output`, or
Expand Down
5 changes: 5 additions & 0 deletions docs/developers-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,11 @@ parsing pipeline.
(`D-REL-001` through `D-REL-003`) in `relations/preamble.rs`.
- Keep typed consumer-facing relation metadata in `ast/relation.rs`, and
inspection-only CST traversal in `ast/relation/inspect.rs`.
- Route reserved-token diagnostics through `src/parser/reserved_tokens.rs`.
That module owns the parser-internal messages and the `rejection_for`
predicate for unsupported legacy tokens. The canonical public policy table
lives in `docs/differential-datalog-parser-syntax-spec-updated.md` section
`9.1`; avoid duplicating it in code comments or local scanner modules.

## Contributor workflow

Expand Down
85 changes: 72 additions & 13 deletions docs/differential-datalog-parser-syntax-spec-updated.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,19 +90,26 @@ ______________________________________________________________________
function calls at parse time. A bare `name(…)` parses as a variable application
and is disambiguated later during name resolution.

These case classes describe the intended naming categories. The current parser
uses the generic `T_IDENT` token for identifiers and does not enforce the
initial-case distinction in every context.

### 2.3 Reserved words and symbols

The following **keywords** and **reserved operators** cannot be used as
identifiers (final list should be kept 1:1 with the lexer):

- **Keywords:** `type`, `function`, `extern`, `transformer`, `input`, `output`,
`relation`,`stream`,`multiset`,`index`,`on`,`primary`,`key`,`apply`,`match`,`if`,`else`,`for`,`in`,`then`,`skip`,`true`,`false`,`var`,`mut`,`return`,`break`,`continue`.
`relation`, `stream`, `multiset`, `index`, `on`, `primary`, `key`, `apply`,
`match`, `if`, `else`, `for`, `in`, `then`, `skip`, `true`, `false`, `var`,
`mut`, `return`, `break`, `continue`, and `as`.
- **Special tokens:** `@`, `:-`, `,`, `;`, `:`, `::`, `.`, `&`, `'` (diff
marker), `-<` (delay introducer), `=>` (implies), brackets and braces
`()[]{}`.
`()[]{}`, and `#` when it starts an attribute prefix `#[...]`.

Reserved but not part of the grammar: `#`, `<=>`. Implementations must reject
their use with a clear diagnostic.
Reserved but not part of the grammar: `<=>` and bare `#` tokens not followed by
`[` as an attribute prefix. Implementations must reject their use with a clear
diagnostic.

#### 2.3.1 Host‑language keyword reservation

Expand Down Expand Up @@ -184,7 +191,10 @@ intentional. -->

**Note:** `++` (concatenation) and `^` (bit‑xor) are part of the operator table
and are recognized as operators. `&` in row 13 is expression-only; head
semantics are described in §7.3.
semantics are described in §7.3. `:` is the implemented expression
type-ascription operator, and `as` is the implemented expression cast operator;
both use the type-operator binding level between shifts and bitwise operators.
`as` is also the import alias keyword.

______________________________________________________________________

Expand All @@ -209,7 +219,7 @@ Attribute ::= '#[' AttrBody ']'
### 5.2 Imports and types

```ebnf
Import ::= 'import' ScopedPath ';'
Import ::= 'import' ScopedPath ('as' LcName)? ';'
Typedef ::= 'type' UcName TypeParams? '=' Type ';'
Type ::= UcName TypeArgs? | TupleType | MapType | VecType | Primitive

Expand All @@ -220,6 +230,10 @@ Primitive ::= 'bool' | 'i8' | 'u8' | 'i16' | … | 'u128'
| 'f32' | 'f64' | 'string' | 'interned'
```

The import alias uses the documented `LcName` category. The current parser
accepts the generic identifier token here and does not enforce that case
restriction.

### 5.3 Functions and closures

```ebnf
Expand Down Expand Up @@ -553,19 +567,64 @@ ______________________________________________________________________
## 9.1 Legacy and compatibility tokens

Implementations may encounter historical tokens from older DDlog parsers. This
spec defines their treatment to aid migration:
spec defines their treatment to aid migration. The lexer keeps the token kinds
so diagnostics can point at the exact source span; unsupported uses are
rejected by the parser.

- `Aggregate(…)`: accepted and normalized during rule-body semantic
extraction to the same canonical `(project, key)` aggregation contract used
for `group_by(project, key)`; linters may emit a deprecation diagnostic.
- `FlatMap`/`Inspect`: not language keywords; represent flatmap via RHS pattern
binds instead. If used as keywords, reject with a targeted message.
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Outdated
- `typedef`: not supported; use `type` definitions. Emit an error with a fix
hint.
- Legacy type names such as `bigint`, `bit`, `double`, `float`, `signed`:
not in the grammar. Use sized integer types (`iN`/`uN`), and `f32`/`f64` for
floating‑point.
- `as`: not a keyword in the updated grammar; reject its use as a keyword.
- `typedef`: rejected.

```plaintext
`typedef` is a legacy DDlog keyword; use `type` instead
```

- `as`: accepted as the import alias keyword and the implemented expression
cast operator; `:` is the implemented expression type-ascription operator.
- `bigint`: rejected.

```plaintext
`bigint` is a legacy type name; use a sized integer such as `i64` or `u64`
```

- `bit`: rejected.

```plaintext
`bit` is a legacy type name; use an unsigned sized integer such as `u32`
```

- `double`: rejected.

```plaintext
`double` is a legacy type name; use `f64`
```

- `float`: rejected.

```plaintext
`float` is a legacy type name; use `f32`
```

- `signed`: rejected.

```plaintext
`signed` is a legacy type name; use a signed sized integer such as `i32`
```

- `#`: accepted only as `#[...]`; bare uses are rejected.

```plaintext
`#` is reserved; only `#[...]` attribute syntax is accepted
```

- `<=>`: rejected.

```plaintext
`<=>` was reserved upstream but has no semantics in DDlog; remove it
```

Rationale and resolution status for the aggregation boundary:

Expand Down
Loading
Loading