feat: [6659773700] relax window query SELECT list restrictions

[Simon9997]

Description

Issue(s)

• Close/close/Fix/fix/Resolve/resolve: Issue Link

Checklist

Please check the items in the checklist if applicable.

• Is the user manual updated?
• Are the test cases passed and automated?
• Is there no significant decrease in test coverage?

[gemini-code-assist]

Code Review

This pull request introduces a new window projection mode to the query engine, allowing for more flexible handling of scalar expressions and projections in windowed queries. It adds the necessary infrastructure, including new enum types and AST/planner updates, to support this mode. My review identified a potential logic error where auxiliary columns (pseudo-columns and group keys) required for HAVING or ORDER BY clauses might be missed when the projection mode is active, as well as a concern regarding the robustness of the logic used to identify degenerate projection lists in subqueries.

[Copilot]

Pull request overview

This PR expands TDengine’s window-query capabilities by relaxing SELECT-list restrictions for window queries (especially “projection-mode” windows without aggregates), and updates the planner/executor to support the new behavior with additional CI coverage.

Changes:

• Add projection-mode support for window queries (raw columns / scalar expressions without aggregates) across planner splitting, logic planning, translation checks, and executor indefinite-rows runtime.
• Introduce SCALAR / AGG window-mode selectors in the SQL grammar and tokenizer.
• Add a comprehensive new window-projection in/ans test suite and update existing tests that previously expected syntax errors.

Reviewed changes

Copilot reviewed 40 out of 47 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
test/ci/cases.task	Add CI entry to run the new window-projection pytest suite.
test/cases/query/test_window_projection.py	New pytest suite building datasets and running negative + in/ans validations for window projection.
test/cases/query/in/test_window_projection_small.in	New small dataset query set for in/ans comparison (includes many window projection scenarios).
test/cases/query/in/test_window_projection_mode.in	New coverage for SCALAR/AGG mode variants in window projection queries.
test/cases/query/in/test_window_projection_large.in	New multi-vgroup in/ans workload for window projection queries.
test/cases/query/in/test_window_projection_largedata.in	New cross-block (large row count) in/ans workload for window projection.
test/cases/query/in/test_window_projection_partition.in	New partition-by focused in/ans workload for window projection.
test/cases/query/in/test_window_projection_edge.in	New empty/single-row edge-case in/ans workload.
test/cases/query/ans/test_window_projection_small.ans	Expected output for the small in/ans workload.
test/cases/query/ans/test_window_projection_mode.ans	Expected output for the mode-variation workload.
test/cases/query/ans/test_window_projection_large.ans	Expected output for the large in/ans workload.
test/cases/query/ans/test_window_projection_largedata.ans	Expected output for the cross-block workload.
test/cases/query/ans/test_window_projection_partition.ans	Expected output for the partition workload.
test/cases/query/ans/test_window_projection_edge.ans	Expected output for the edge-case workload.
test/cases/13-TimeSeriesExt/06-SessionWindow/test_session.py	Update assertions: some session window queries previously expected to error are now expected to succeed.
test/cases/13-TimeSeriesExt/04-StateWindow/test_state_window.py	Update assertion: a state_window query previously expected to error is now expected to succeed.
test/cases/11-Functions/01-Scalar/test_fun_sca_upper.py	Remove scalar+interval query from the “should error” list (behavior change).
test/cases/11-Functions/01-Scalar/test_fun_sca_substr.py	Remove scalar+interval query from the “should error” list (behavior change).
test/cases/11-Functions/01-Scalar/test_fun_sca_rtrim.py	Remove scalar+interval query from the “should error” list (behavior change).
test/cases/11-Functions/01-Scalar/test_fun_sca_ltrim.py	Remove scalar+interval query from the “should error” list (behavior change).
test/cases/11-Functions/01-Scalar/test_fun_sca_lower.py	Remove scalar+interval query from the “should error” list (behavior change).
test/cases/11-Functions/01-Scalar/test_fun_sca_length.py	Remove scalar+interval query from the “should error” list (behavior change).
test/cases/11-Functions/01-Scalar/test_fun_sca_concat2.py	Remove scalar+interval query from the “should error” list (behavior change).
test/cases/11-Functions/01-Scalar/test_fun_sca_concat.py	Remove scalar+interval query from the “should error” list (behavior change).
test/cases/11-Functions/01-Scalar/test_fun_sca_concat_ws.py	Remove scalar+interval query from the “should error” list (behavior change).
test/cases/11-Functions/01-Scalar/test_fun_sca_char_length.py	Remove scalar+interval query from the “should error” list (behavior change).
test/cases/10-Operators/01-Arithmetic/test_arithmetic.py	Convert previously-invalid interval arithmetic queries into validated successful queries.
test/cases/09-DataQuerying/14-Tags/test_tag_basic.py	Convert some interval window tag projection queries from expected-error to expected-success.
test/cases/09-DataQuerying/10-Distinct/test_query_distinct.py	Add positive coverage for DISTINCT ... INTERVAL(...) cases and remove them from the error list.
source/libs/planner/src/planSpliter.c	Adjust stable split strategy for interval/external windows when running in projection-mode.
source/libs/planner/src/planLogicCreater.c	Build window logic targets from projections for projection-mode; adjust handling for HAVING/slot binding.
source/libs/parser/src/parTranslater.c	Detect window projection-mode and relax/adjust group-by/expr rewriting; enforce restricted FILL modes for projection-mode windows.
source/libs/parser/src/parTokenizer.c	Add AGG and SCALAR to the SQL keyword table.
source/libs/parser/src/parAstCreater.c	Add AST helper to set select-statement window mode.
source/libs/parser/inc/sql.y	Extend SELECT grammar to accept optional SCALAR/AGG window-mode selector.
source/libs/parser/inc/parAst.h	Declare setSelectStmtWindowMode.
source/libs/executor/src/timewindowoperator.c	Pass projection list into indefinite-rows runtime initialization for window operators.
source/libs/executor/src/executorInt.c	Extend indefinite-rows runtime to support a projection expression path (projSupp) and fill pseudo-columns for it.
source/libs/executor/src/eventwindowoperator.c	Pass projection list into indefinite-rows runtime initialization for event windows.
source/libs/executor/src/countwindowoperator.c	Pass projection list into indefinite-rows runtime initialization for count windows.
source/libs/executor/inc/executorInt.h	Extend SIndefRowsRuntime and update initIndefRowsRuntime signature.
include/libs/nodes/querynodes.h	Add EWindowMode, store windowMode, and add hasScalarExpr flag to SSelectStmt.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 40 out of 47 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 48 out of 55 changed files in this pull request and generated 3 comments.

[Copilot]

Pull request overview

Copilot reviewed 49 out of 56 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 49 out of 56 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 49 out of 56 changed files in this pull request and generated 4 comments.

[dapan1121]

Code Review — window projection mode (SCALAR/AGG keywords)

Reviewed by GitHub Copilot (tsdb-code-review skill v0.1.0)

Summary

This PR introduces window projection mode (v3.4.2.0): window queries may now include plain column expressions in the SELECT list, causing each window to output all original rows instead of one aggregated row. Two new SQL keywords SCALAR/AGG disambiguate mode when only pseudocolumns/tags/constants are selected.

5 verified issues found (2 BLOCKER, 3 MAJOR):

---

🔴 BLOCKER

1. O(N²) complexity — checkWindowProjectionMode called per node in tree walk
source/libs/parser/src/parTranslater.c line 5413

checkWindowProjectionMode is called inside doCheckExprForGroupBy (a nodesRewriteExprs tree-walk callback, invoked O(T) times), and itself runs nodesWalkExprs(pSelect->pProjectionList, …) — another O(T) walk. For a SELECT tree of total size T, the group-by semantic check phase now costs O(T²).

Fix: Compute isScalarMode once before the walk and pass it via STranslateContext, or cache it in SSelectStmt. Do not compute it inside a per-node callback.

---

2. SCALAR/AGG accepted silently on queries without a window clause
source/libs/parser/src/parAstCreater.c line 2770

There is no validation that windowMode != WINDOW_MODE_NONE requires a window clause to be present. SELECT SCALAR c1 FROM t1 WHERE ts > 0 parses and translates without error. If doCheckExprForGroupBy is subsequently invoked for that query (e.g., triggered by ORDER BY context), checkWindowProjectionMode returns true (WINDOW_MODE_SCALAR, no agg/groupby/indefrows, ctx.hasScalarExpr = true), causing all partition-key and tag-column rewriting to be silently skipped and producing an incorrect query plan.

Fix:

// In translateSelectFrom or a dedicated checkWindowMode():
if (pSelect->windowMode != WINDOW_MODE_NONE && pSelect->pWindow == NULL) {
    return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_OPTR_USAGE,
        "SCALAR/AGG can only be used with a window clause");
}

Also add || pSelect->pWindow == NULL to the early-return guard in checkWindowProjectionMode.

---

🟠 MAJOR

3. Potential memory leak of pProjExprInfo in initIndefRowsRuntime error path
source/libs/executor/src/executorInt.c line 1337–1340

createExprInfo allocates pProjExprInfo. If initExprSupp fails before it stores the pointer in pRuntime->projSupp.pExprInfo, the _return path's cleanupExprSupp(&pRuntime->projSupp) will not free it (because projSupp.pExprInfo is still NULL).

Fix: Transfer ownership immediately after allocation so cleanupExprSupp always handles it:

code = createExprInfo(pProjs, NULL, &pProjExprInfo, &numProjs);
QUERY_CHECK_CODE(code, lino, _return);
pRuntime->projSupp.pExprInfo = pProjExprInfo;  // transfer before initExprSupp
pProjExprInfo = NULL;
code = initExprSupp(&pRuntime->projSupp, pRuntime->projSupp.pExprInfo, numProjs, pFuncStore);
QUERY_CHECK_CODE(code, lino, _return);

Or verify that initExprSupp always stores the pointer before any failure point.

---

4. INTERVAL projection mode routes to stbSplSplitSessionOrStateForBatch — confirm correctness
source/libs/planner/src/planSpliter.c line ~935

In stbSplSplitWindowForCrossTable, INTERVAL windows in projection mode (!pWin->pFuncs && pWin->pProjs) are dispatched to stbSplSplitSessionOrStateForBatch, a function written for SESSION/STATE windows. These have different partitioning semantics from INTERVAL (no time-alignment, no cross-vnode merge). Using it for multi-vnode INTERVAL projection queries may produce incorrect data distribution or output ordering.

Fix: Add a multi-vnode integration test: SELECT _wstart, ts, c1 FROM stable PARTITION BY tbname INTERVAL(10s). If results are correct, add a comment explaining why this split strategy is intentionally reused here. If not, introduce a dedicated split path.

---

5. hasScalarExpr and checkWindowProjectionMode can diverge

There are two independent mode-detection mechanisms:

• checkWindowProjectionMode re-walks pProjectionList looking for COLUMN_TYPE_COLUMN nodes
• pSelect->hasScalarExpr is set in doCheckExprForGroupBy under !hasSelectFunc && !isRelatedToOtherExpr

These conditions are not equivalent and can diverge (e.g., a column that isRelatedToOtherExpr would be seen by checkWindowProjectionMode but not set hasScalarExpr). Downstream code in planLogicCreater.c gates pWindow->indefRowsFunc and projectionMode on hasScalarExpr, while the translator skip logic uses checkWindowProjectionMode. A mismatch between the two means the planner and the translator disagree on the query's mode.

Fix: Use a single cached field (e.g. bool windowScalarMode in SSelectStmt) as the sole source of truth, set once early in translateSelectFrom, and used by both the translator and the planner.

---

🟡 MINOR

• line vs lino naming — fillIndefRowsWindowPseudoCols uses line instead of lino (codebase convention, executorInt.c line 1297)
• Removed negative test without comment — parSelectTest.cpp line 337 drops run("SELECT c1 FROM t1 INTERVAL(10s)", TSDB_CODE_PAR_INVALID_OPTR_USAGE) without explanation; add a comment noting it is now valid in projection mode
• Missing comment on FILL restriction scope — the FILL mode restriction check only handles INTERVAL_WINDOW; other window types silently pass without a note explaining that they cannot carry a FILL node syntactically

[Copilot]

Pull request overview

Copilot reviewed 50 out of 57 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 52 out of 59 changed files in this pull request and generated 2 comments.