Skip to content

refactor: delegate generate_series/get_powers_of_two to dbt-core equivalents#1088

Open
sdebruyn wants to merge 1 commit into
dbt-labs:mainfrom
sdebruyn:delegate-generate-series-to-dbt-core
Open

refactor: delegate generate_series/get_powers_of_two to dbt-core equivalents#1088
sdebruyn wants to merge 1 commit into
dbt-labs:mainfrom
sdebruyn:delegate-generate-series-to-dbt-core

Conversation

@sdebruyn

Copy link
Copy Markdown
Contributor

Problem

dbt_utils.generate_series and dbt_utils.get_powers_of_two are byte-for-byte identical to the dbt.generate_series / dbt.get_powers_of_two macros that now live in dbt-adapters' global_project. The only difference between the two pairs is the dispatch namespace argument.

Maintaining two copies has concrete downsides:

  1. Adapter maintainers must register the same override twice, on both the dbt_utils namespace and the dbt namespace, to give users consistent behavior across entry points.
  2. Behavior can silently diverge for users depending on whether they call dbt_utils.generate_series directly or reach it indirectly through dbt.date_spine (which calls dbt.generate_series).
  3. Two dispatch paths mean two places to debug when adapter behavior is wrong.

Concrete impact (one example)

dbt-fabric-samdebruyn ships a fabric__generate_series override on the dbt_utils namespace that uses Microsoft Fabric's native GENERATE_SERIES table-valued function. The TVF is dramatically faster than the cross-join power-of-two CTE for non-trivial bounds. But users who reach this code path via dbt.date_spine (which calls dbt.generate_series, not dbt_utils.generate_series) get the slow CTE version, because the override sits on the other namespace. The same trap exists for every adapter that wants to optimize generate_series.

History

The duplication was introduced intentionally in dbt-labs/dbt-core#8616 (commit aa86fdfe7, "Add date spine macros to core", released in dbt-core 1.7.0). The PR's commit message gives the motivation:

Basically if you are using the semantic layer currently, then it is almost a requirement to use dbt-utils, which is undesireable given the SL is supported directly in core.

In late 2023 the global_project migrated from dbt-core to dbt-adapters (commit 615ad1fe2). The macro now lives at https://github.com/dbt-labs/dbt-adapters/blob/main/dbt-adapters/src/dbt/include/global_project/macros/utils/generate_series.sql and is functionally interchangeable with the dbt-utils version.

Fix

Replace the bodies of default__generate_series and default__get_powers_of_two with a single line that returns the dbt.* equivalent:

{% macro default__generate_series(upper_bound) %}
    {{ return(dbt.generate_series(upper_bound)) }}
{% endmacro %}

{% macro default__get_powers_of_two(upper_bound) %}
    {{ return(dbt.get_powers_of_two(upper_bound)) }}
{% endmacro %}

For users without an adapter override: emitted SQL is identical (delegate target is byte-for-byte the same body).

For users with an adapter override on dbt_utils.generate_series: their override still wins via the existing dispatch path, no behavior change.

For users with an adapter override on dbt.generate_series only (e.g. anything inherited via dbt_external_tables-style global namespace overrides): they now also get their fast path when called through dbt_utils.generate_series. This is the desired new behavior.

Compatibility

dbt.generate_series and dbt.get_powers_of_two were added in dbt-core 1.7.0 (September 2023). This PR bumps require-dbt-version from >=1.3.0 to >=1.7.0 so the delegate target is guaranteed to exist. dbt-core 1.6 and earlier are no longer supported upstream, so the practical impact for active users should be minimal — but flagging this explicitly because the version floor change is the only externally visible behavior in this PR.

If raising the floor is undesirable, an alternative is wrapping the delegate in an {% if context.get('dbt') and ... %} guard, but that adds complexity for a code path that targets versions the project no longer supports.

Testing

I have not run the integration test suite locally for this change. The intent is for CI to validate that dbt_utils.generate_series continues to emit equivalent SQL on all supported adapters — the existing tests should catch any regression because the delegate target is the same code.

Open question for maintainers

Given that dbt.generate_series now lives in dbt-adapters and the semantic layer no longer needs to take a dependency on dbt-utils, should dbt_utils.generate_series and dbt_utils.get_powers_of_two be deprecated outright in favor of the dbt.* equivalents? This PR is a no-op-behavior bridge that makes deprecation safe — a future major release could remove the dbt_utils variants entirely without affecting users who have migrated their calls. Not assuming this is the goal, just flagging it because the duplication question naturally surfaces it.

References

The default__ implementations of dbt_utils.generate_series and
dbt_utils.get_powers_of_two are byte-for-byte identical to the
equivalents that now live in dbt-adapters (formerly dbt-core's
global_project). Maintaining two copies forces adapter maintainers
to register the same override on two dispatch namespaces and means
behavior can silently diverge.

Replace both default__ bodies with a return to the dbt namespace
equivalent. Adapter overrides on dbt_utils.generate_series keep
working through the existing dispatch path; users without any
override get identical SQL as before.

Bump require-dbt-version minimum to 1.7.0, the release in which
dbt.generate_series and dbt.get_powers_of_two were added to core
(dbt-labs/dbt-core PR 8616, commit aa86fdfe7).
@sdebruyn

Copy link
Copy Markdown
Contributor Author

Human reviewed. This would simplify the dbt-fabric adapter. Also, why have the macro in 2 places?

@sdebruyn sdebruyn marked this pull request as ready for review May 19, 2026 12:42
@sdebruyn sdebruyn requested a review from a team as a code owner May 19, 2026 12:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant