Repository navigation
ci: add a report-only DuckDB schema-review pilot - #1092
Open
PresentJay wants to merge 2 commits into
Open
PresentJay wants to merge 2 commits into
PresentJay wants to merge 2 commits into
Conversation
Contributor
|
👋 @PresentJay |
PresentJay
requested a deployment
to
elementary_test_env
October 26, 2026 07:44 — with
GitHub Actions
Waiting
PresentJay
requested a deployment
to
elementary_test_env
October 26, 2026 08:47 — with
GitHub Actions
Waiting
PresentJay
marked this pull request as ready for review
October 26, 2026 08:50
This branch is waiting to be deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds a per-PR report of predicted column changes to Elementary's persisted artifact tables. The intended benefit is that a reviewer can see a named column removal and decide whether it belongs in the change, before updating an existing installation.
I maintain dbt-plan. This is a report-only pilot proposal; Elementary's maintainers have not agreed to adopt it.
Why this fits Elementary
dbt_modelsanddbt_snapshotsboth callget_dbt_models_empty_table_query(), and both useincrementalwithon_schema_change="sync_all_columns". One edit to the shared macro can therefore change the schemas of two persisted artifact tables even though neither model SQL file changed.In a controlled local change, I removed only the
descriptionentry from that macro. The released analyzer then reported:This was an injected control, not a bug or production incident discovered in Elementary. An intentional removal is still useful review context: the report makes its persisted-schema effect explicit.
The existing warehouse integration tests continue to check runtime behavior. This job adds a comparison between the PR base and current compiled schemas directly to the Actions summary. It does not claim to validate uploaded values, hooks, all consumer projects, or every warehouse-specific schema.
What maintainers would take on
pull_requestjob with read-only repository permissions. It uses the existing DuckDB profile generator and an in-memory database, with no cloud credentials or external analysis service.fail-on: never). Installation, compilation and invalid-report failures remain visible job failures. The PR does not change branch protection or make this a required check.Validation with the published release
Both the Action commit and installed package are pinned to v0.18.3. The derived-table improvement from dbt-plan #312 is now published; this proposal no longer depends on unreleased code.
At upstream
e3ab23d8c2e3bd7ad455de8bb1d23bf7ee84e675, using Core 1.11.14, dbt-duckdb 1.10.1 and SQLGlot 30.24.0:descriptiondrops in both models (exit 1) and retained a separate unresolved downstream warning.actionlintand the project's existing Prettier hook passed.The SQL control is not part of this PR. Full cloud integration was not run locally, and existing cloud CI still requires maintainer approval.
Feedback sought: does a named macro-generated schema delta add useful review context for this package, and is the extra CI job worth its installation and compile overhead?
Summary by CodeRabbit