Repository navigation
Conversation
A parent missing from catalog.json (uncataloged source, ephemeral model, or an incomplete dbt docs generate) was written to the sqlglot schema as an empty mapping. sqlglot rejects zero-column tables, so the whole model fell back to transformation_type=unknown, including columns from cataloged parents. Leave such parents out of the schema in both _build_schema_from_aliases and the Jinja fallback loop in get_cll. Table-qualified and cataloged columns now resolve; columns that only the missing parent could supply stay unattributed. Implements [DRC-4918](https://linear.app/recce/issue/DRC-4918) Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com> Signed-off-by: Jared Scott <jared.scott@datarecce.io>
The change-analysis schema in get_change_analysis_cached wrote an empty table for any parent with no manifest columns. sqlglot then skips qualify() for the whole statement, so a select * could not expand and the column diff was wrong. Skip such parents, matching the CLL schema builders. Adds a regression test where the empty parent is in depends_on but not referenced in the SQL. Known limit: a select * over a join that includes a schemaless table still cannot be expanded. Option 1 leaves that unattributed by design. Implements [DRC-4918](https://linear.app/recce/issue/DRC-4918) Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com> Signed-off-by: Jared Scott <jared.scott@datarecce.io>
Codecov Report✅ All modified and coverable lines are covered by tests.
... and 3 files with indirect coverage changes 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR checklist
What type of PR is this?
Bug fix
What this PR does / why we need it:
DbtAdapterbuilds the sqlglot schema forcll(). A parent missing fromcatalog.json(uncataloged source, ephemeral model, incompletedbt docs generate) was written as an empty column mapping. sqlglot rejects zero-column tables, so the whole model fell back totransformation_type="unknown", including columns that come from cataloged parents.This PR skips parents with no catalog columns in both schema builders:
_build_schema_from_aliases(pre-compiled path)get_cllCataloged parents and table-qualified columns now resolve. Columns that only the missing parent could supply stay unattributed. Attributing them to the single schemaless source is a separate product decision (option 2 in the issue) and is not included here.
Tests in
tests/adapter/dbt_adapter/test_dbt_cll.py:test_build_schema_from_aliases_omits_parents_without_catalog_columns(unit)test_cll_pre_compiled_with_parent_missing_from_catalog_keeps_cataloged_columnstest_cll_jinja_path_with_parent_missing_from_catalog_keeps_cataloged_columnstest_change_analysis_ignores_parent_without_columns(intest_dbt_adapter.py)Run:
python -m pytest tests/adapter/dbt_adapter/test_dbt_cll.py tests/adapter/dbt_adapter/test_dbt_adapter.py→ 65 passed.Which issue(s) this PR fixes:
Implements DRC-4918
Special notes for your reviewer:
_get_schemahelper (get_change_analysis_cached) had the same empty-entry bug. Its regression test istest_change_analysis_ignores_parent_without_columns. Before the fix the result was{'id': 'added', 'a': 'added'}, and after it is{'b': 'removed'}.select *over a join that includes a schemaless parent still cannot be expanded, so its column diff stays imprecise. Attributing those columns is option 2, which is out of scope.Does this PR introduce a user-facing change?:
🤖 Generated with Claude Code