fix(validate): fold uppercase metadata to the engine's identifier case; skip existence checks without a schema

Merged
SA
2mo ago
+729-81

Issue for this PR

Closes #1333

Type of change

  • Bug fix

What does this PR do?

Two defects in altimate_core_validate, both reproduced with the issue's script:

  1. Case. The engine compares an unquoted identifier in lowercase and a quoted one exactly (verified: "Customer_Region" stays mixed-case). Snowflake metadata from schema_inspect / snowflake_get_table_stats is UPPERCASE, so a correct query failed with ColumnNotFound — with the engine's own DidYouMean pointing at the same column in uppercase at confidence 1 — and uppercasing the SQL did not help. normalizeSchemaContext now folds an all-uppercase table or column name (the storage form of an identifier created unquoted) to lowercase before the schema reaches the engine. Mixed-case names are left alone: they were created quoted, must be referenced quoted, and the engine matches them exactly — folding them would break that. Lowercase names are already in the engine's form. Applies to both schema_context shapes (flat map and SchemaDefinition). schema_path files are not folded (out of scope; the issue is about inline metadata from the warehouse tools).

  2. No schema. The engine ran against the _empty_ placeholder and reported every table as missing, while the tool's description promises existence checks are skipped. The handler now drops TableNotFound/ColumnNotFound (E001/E002) findings when no schema was given and recomputes valid from what remains, so syntax and dialect errors still surface.

How did you verify your code works?

  • test/altimate/validate-identifier-case.test.ts (9 tests): the exact query and uppercase metadata from the issue validate through the dispatcher, in lowercase and uppercase SQL; a genuinely misspelled column is still ColumnNotFound; no schema → valid: true for the correct query, and a real syntax error (selec x fro t, engine E000) is still reported; unit cases for foldIdentifierCase (upper → lower, mixed kept, lowercase untouched, digits) and the existence classifier.
  • Mutation-checked: removing the fold fails 5 tests; keeping existence errors without a schema fails 1.
  • altimate-core-e2e, sql-validation-adversarial, tool-error-propagation, issue-261-e2e and the new file: 244 pass. Typecheck clean.

One note for reviewers: the fold makes the validator lenient in one direction on case-sensitive-by-default engines (Postgres stores unquoted as lowercase, so an all-uppercase Postgres column must have been created quoted and referenced quoted; after folding, an unquoted lowercase reference would validate). That is a missed error rather than a false positive, and the issue's cost was the false positives.

Screenshots / recordings

N/A.

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

🤖 Generated with Claude Code

https://claude.ai/code/session_012Q51zFUmPg1WwtS5CrGJE6

Summary by cubic

Fixes altimate_core_validate rejecting correct Snowflake queries against warehouse metadata and reporting every table as missing when no schema is given.

  • Uppercase metadata (e.g. ORDER_MONTH) folds to the engine's lowercase form before validation; mixed-case quoted names stay exact, and a fold that would collide with an existing name keeps the entry as written.
  • Quoted all-uppercase references ("ORDER_MONTH", dbt/BI style) and BigQuery/Databricks backticks are lowercased inside their quotes when a schema is present; string literals and comments are left alone.
  • A shared prepareSql folds both halves of the SQL-plus-schema pair for every such operation (altimate_core.* and sql.*), and applies the same preparation to sibling SQL (base_sql, the equal query, lineage batches).
  • JSON/YAML schema files are folded like inline contexts; DDL files load as-is.
  • Generated SQL (rewrites, fixes, optimizations) returns in the caller's spelling; sql.diff renders its text diff from the raw inputs and folds only for the equivalence check.
  • Without a schema, TableNotFound/ColumnNotFound findings are dropped and valid is recomputed, so syntax and dialect errors still surface; a schema context that normalizes to no table takes the no-schema path, and a schema file that fails to load is an engine failure rather than a valid result.

Summary by CodeRabbit

  • Bug Fixes
    • Improved SQL validation and analysis for uppercase, quoted, and mixed-case identifiers.
    • Improved handling of schema files, inline schemas, schema context, and schema-free validation.
    • Preserved original identifier casing in generated SQL and rewrites.
    • Case-only SQL differences are now displayed correctly.
    • Improved processing across SQL analysis, optimization, fixing, rewriting, and lineage checks.
    • Validation now clearly reports engine failures.
    • Missing tables and columns continue to be reported when schema information is available.
  • Tests
    • Added coverage for identifier casing, schema formats, lineage, and schema-free validation.