capture previous metadata during schema migration to support rollback

Merged
AJ
4mo ago
+1934-170

Why

When a branch merge fails partway through, the recovery path rolls each affected
vertex back by restoring updated_at/updated_by from the
previous_updated_at/previous_updated_by snapshot the merge recorded before it
bumped the metadata. Schema migrations (inheritance changes, attribute add/remove,
kind changes, etc.) also bump updated_at/updated_by on the Node/Attribute/
Relationship vertices they touch — but they never wrote the previous_updated_*
snapshot. So if a merge that included a schema migration failed, rollback had
nothing to restore from and the affected objects were left with the migration's
timestamp/author instead of their original values.

Goal: make every schema-migration query snapshot the prior user-timestamp
metadata into previous_updated_at/previous_updated_by before it overwrites
updated_at/updated_by, so merge-failure rollback can restore it — matching what
the merge path already does.

Non-goals: no change to the rollback query itself, to the merge flow, or to
which vertices a migration touches. This only adds the snapshot write. The rollback
query already handles resetting the updated_at/by metadata using previous_updated_at/by.
This change is just to correctly set the previous_ properties during schema migrations.

Closes IFC-2716

What changed

  • Recovering from a failed branch merge now restores updated_at/updated_by for
    objects, attributes, and relationships that a schema migration touched (e.g.
    changing schema inheritance or adding an attribute), instead of leaving them
    stamped with the migration's time/author.

Each migration Cypher query that bumps updated_at/updated_by on a vertex now
first snapshots the prior values into previous_updated_at/previous_updated_by,
guarded so a migration that touches the same vertex twice in one run does not
clobber the original snapshot:

SET v.previous_updated_at = CASE
        WHEN v.updated_at IS NULL OR v.updated_at <> $current_time THEN v.updated_at
        ELSE v.previous_updated_at
    END,
    v.previous_updated_by = CASE
        WHEN v.updated_at IS NULL OR v.updated_at <> $current_time THEN v.updated_by
        ELSE v.previous_updated_by
    END
SET v.updated_at = $current_time, v.updated_by = $user_id

The CASE condition means: only overwrite the snapshot when this is the first
write of the current migration timestamp. If the vertex's updated_at already
equals $current_time, the snapshot has already been taken this run and is left
untouched — the migration is idempotent with respect to the snapshot.

Vertices created by the migration (a newly added attribute, the renamed
attribute vertex, the duplicated node) have no prior metadata, so they are not
snapshotted — noted inline where it applies.

Queries updated:

  • query/attribute_add.py — snapshot the parent Node
  • query/attribute_remove.py — snapshot the Attribute and its Node
  • query/attribute_rename.py — snapshot the Node
  • query/node_duplicate.py — snapshot the original Node
  • schema/attribute_kind_update.py — snapshot the Attribute and its Node
  • schema/node_relationship_remove.py — snapshot the Relationship, Node, and peer
  • schema/node_remove.py — snapshot the active Node and its peers (in/out)

No schema changes, no new dependencies, no API changes. The rollback query and
merge flow are untouched.

How to review

Start with one query (e.g. query/attribute_remove.py) to see the snapshot
pattern, then confirm the same CASE-guarded block was added everywhere a
migration does SET v.updated_at = .... The tests are the bulk of the diff.

Key thing to scrutinize: the idempotency guard. A single migration run can pass
over the same vertex more than once (e.g. a node reachable as both active node and
peer); the updated_at <> $current_time check must prevent the second pass from
overwriting the real snapshot with the migration's own timestamp.

Tests

New coverage:

  • test_all_migrations_rollback.py — end-to-end: run each migration, capture a
    metadata + edge fingerprint before, roll back, assert the branch is restored
    exactly (empty diff).
  • Per-migration tests (test_attribute_kind_update.py, test_attribute_name_update.py,
    test_node_attribute_add.py, test_node_attribute_remove.py,
    test_node_kind_update.py, test_node_relationship_remove.py, test_node_remove.py)
    extended to assert the previous_updated_* snapshot is written on migration and
    cleared on rollback.
  • metadata_helpers.py / helpers/db_validation.py — shared fingerprint helpers.

Impact & rollout

  • Backward compatibility: no breaking changes. Existing vertices without a
    snapshot are handled by the updated_at IS NULL branch. No data migration needed.
  • Performance: negligible — a couple of extra property writes on vertices the
    migration already writes.
  • Config/env changes: none.
  • Deployment notes: safe to deploy.

Checklist

  • Tests added/updated
  • Changelog entry added (+migration-previous-metadata.fixed.md)
  • External docs updated (if user-facing or ops-facing change) — n/a, internal recovery behavior
  • Internal .md docs updated (internal knowledge and AI code tools knowledge)
  • I have reviewed AI generated content

Summary by cubic

Adds snapshotting of previous metadata in schema migrations so rollback after a failed merge can restore updated_at/updated_by. Adds end-to-end tests that confirm full graph and metadata restoration (IFC-2716).

  • Bug Fixes
    • Save previous_updated_at/previous_updated_by before updates in: attribute add/remove/rename, attribute kind update, node relationship remove, node remove, and node duplicate (original node).
    • Guard against overwriting snapshots when the same timestamp is reused during a run.
    • Rollback restores vertex metadata from snapshots; added per-migration tests and an all-migrations merge/rollback test with shared helpers (branch_metadata_fingerprint, branch_edge_fingerprint, vertex metadata getters, count_branch_edges_at); removed dead test code; fixed a race in graph test 040 data prep.
    • No schema or API changes; changelog entry added.