fix(sync): stop small appends from holding the database lock for seconds
Sync in the OpenTubeX app sometimes fails with SyncServerError: internal database error. On sync.opentubex.org these errors came in bursts of 500 responses that each took almost exactly 30 seconds, which is the SQLite busy_timeout. In each burst, one write transaction held the write lock for 36 to 91 seconds, and every request waiting behind it gave up. In one burst the slow request was a POST /v1/encrypted_sync/history/changes, the endpoint meant to make small history changes cheap.
append_change inserted the change and then updated revision and base_revision on the encrypted_sync row, which also holds the collection's full payload. SQLite has to read a row's whole overflow chain to update any of its columns, and it rewrites the chain when the record size changes, for example when the revision goes from 1 to 2. On the old code, one append of a few bytes to a 1 MiB collection wrote 2.1 MB to the WAL.
Appends now only insert into encrypted_sync_change. The row's revision and base_revision describe its stored payload, and a collection's revision is its newest change above the row's revision, or the row's revision if there is none. Every read selects that revision in the same SQL statement as the row, so PostgreSQL reads both at one snapshot. Existing data needs no migration. Rows that older appends updated already have their newest change as their revision.
Database lock timeouts (SQLite database is locked, PostgreSQL serialization failures) now answer 503 with Retry-After: 30 and "the database is busy; try again shortly" instead of a bare 500. The handlers also log database errors instead of discarding them, so the server log shows what failed.
After deploying this, don't roll back to an older server once it has accepted appends. The old code would not see those changes until the next full upload of each collection.
Proof: a new test fails on the old code (2.1 MB written for one append) and passes with the fix. Another test holds the SQLite write lock and checks for the 503 with Retry-After. I also ran the new PostgreSQL queries against PostgreSQL 18 and checked the returned revisions and column types. cargo test, cargo fmt --check, and cargo clippy --all-targets pass, and clippy also passes for --no-default-features --features postgres.
Reviewed with sol-loop: 2 rounds with gpt-6.1-sol on xhigh.
Sol verdict: approved
Changes made by Claude Opus 5.5 in Claude Code (T3 Code).