fix: avoid "database is locked" when several CLI processes start on a fresh data dir

Merged
AN
3d ago
+300-23

Issue for this PR

Closes #1540

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

Why. Several altimate-code processes starting at once on a fresh data directory (first run, CI, parallel agents) can make one of them exit with database is locked. Later starts are fine, so the failure is intermittent and hard to diagnose.

Root cause. Two races on first run:

  • packages/core/src/database/sqlite.bun.ts runs PRAGMA journal_mode = WAL before PRAGMA busy_timeout. The WAL switch needs a write lock, so it fails immediately instead of waiting.
  • Both database layers (packages/opencode/src/storage/db.ts and packages/core/src/database/database.ts) run first-run migrations concurrently. SQLite cannot queue a deferred read-to-write upgrade, so the loser gets SQLITE_BUSY at once.

What changed.

  • busy_timeout is installed before the WAL switch in both openers (sqlite.bun.ts, sqlite.node.ts, db.ts, database.ts).
  • New packages/core/src/database/setup-lock.ts: an exclusive transaction on a sentinel file (<db>.setup-lock) is a cross-process mutex that the OS releases if the holder dies. Synchronous setup blocks on it; the effect-based migration polls so in-process peers keep progressing.
  • Database.layer in core now takes an optional filename (layer(filename?)); layerFromPath passes it. No other callers.
  • Before: 3 simultaneous fresh starts failed in 6 of 20 rounds. After: 0 of 20.
  • Behavior delta: a new empty <db>.setup-lock file appears next to the database. In-memory databases are unaffected.

Tenant/user impact. None beyond the fix. No schema or data migration.

How did you verify your code works?

  • New regression test packages/opencode/test/storage/concurrent-open.test.ts (4 simultaneous processes, 3 rounds, fresh data dir). It fails 3 of 3 runs on origin/main (detached worktree with only the test added) with database is locked, and passes with the fix.
  • bun test test/storage test/session in packages/opencode: 1266 pass, 0 fail.
  • bun test test/database-migration.test.ts in packages/core: 21 pass, 0 fail.
  • bun run typecheck in packages/opencode and packages/core: clean.
  • 3-process concurrent start on a fresh temp data dir, 20 rounds: before 6 failed, after 0 failed.
  • bun run script/upstream/analyze.ts --markers --base origin/main --strict: ok.

Not run: Windows, and the node (non-bun) SQLite opener beyond typecheck.

Known limitation (follow-up): within a single process, a synchronous legacy database open that runs while the core migration holds its write transaction can still wait for the busy timeout and fail. This is pre-existing (the two layers had no coordination before this PR); the structural fix is to complete core setup before any synchronous legacy open, which changes startup ordering and is left for a follow-up.

Screenshots / recordings

N/A (no UI change).

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

🤖 Generated with Claude Code

[!NOTE]
Medium Risk
Touches critical startup paths for all SQLite openers and migrations; behavior change includes a new .setup-lock sentinel file beside the database.

Overview
Fixes intermittent database is locked / SQLITE_BUSY when several processes cold-start against the same fresh data directory (parallel CLI, CI, first run).

Cross-process setup serialization: New setup-lock.ts uses an exclusive transaction on a sentinel file (<db>.setup-lock) so only one process runs first-run WAL setup and migrations at a time; in-process callers use semaphores and skip blocking when a cooperative hold is already active. Core Database.layer now accepts an optional filename and wraps WAL + DatabaseMigration.apply in SetupLock.hold; legacy db.ts wraps its lazy open in SetupLock.acquireSync.

Lock ordering: All open paths set PRAGMA busy_timeout before journal_mode = WAL, and the WAL switch runs under the setup lock on Bun/Node SQLite layers. Bun read-only opens no longer allow readwrite/create, skip WAL, and are covered by a new test.

Same-process race: Legacy adopt/initialize schema checks run inside one BEGIN IMMEDIATE transaction so core migrations cannot interleave between checks and CREATEs.

Adds a regression test that spawns four processes opening both storage layers on a fresh shared data dir.

Reviewed by Cursor Bugbot for commit 13951136f9dc3328f422b09fcbe04b650051d3e5. Bugbot is set up for automated code reviews on this repo. Configure here.

Summary by cubic

Fixes intermittent database is locked failures when several processes start at once on a fresh data directory (first run, CI, parallel agents), closing #1540.

  • Installs PRAGMA busy_timeout before the WAL switch in all database openers, because switching to WAL needs a write lock that previously failed immediately.
  • Serializes first-run migrations and the WAL switch across processes using an exclusive transaction on a new <db>.setup-lock sentinel file; the OS releases the lock if the holder dies, and the wait is interruptible so cancelling startup doesn't hang.
  • Runs the legacy adopt/initialize schema checks inside one BEGIN IMMEDIATE transaction so a same-process core migration can't create the schema mid-check.
  • Makes read-only Bun opens truly read-only: Bun honors readwrite/create over readonly, so such opens could previously write and create a missing DB file.
  • Adds regression tests: four simultaneous processes opening both database layers against a fresh data directory, and read-only opens (no write, no file creation, journal mode untouched).
  • Behavior delta: an empty .setup-lock file now appears next to the database; in-memory databases are unaffected.

Summary by CodeRabbit

  • Bug Fixes
    • Improved reliability when multiple processes open the same database simultaneously, reducing setup conflicts and locked-database errors.
    • Configured SQLite to wait briefly for busy databases before continuing setup, helping initialization complete when another process is using the database.
    • Serialized database setup across concurrent open attempts, reducing conflicts during schema checks and initialization.
    • Database setup now completes more consistently when several application processes start at the same time.