fix(db): make seed script deterministic and produce realistic funnels

Closed
FI
10d ago
+293-5

Addresses part of #969

Problem

bun run db:seed has two real bugs, separate from the larger scenario-engine proposal in #969:

  1. It never calls faker.seed(), so two runs with the same arguments produce different data — no way to diff output for screenshot comparisons or reproduce a specific dataset.
  2. Paths are picked independently per event, so any funnel or goal built on the seeded data shows near-zero, meaningless conversion — there's no correlation between the pages one visitor views in sequence.

Fix

  • Added a --seed <n> CLI flag (default 42), applied via faker.seed() before any faker call, including the module-level USER_POOL/SESSION_POOL generation. Existing positional usage (db:seed <WEBSITE_ID> [EVENT_COUNT]) is untouched — the flag is stripped from process.argv before positional parsing so both forms compose.
  • Replaced the independent per-event random path pick with a weighted page-transition table and per-session path tracking, so a stable share of visitors who view one page go on to view a related next page (e.g. pricing → signup), instead of i.i.d. random paths.

Verification

  • bun run check-types (36 packages) and bun run lint (ultracite + lint:policies): clean.
  • packages/db's own test suite: 115 pass, 0 fail.
  • Full monorepo pre-push suite: 1101 pass, 0 fail.
  • Determinism: ran the seed script twice with the same --seed, different website ids, then queried ClickHouse for the full event row shape ordered by time — 300/300 rows byte-identical between runs.
  • Funnel realism: seeded 20,000 events and queried ClickHouse — sessions that viewed /pricing then later /signup in the same session: 24.9% (686/2752), vs. 16.8% baseline /signup visitation across all sessions. /signup → /dashboard: 21.7% vs. 12% baseline. Confirms funnels built on these paths now show non-degenerate conversion.

Explicitly out of scope

This is two objective bug fixes to the existing script, not the scenario-engine redesign proposed in #969 (SaaS journey model, custom events, revenue, profiles, uptime/alarms, flags, links, or reconciling with the separate e2e ClickHouse generator). That's gated on an open product question in the issue ("Is SaaS the right story, or e-commerce?") only the maintainer can answer. packages/db/src/seed.ts also still has no DB-target safety guard of its own (packages/db/src/test-env.ts's loopback guard is scoped to test bootstrap, not this script) — adding one is part of that larger proposal, not one of the two bugs here, so I left it out; verification above was done against localhost only.

Disclosure: implemented with AI assistance (Claude Code); verified and tested by hand — ran the seed script against local Postgres/ClickHouse, confirmed byte-identical output across two runs with the same --seed, and confirmed pricing→signup and signup→dashboard conversion lift via ClickHouse queries before submitting.

Summary by cubic

Makes bun run db:seed deterministic and gives seeded visitors realistic page-to-page sequences so funnels and goals built on seeded data show meaningful conversion.

  • Adds a --seed <n> CLI flag (default 42) applied via faker.seed() before any faker call, so two runs with the same arguments produce identical data. Existing positional usage is untouched.
  • Replaces independent per-event random path picks with a weighted page-transition table and per-session path tracking, so visitors who view one page have a stable chance of viewing a related next page (e.g. pricing → signup).
  • Fixes created_at to use the event's own time instead of a per-event Date.now() call, and moves flag parsing and path selection into seed-helpers.ts with unit tests.