ci: cut a PR push from ~15 min of CI to ~2.5

Merged
SA
1mo ago
+1261-3646

Why

A PR push took about 15 minutes of CI (median of 40 runs): Build 14.4 min, Test 8.6 min, Typecheck 6.4 min, and preview deploys about 7 min. It now takes about 2.5 minutes, with the required checks done in about 1.5.

Timings on the last run

CheckBeforeNow
Test (required)8.6 min1m18s
Typecheck (required)6.4 min1m29s
Lint (required)~1m15s1m02s
Desktop Build14.4 minremoved
Build CLI4 targetslinux-x64 on PRs, 1m39s
Preview deploys (whole run)~7 min2m35s
Sandbox Boot twice~4 min1m04s warm

What changes

CI

  • The desktop Build job is gone. In about 2,000 runs it failed alone twice, both out-of-memory exits. "Build" was removed from the main ruleset.
  • Test runs on a Blacksmith 8 vCPU runner, four turbo tasks at a time, with the terminal node-tests as a turbo task in the same step. Typecheck runs on Blacksmith 4 vCPU.
  • The CLI build runs on every PR (linux-x64 only); pushes to main build all four targets.
  • Sandbox Boot twice uses Blacksmith's persistent Docker layer cache.

Toolchain

  • Bun 1.4.2 with a regenerated lockfile, TypeScript 7.0.2 everywhere. Three desktop files that use the compiler API import @typescript/typescript6.
  • Every package's tests run with bun test --isolate.
  • Overrides pin happy-dom 20.11.2 and mdast-util-to-markdown 2.1.2; the metro patch covers the three resolved versions.

Tests

  • Deleted tests that slept on real timeouts (25 s, 35 s and 20 s cases), the generated adversarial and bug-hunt-2 suites, and git-watcher-lazy-registration.integration.test.ts, which died with exit 137 in 30 of 32 flaky runs for a reason we did not find.
  • trpc tests set env through a helper instead of mock.module; CLI tests drive the real notify function.
  • AGENTS.md has a Tests section with the rules this came from.

Deploys

  • Preview apps start after a small create-database job and no longer wait for migrations.
  • Preview and production Vercel deploys run on Blacksmith with the bun cache. Production writes the Next build cache; previews only read it.
  • Previews run on every PR, including desktop-only ones.
  • Vercel's "On-Demand Concurrent Builds" was switched to run all builds immediately.

Marketing

  • cacheComponents is on. Pages are prerendered for the default locale and the session buttons stream in; other locales and unlisted slugs render on first visit and are cached.
  • leaderboard, stats, starchart, fight, the-production-run and profile pages stay per request.
  • public/ video, GIFs and unused folders are excluded from the server functions.

Behaviour changes to know about

  • Unknown top-level marketing URLs return 200 with the not-found page and a noindex tag, where they returned 404. They are treated as profile handles, and the page shell is sent before the lookup. Unknown blog, changelog, compare, legal, team and theme URLs still return 404.
  • Previews go live before migrations finish. A failing migration shows as failed in the PR comment while the app links work against an unmigrated branch.
  • Production deploys depend on Blacksmith runners.
  • Fork PRs get failing deploy jobs on every push, because forks have no Vercel or Neon secrets.
  • Static marketing pages share a cached value for up to a minute, so a failed feature-flag or star-count read is served to everyone until the next refresh.
  • removeDirectoryTree also recovers on ENOTEMPTY; LocalGitCredentialProvider ignores EPIPE on git's stdin.

Verified

  • CI, previews and Sandbox green on the head commit; Test passed six runs in a row at four tasks (1m11s to 1m18s).
  • Marketing on the Vercel preview: prerendered pages, first-visit caching for other locales and slugs, 404s, OG images.
  • Desktop prebuild (electron-vite compile, native modules, runtime validation) passes locally on macOS arm64.
  • An independent review of the diff; its two findings (build cache size, cached API failures) are fixed.

Not verified

  • Desktop packaging and signing on the new toolchain. Needs a canary after merge.
  • macOS CLI builds with --ignore-scripts; they first run on push to main.
  • How well a production-built Next cache warms a preview build; nothing writes it until this merges.
  • The logged-in "Dashboard" button on marketing, and the now-static text routes (llms.txt, robots.txt, schemamap.xml, .well-known) against production.
  • A cold expo export with the metro patches.

After merge

  • Add the five deploy checks to the main ruleset.
  • Cost: about $290 to $390 a month on Blacksmith at last month's volume, plus roughly $95 on Vercel build minutes.

Summary by CodeRabbit

  • Improvements
    • Updated the marketing site’s page rendering and caching behavior, including dynamic data pages and profile listings.
    • Improved workspace cleanup reliability when directories cannot be removed on the first attempt.
    • Updated the minimum supported Bun version to 1.4.2.
  • Developer Experience
    • Updated development tooling and libraries, and improved test isolation and CI workflows.
    • Added guidance for writing reliable tests.