build(i18n): generate compiled Lingui catalogs instead of committing them

Merged
SA
2mo ago
+53-18

Summary

  • Stop committing the compiled Lingui catalogs (packages/i18n/locales/*/messages.ts); generate them from the .po sources instead.
  • Wire lingui compile into the build graph so they exist everywhere they're read: turbo ^build for dependents, a self-dependency for i18n's own typecheck/test, and postinstall for the paths that bypass turbo.

Why / Context

lingui compile emits one minified line per locale — 248 KB on a single line for en, zero newlines. Git's merge is line-based, so a one-line file can never be three-way merged: if two branches both touched it, that's a whole-file conflict, 100% of the time.

The result was that every PR adding any string conflicted with every other PR adding any string, on all 17 compiled catalogs, regardless of what the strings were. #7179 hit it against three unrelated PRs (#6901, #7176, #7175) that had landed while it was open. The resolution is never a hand-merge — it's "take the merged .pos and regenerate" — which is a strong hint the file shouldn't be in git at all.

The .po sources merge cleanly and always did, because packages/i18n/lingui.config.ts sets orderBy: "message" and origins: false, keeping them deterministic, alphabetically-ordered, multi-line text.

Lingui's own CLI docs say not to commit the compiled output:

"The compiled files can be safely ignored by your version control system, since these files must be created each time you deploy to production."
"We recommend you to create the compiled catalogs in CI as part of your deployment process."

How It Works

packages/i18n gains a build script that runs compile only — extract rewrites the .po sources from app code and has no business running inside a build. Three entry points then cover everything that reads the catalogs:

pathcovered by
turbo typecheck / test / build in any dependentroot pipeline's existing ^build
@superset/i18n's own typecheck / testnew packages/i18n/turbo.json, self-dependency on build
turbo dev (incl. Metro/Expo)root dev gains dependsOn: ["@superset/i18n#build"], scoped to that one package so dev start doesn't drag in every other build
starting an app directly (cd apps/mobile && bunx expo start)a predev hook on each app that imports @superset/i18n
packages/cli build:dist (invoked directly, not via turbo)a prebuild:dist hook
bun run --filter=<pkg> typecheck on a fresh clonescripts/postinstall.sh — local only, CI installs with --ignore-scripts in 26 steps

The package-level turbo.json also declares inputs (locales/*/messages.po, lingui.config.ts) and outputs (locales/*/messages.ts) so the compile step caches instead of re-running on every task.

postinstall's compile is deliberately non-fatal — a missing translation must fail a real build, not bun install. It prints a pointer to bun run check:i18n and moves on; the strict compile in the build graph is what actually gates.

Note that postinstall covers local clones only: CI installs with --ignore-scripts almost everywhere. Anything CI builds must therefore reach the catalogs through turbo ^build or an explicit pre-hook — which is why build:dist got one after Build CLI caught this on Linux.

Manual QA

Verified by deleting all 17 compiled catalogs and running each entry point cold:

  • ./scripts/postinstall.sh → 17 catalogs regenerated
  • turbo run typecheck --filter=@superset/i18n → runs build first, then typechecks; 17 regenerated
  • turbo run typecheck --filter=@superset/desktop → 6 tasks successful; 17 regenerated
  • turbo run build --filter=@superset/desktop from zero catalogs
  • CI's exact gate — bun run check:i18n && git diff --exit-code --stat HEAD -- packages/i18n/locales — passes clean, and still tracks all 17 .po files, so stale-catalog drift is still caught
  • git check-ignore confirms the compiled catalogs are ignored; git ls-files shows 0 tracked messages.ts, 17 tracked messages.po
  • Direct dev is not stale: edited en/messages.po, ran apps/web's predev, confirmed the new message reached the compiled catalog. The same probe before the predev hooks showed the catalog unchanged — i.e. a direct dev start served the previous compile.
  • Conflict class is gone: two throwaway branches each inserting a message at a different alphabetical position in en/messages.po (the way extract writes them) now merge with zero conflicts. The identical experiment before this change conflicted on all 17 messages.ts.

Testing

  • bun run lint — clean
  • bun run --filter=@superset/i18n test — 120 pass
  • bun run check:i18n — clean
  • bun turbo run typecheck --filter=@superset/desktop — clean
  • bun turbo run build --filter=@superset/desktop — clean from a zero-catalog state

Risks / Rollout

  • Risk: a path that reads the catalogs without going through turbo, predev, or postinstall would now see missing files. The four covered entry points above are the ones I could find; the failure mode is a loud Cannot find module '../locales/en/messages' at typecheck, not a silent runtime bug.
  • Rollout: no flag, no ordering constraint. First bun install or any turbo task after merging regenerates locally.
  • Rollback: revert. The compiled catalogs come back under version control and the conflict class returns with them; nothing is lost, since they're fully derived from the .po files that stay tracked.

Follow-ups

  • The other Lingui friction is unaddressed here and worth its own look: the CLI silently no-ops under Node <22.18 and compile --strict exits 0 under Bun's worker pool (why our scripts pin bunx --bun … --workers 1), and bun test needs macro shims in apps/desktop/test-setup.ts because macros are compile-time.

https://claude.ai/code/session_01DNoj8JhjGwTRs9AoxYYRB9

Summary by CodeRabbit

  • Build & Development

    • Translation catalogs are now compiled automatically before development servers, builds, type checks, and tests.
    • Localization files are generated during installation to improve setup reliability without blocking installation if generation fails.
    • Updated application and command-line workflows ensure translations are available before startup or packaging.

  • Chores

    • Generated translation catalog files are now excluded from version control.