fix(files): index generated docs without compiling and fix the docx sandbox bundle

Merged
IC
2mo ago
+723-162

Summary

  • Workspace file search indexing no longer compiles generated documents. The indexer reads the compiled artifact when one exists and otherwise indexes the generation source as text, through the same read-only resolver the public share route uses. It previously went through the compile-or-load download path, which inside a Trigger.dev worker took the isolated-vm fallback and tried to run Python and Node-style document sources as sandbox JavaScript.
  • Parser failures in the indexer follow the file tool's policy: a structured parser rejecting UTF-8 bytes (a .json file wrapped in a markdown code fence, for example) falls back to raw text, and rejecting binary bytes marks the revision skipped instead of retrying three times and failing it.
  • The docx sandbox bundle threw ReferenceError: __require is not defined on load since the docx 9.7.x bump: docx inlines JSZip's UMD together with esbuild's __require helper, and Bun's browser/iife build rewrites the bare require references inside it to a runtime helper it never emits. _polyfills.ts defines the stub, build.ts evaluates every bundle in a bare node:vm context before writing it, and a test loads the checked-in bundles the same way. All three bundles are regenerated.
  • trigger.config.ts syncs the doc-sandbox template variables and the copilot artifact bucket, so workers read generated documents through the compiler the app authored them for and can load the compiled artifacts. The values still have to exist in the Trigger.dev environment.
  • A cancelled workspace-file download now surfaces as the abort instead of being rewrapped as a transport failure.

Type of Change

  • Bug fix

Testing

  • bun run test for the new extract.test.ts, verify.test.ts, and workspace-file-manager-download.test.ts, plus the uploads, doc-compile, serve-route, and workspace-files suites
  • Bare node:vm smoke: real docx, pdf, and pptx documents generated through the rebuilt bundles
  • type-check, lint, check:audits, boundary and egress checks

Revisions that already failed are terminal (the dispatcher only re-claims pending), so once the fixed worker is live and the worker environment has the doc template and copilot bucket, re-drive them:

UPDATE workspace_file_search_index
SET status = 'pending', dispatched_at = NULL, failure_reason = NULL, updated_at = now()
WHERE status = 'failed' AND failure_reason = 'indexing_error';

INSERT INTO workspace_file_search_dispatch_queue (workspace_id, enqueued_at, updated_at)
SELECT DISTINCT workspace_id, now(), now()
FROM workspace_file_search_index
WHERE status = 'pending' AND dispatched_at IS NULL
ON CONFLICT (workspace_id) DO UPDATE SET updated_at = now();

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

šŸ¤– Generated with Claude Code