feat(box): re-fetch failed documents by id (Resolver)

Merged
EV
2d ago
+615-10

Description

BoxConnector now implements Resolver.reindex. Targeted reindex can re-fetch Box documents by id, without a crawl or a checkpoint.

Document types covered

  • Files (box-file-<id>): fetched with files.get_file_by_id.
  • Web links (box-weblink-<id>): fetched with web_links.get_web_link_by_id, only when include_web_links is on.

The Box connector emits no other document types, so all types are covered.

How it works

  • The item's path_collection gives its folder chain. The connector finds the outermost configured entry folder in that chain and rebuilds the crawl's frontier entries from that folder down to the parent. The path metadata, parent_hierarchy_raw_node_id and folder HierarchyNodes match the crawl.
  • Documents are built with the crawl's own _convert_file / _convert_web_link. The folder HierarchyNode builder moved into a shared _folder_hierarchy_node helper.
  • Folder entries are cached for the whole call, so targets in the same folder share upstream calls. Each folder node is yielded once.

Failures (each keyed on the input document id)

  • Ids that do not parse as box-file-<digits> or box-weblink-<digits>.
  • Items that are not found (404), not readable (403), trashed, or deleted.
  • Items outside the configured folders.
  • Web links when include_web_links is off.
  • Files that the crawl skips (unsupported type, over the size threshold, empty download), and conversion errors.

A BoxAPIError on one target does not stop the other targets. Credential errors (HTTP 401, or OAuth invalid_client / unauthorized_client) are raised again. Entity failures are skipped, as in Confluence and Google Drive.

Permissions
With include_permissions=True, access uses the same EE helpers as the perm-sync crawl: resolve_box_ancestor_access for the folders above the entry folder, resolve_box_folder_access for each folder down to the parent, then resolve_box_file_access / resolve_box_web_link_access for the item. Folder nodes carry the same access too.

How Has This Been Tested?

New daily test backend/tests/daily/connectors/box/test_box_resolver.py, run against the Box CI enterprise:

uv run --env-file .vscode/.env pytest -xv backend/tests/daily/connectors/box/test_box_resolver.py
13 passed

It covers: one file, all files plus the web link, include_permissions=True (the test asserts external_access on docs and nodes equals the perm-sync crawl's), a disabled web link, a file outside the configured folder, bogus file and web link ids, unparseable ids, a bad target next to a good one, empty input, and entity-only input.

Also run: backend/tests/daily/connectors/box/test_box_basic.py (6 passed) and backend/tests/unit/onyx/connectors/box (61 passed).

Additional Options

  • [Optional] Please cherry-pick this PR to the latest release version.
  • [Optional] Override Linear Check

🤖 Generated with Claude Code

Summary by cubic

BoxConnector now implements Resolver.reindex, so failed documents can be re-fetched by ID without a crawl or checkpoint.

  • File (box-file-<id>) and web link (box-weblink-<id>) targets are fetched via the Box API; web links only when include_web_links is on.
  • The connector rebuilds the crawl's folder chain from the innermost configured entry folder down to the item's parent, matching how the crawl roots nested configured folders, and reuses _convert_file / _convert_web_link for metadata, HierarchyNodes, and permissions.
  • Failures are keyed on the input document ID for unparseable IDs, trashed or inaccessible items, out-of-scope folders, disabled web links, and items the crawl would skip. One bad target does not abort the others; credential errors, including from downloads and permission reads, are re-raised.
  • With include_permissions=True, folder and document access match the perm-sync crawl via the existing EE helpers.
  • Adds daily and unit tests covering success paths, permission parity, nested configured folders, and each failure case.

Summary by CodeRabbit

  • New Features
    • Added targeted reindexing for Box files and web links using document IDs.
    • Reindexed items retain their metadata, folder hierarchy, and permissions.
    • Missing parent folders can be rebuilt during reindexing.
    • Multiple items can be processed while reporting individual successes and failures.
    • Clear failures are provided for malformed, unknown, inactive, out-of-scope, or unavailable items.
    • Reindexing now respects Box item status and configured folder or web-link scope.