feat(jira): re-fetch failed documents by id (Resolver)

Merged
EV
3d ago
+578-50

Description

JiraConnector now implements Resolver.reindex. Targeted reindex can now re-fetch failed Jira documents by id.

Document types covered

  • Jira issues. The issue is the only document type the connector writes. Comments are part of the issue text, and the connector does not index attachments. The document id is the issue URL (<base>/browse/<KEY>).

Document types not covered

  • None.

How it works

  • reindex parses the issue key from each document URL. The URL must start with the connector's own <base>/browse/, and the key must be a valid Jira key. Other ids yield a ConnectorFailure with the input id.
  • It fetches the issues with one JQL search per batch of 50 keys: key in (...), limited to the configured project or JQL (new scope_jql helper, which build_jql_query now uses too). The search uses the same paths as the crawl (Cloud: enhanced search + bulk fetch; Server/DC: v2 search).
  • Server/DC rejects a JQL query that names a missing key. When a batch fails, the connector fetches each key alone, so one bad key does not fail the rest. A 401 or 403 still fails the whole run, as in the crawl.
  • Each input id that does not give a document yields a ConnectorFailure with the reason: fetch error, conversion error, skipped by label or size, or not found in scope (deleted, moved, or out of scope).
  • Entity failures (failed_document is None) are skipped, as in Confluence.
  • The per-issue conversion (hierarchy nodes, then the document) moves out of _load_from_checkpoint into _issue_to_items. The crawl and reindex both use it. reindex yields the project node and the Epic nodes before the documents.

Permissions

  • With include_permissions=True, reindex sets external_access with _get_project_permissions(project_key, add_prefix=True). This is the same call that load_from_checkpoint_with_perm_sync uses, and it caches each project.

How Has This Been Tested?

  • New daily test backend/tests/daily/connectors/jira/test_jira_resolver.py, against the live CI site (project AS):
    uv run --env-file .vscode/.env pytest -xv backend/tests/daily/connectors/jira/test_jira_resolver.py -> 9 passed.
    It covers: reindex of one issue matches the crawl output and hierarchy nodes; Story and Epic together (classic and scoped token); missing key; out-of-scope issue (JQL scope); unparseable ids; empty input; entity-only input; include_permissions=True sets external_access.
  • New unit test backend/tests/unit/onyx/connectors/jira/test_jira_reindex.py for the Server/DC per-key fallback and the 401 path. All Jira unit tests after the rebase onto main: 141 passed. The daily results below are from before the rebase.
  • All Jira daily tests: 29 passed.

Additional Options

  • [Optional] Please cherry-pick this PR to the latest release version.
  • [Optional] Override Linear Check

🤖 Generated with Claude Code


Summary by cubic

Adds Resolver.reindex to the Jira connector so targeted reindexes can re-fetch failed issues by document id.

  • Parses each document URL's issue key, fetches issues in batches via key in (...) JQL scoped to the configured project or JQL, and yields the project and Epic hierarchy nodes before each document.
  • On Server/DC, a batch that fails on a missing key falls back to per-key fetches; 401, 403, and 429 errors stop the run as in the crawl.
  • Moves per-issue conversion into a shared helper used by both crawl and reindex; unknown, unparseable, or out-of-scope ids yield a ConnectorFailure.

Summary by CodeRabbit

  • New Features

    • Added targeted Jira issue reindexing from issue URLs.
    • Supports scoped and unrestricted Jira searches while preserving custom queries and project filters.
    • Rebuilds Jira document content, hierarchy, and access permissions where available.
  • Bug Fixes

    • Improved handling of invalid, missing, out-of-scope, and malformed Jira issues.
    • Reports per-issue failures while preserving successful results.
    • Handles batch-search fallback, credential errors, permissions, and rate limits more reliably.
  • Tests

    • Added unit and live integration coverage for Jira reindexing scenarios.