feat(jira): re-fetch failed documents by id (Resolver)
Merged
EV
3d ago+578+578+012345678901234567890123456789-50-50-01234567890123456789
Description
JiraConnector now implements Resolver.reindex. Targeted reindex can now re-fetch failed Jira documents by id.
Document types covered
- Jira issues. The issue is the only document type the connector writes. Comments are part of the issue text, and the connector does not index attachments. The document id is the issue URL (
<base>/browse/<KEY>).
Document types not covered
- None.
How it works
reindexparses the issue key from each document URL. The URL must start with the connector's own<base>/browse/, and the key must be a valid Jira key. Other ids yield aConnectorFailurewith the input id.- It fetches the issues with one JQL search per batch of 50 keys:
key in (...), limited to the configured project or JQL (newscope_jqlhelper, whichbuild_jql_querynow uses too). The search uses the same paths as the crawl (Cloud: enhanced search + bulk fetch; Server/DC: v2 search). - Server/DC rejects a JQL query that names a missing key. When a batch fails, the connector fetches each key alone, so one bad key does not fail the rest. A 401 or 403 still fails the whole run, as in the crawl.
- Each input id that does not give a document yields a
ConnectorFailurewith the reason: fetch error, conversion error, skipped by label or size, or not found in scope (deleted, moved, or out of scope). - Entity failures (
failed_document is None) are skipped, as in Confluence. - The per-issue conversion (hierarchy nodes, then the document) moves out of
_load_from_checkpointinto_issue_to_items. The crawl andreindexboth use it.reindexyields the project node and the Epic nodes before the documents.
Permissions
- With
include_permissions=True,reindexsetsexternal_accesswith_get_project_permissions(project_key, add_prefix=True). This is the same call thatload_from_checkpoint_with_perm_syncuses, and it caches each project.
How Has This Been Tested?
- New daily test
backend/tests/daily/connectors/jira/test_jira_resolver.py, against the live CI site (project AS):
uv run --env-file .vscode/.env pytest -xv backend/tests/daily/connectors/jira/test_jira_resolver.py-> 9 passed.
It covers: reindex of one issue matches the crawl output and hierarchy nodes; Story and Epic together (classic and scoped token); missing key; out-of-scope issue (JQL scope); unparseable ids; empty input; entity-only input;include_permissions=Truesetsexternal_access. - New unit test
backend/tests/unit/onyx/connectors/jira/test_jira_reindex.pyfor the Server/DC per-key fallback and the 401 path. All Jira unit tests after the rebase onto main: 141 passed. The daily results below are from before the rebase. - All Jira daily tests: 29 passed.
Additional Options
- [Optional] Please cherry-pick this PR to the latest release version.
- [Optional] Override Linear Check
🤖 Generated with Claude Code
Summary by cubic
Adds Resolver.reindex to the Jira connector so targeted reindexes can re-fetch failed issues by document id.
- Parses each document URL's issue key, fetches issues in batches via
key in (...)JQL scoped to the configured project or JQL, and yields the project and Epic hierarchy nodes before each document. - On Server/DC, a batch that fails on a missing key falls back to per-key fetches; 401, 403, and 429 errors stop the run as in the crawl.
- Moves per-issue conversion into a shared helper used by both crawl and reindex; unknown, unparseable, or out-of-scope ids yield a
ConnectorFailure.
Summary by CodeRabbit
-
New Features
- Added targeted Jira issue reindexing from issue URLs.
- Supports scoped and unrestricted Jira searches while preserving custom queries and project filters.
- Rebuilds Jira document content, hierarchy, and access permissions where available.
-
Bug Fixes
- Improved handling of invalid, missing, out-of-scope, and malformed Jira issues.
- Reports per-issue failures while preserving successful results.
- Handles batch-search fallback, credential errors, permissions, and rate limits more reliably.
-
Tests
- Added unit and live integration coverage for Jira reindexing scenarios.