docs: add a performance tuning page for deployment settings

Merged
FA
4mo ago
+107-0

Why

The configuration reference lists every setting with a one-line description, but nothing tells an operator which ones matter once an instance grows, or what changing one costs. Someone whose merges have become slow because a dozen branches are open, or whose queries round-trip heavily over a large dataset, has no page to land on.

Goal: one page under Deployment & Management naming the settings worth revisiting at scale — the symptom each addresses, and the trade-off for changing it.

Non-goals: no behavior or default changes. Scale-out levers (WEB_CONCURRENCY, task-worker replicas, INFRAHUB_BROKER_MAXIMUM_CONCURRENT_MESSAGES) and the DB retry/backoff settings were considered and deliberately left out of this pass.

What changed

New page docs/docs/deploy-manage/install-configure/performance-tuning.mdx — Tune performance — in the sidebar under Install & configure, between "Configure Infrahub" and the configuration reference link.

It opens with a symptom → setting → default lookup table, then one section per setting, each closing with a What you trade paragraph:

SectionSettingDefault
Diff updates after a mergeINFRAHUB_DIFF_UPDATE_AFTER_MERGEtrue
Query size on large datasetsINFRAHUB_DB_QUERY_SIZE_LIMIT5000
Concurrent queries against the databaseINFRAHUB_DB_MAX_CONCURRENT_QUERIES, _DELAY0 (unlimited), 0.01
Graph traversal timeoutsINFRAHUB_DB_PATH_TRAVERSAL_QUERY_TIMEOUT, INFRAHUB_DB_REACHABLE_NODES_QUERY_TIMEOUT30, 75
Hierarchy search depthINFRAHUB_DB_MAX_DEPTH_SEARCH_HIERARCHY5
Task list on a long task historyINFRAHUB_WORKFLOW_FLOW_RUN_COUNT_CACHE_THRESHOLD100000

Cross-links added from Configure Infrahub and Production deployment (Related resources sections).

What stayed the same: no code, config, defaults, or generated files touched. Documentation only.

Review feedback addressed

@ajtmccarty's review round, all resolved in db443e57a (branch squashed to a single commit):

  • A Branch cleanup after a merge section for INFRAHUB_DELETE_BRANCH_AFTER_MERGE was removed. Its premise — that a merged-but-undeleted branch still costs post-merge diff work — was wrong. merge_branch marks the merged branch's diff root is_merged = TRUE (core/diff/query/merge_tracking_id.py:23) and freezes it, rewriting tracking_id from branch.<name> to frozen.<name> (core/diff/query/freeze_by_branch.py:38-43), both at core/branch/tasks.py:495-496 — before the post-merge flow is submitted. The post-merge loop then skips it three ways over: get_roots_metadata defaults to exclude_merged=True, the loop requires not diff_root.is_frozen, and it requires a BranchTrackingId (a FrozenTrackingId fails that check). The #8507 fix in 1.8/1.9 already covers this, so the setting is branch lifecycle, not a performance lever, and doesn't belong on this page.
  • The diff-update opener was rewritten to state the cause rather than the diff-root mechanics: "Merging a branch changes the branch you merged into, which makes the stored diff of every other open branch out of date. Infrahub queues a diff update for each of them."

How to review

The page is 107 new lines of prose — read it end to end. Every mechanical claim was traced to source at this commit; these are the load-bearing ones if you want to spot-check:

  • Post-merge diff updates cover every other open branch → backend/infrahub/core/branch/tasks.py:668-685, with the merged branch excluded per the evidence in the section above
  • Diffs refresh on request rather than automatically → frontend/app/src/entities/diff/ui/diff-refresh-button.tsx; a proposed-change pipeline recomputes its own diff → backend/infrahub/proposed_change/tasks.py:395, :1556
  • Offset pagination loop → backend/infrahub/core/query/__init__.py:609-631; derived diff batch sizes → backend/infrahub/core/diff/calculator.py:144-146, backend/infrahub/core/diff/repository/repository.py:137,166
  • Connection-pool comparison and wait loop → backend/infrahub/database/__init__.py:338-344; metric name → backend/infrahub/database/metrics.py:20
  • Traversal timeout semantics → backend/infrahub/config.py:344-362
  • Hierarchy depth applied as double the path length → backend/infrahub/config.py:321, backend/infrahub/core/schema/relationship_schema.py:179, and the diff engine's use at backend/infrahub/core/diff/repository/repository.py:136,165
  • Flow-run count cached for one minute at or above the threshold → backend/infrahub/task_manager/task.py:88

The example values in the .env snippets are illustrative, not measured recommendations — the page says to change one setting at a time and measure. Flag any you'd rather see stated differently.

How to test

cd docs && npm run build     # onBrokenLinks / onBrokenAnchors / onBrokenMarkdownLinks are all "throw"
uv run invoke docs.lint      # markdownlint-cli2 + Vale

Both pass. Note for anyone reproducing locally: the build was run with Node 22 (Docusaurus requires ≥ 20), and Vale/markdownlint were invoked directly rather than through invoke — neither binary is on PATH in a fresh worktree.

Impact & rollout

  • Backward compatibility: documentation only, nothing to migrate
  • Config/env changes: none — the page documents settings that already exist
  • Deployment notes: safe to merge at any time

Open question for the reviewer

  • INFRAHUB_MISC_MAXIMUM_VALIDATOR_EXECUTION_TIME was left out. It is defined at backend/infrahub/config.py:820 and exported to the containers by both compose files, but nothing in this repository reads it. If Infrahub Enterprise consumes it, the page should carry a section for it with an Enterprise badge — please confirm which it is.

Checklist

  • Tests added/updated — n/a, documentation only
  • Changelog entry — n/a, matching recent docs-only PRs (#10071, #10072, #10073)
  • External docs updated
  • Internal .md docs updated — n/a
  • I have reviewed AI generated content