feat(backend): read the repository commit log live from a worker [IFC-3149]
Why
The commit-log query (InfrahubRepositoryCommits) was published in IFC-3146 but answers every git-derived field with UNAVAILABLE / NOT_IMPLEMENTED. A user still cannot see, from inside Infrahub, which commits a repository has, which one Infrahub has imported, or what is pending.
Goal: answer the commit log for real, read live from a task worker's existing local copy, without ever cloning, fetching or taking the repository lock on the read path.
Non-goals: the per-branch drift column still reports NOT_IMPLEMENTED (that is IFC-3147's branch-heads read); the Commits tab is IFC-3150; marking a deleted read-only ref on sync_status is a Phase 4 follow-up (T108/T102). No graph writes, no schema migration.
Part of the IFC-3101 commit-visibility feature (PR 5 in dev/specs/ifc-3101-repo-commit-visibility/tasks.md: T037-T048, T092, T093, T096).
Jira: IFC-3149
Depends on IFC-3246 (#10835), now merged into the epic branch and merged into this branch. The read resolves the remote head from the answering worker's own copy, so the commit view gives the same answer on every worker only when every worker's copy has been told to fetch. #10835 makes that broadcast independent of the worker that runs the refs check, and forces the fetch for a read-only repository so a moved tag reaches every worker.
What changed
Behavioral changes
InfrahubRepositoryCommitsreturns a paged, newest-first commit log from a worker's clone, with every commit labelledHEAD,IMPORTED,PENDING,HISTORYorUNRELATED, and the branch's condition:IN_SYNC,BEHIND(with a pending count when selected),REWRITTEN,ORPHANED,NO_REMOTE,NOT_TRACKED, or the newREF_MISSING.REF_MISSING(newRepositoryGitConditionmember, additive): a read-only repository whose tracked branch or tag was deleted upstream is reported as an error to act on rather than a neutral state, and still lists the history Infrahub is running. A bare ref is accepted only as a commit hash, so a stale local branch can no longer stand in for a deleted remote one.- A worker with no local copy answers
UNAVAILABLE / NOT_CLONEDimmediately and claims a single warm-up for the pool. The warm-up clones on whichever worker runs it, resets that copy to the imported commit, and broadcasts a pinned fetch so every other worker converges. A read-only branch with nothing imported gets a new clone-onlyrefresh.git.clonebroadcast; a read-write repository with nothing imported is left to its sync, which creates the copy along with its first import. checked_atis resolved from the refs-check cache key for read-only repositories (best-effort: an unreachable cache reads as "never checked").WORKER_TIMEOUT(from IFC-3148) now surfaces on the commit log when no worker answers.
Implementation notes
git/state/grows the pureclassification.py, the worker-sidelog_reader.py(the only module that touches a clone),bus_reader.py(the only module that knows the routing key),commit_log_wire.py(both-way conversion between the bus messages and the domain dataclasses) andwarm_up.py. The factory now returns the bus-backed reader; no resolver change was needed for the swap.- The read runs in
asyncio.to_thread. UnderBEHIND, a singlerev-list imported..headplaces every listed commit and gives the pending count, so labels follow ancestry, not list position (covered by a real merge-history test). - Ordering that matters: an unresolvable imported hash is decided as
ORPHANEDbefore any ancestry call, becauseis_ancestorraises on it rather than answering. - The warm-up reads the imported commit only once it holds the repository lock, so a sync that finished while it waited cannot be rolled back; it also resets its own copy, so that worker holds the commit the broadcast pins every other worker to rather than a remote head Infrahub has not imported. It fetches through the shared
fetch_from_originfrom #10835. It is the one caller that holds the repository lock while initializing a copy, which takes the same lock again; the warm-up tests run on the global lock registry so they cover that.
What stayed the same
- No graph schema change and no write to
commitorref. The GraphQL change is one additive enum member. - The drift resolver and the refs check from IFC-3147 / the refs-check PR are unchanged apart from
checked_at.
Suggested review order
backend/infrahub/git/state/classification.pyand its unit tests: the decision rules.backend/infrahub/git/state/log_reader.py: the worker read, warm-up claim and error handling.backend/infrahub/git/state/warm_up.pyandmessage_bus/operations/git/repository.py::clone: convergence.bus_reader.py,commit_log_wire.py, the handler, the resolver'schecked_at.- Tests, then the spec set and the regenerated artefacts.
How to review
Focus on the warm-up's interaction with the sync, which since #10881 also imports a branch whose commit in the graph differs from the remote head, and on REF_MISSING precedence in classify.
Mechanical, skim only: schema/schema.graphql, frontend/app/src/shared/api/graphql/generated/*, docs/docs/reference/message-bus-events.mdx are generated.
Known gaps, recorded as follow-ups rather than fixed here (tasks.md T103-T109):
- T103, the rebuild of a local copy during a read. A clone in progress is recognised only while the 60-second warm-up claim lasts, and clones started by a broadcast hold no claim of their own, so a large clone can turn a read into an RPC error. The same rebuild can delete the copy under an open read, and only
GitCommandErroris converted to a message without paths, so anOSErrorraised mid-read would reach the client with the clone's path. Both are fixed by cloning into a temporary directory and renaming it into place, which is also what makes the wider error conversion testable: today no test can delete the copy mid-read without a test hook in production code. - T108, a deleted read-only ref. The refs check skips a ref missing from the remote, so it never announces the deletion, and each worker reports
REF_MISSINGonly once one of its own fetches has prunedorigin/<ref>. - T105, the warm-up holds the global repository lock across the clone, as the sync bootstrap does.
- T106, the
BEHINDwalk is not bounded by the page. - T107, the read-only "import last commit" action can mark a commit imported without importing it when the local copy is already at the head, a pre-existing issue.
- T109, a read-write sync whose branch import fails sends no broadcast, so workers disagree on
BEHINDuntil the import succeeds. Pre-existing in the sync, made visible by the commit view.
Local cubic review findings declined: backend/tests/unit/git/test_tasks.py checks that the warm-up flow tags its run without the namespace by parsing the flow's source with ast and inspect.getsource, which is what .agents/rules/testing-python.md asks wiring tests to do.
How to test
# Classification, wire conversion, bus reader (no database)
uv run pytest backend/tests/unit/git/state -v
# The worker read against real clones: behind, rewritten, orphaned, REF_MISSING, tags, commit-hash refs,
# merge history, paging, warm-up claim and release, lock independence
INFRAHUB_USE_TEST_CONTAINERS=true uv run pytest backend/tests/component/message_bus/operations/git -n 0 -v
# Resolver, permissions, checked_at
INFRAHUB_USE_TEST_CONTAINERS=true uv run pytest backend/tests/component/graphql/queries/test_repository_git_state.py backend/tests/component/graphql/auth/test_repository_git_state_permissions.py -n 0 -v
# End to end through the real handler, the warm-up against a live API, and the refs check
INFRAHUB_USE_TEST_CONTAINERS=true uv run pytest backend/tests/integration/git/test_repository_commits_query.py backend/tests/integration/git/test_repository_warm_up.py backend/tests/integration/git/test_readonly_refs_check.py -n 0 -v
Locally, after merging the epic branch with #10835, #10881 and the SDK's WORKER_TIMEOUT binding: the full backend unit suite (3274), 266 component tests under git, message_bus/operations/git and the git-state GraphQL suites, and 14 integration tests pass; ruff, mypy, ty, backend.validate-generated and docs.validate are clean.
Impact & rollout
- Backward compatibility:
REF_MISSINGis a new member of a published enum; consumers already handle the condition as a string. No other contract change. - Performance: a commit-log read costs one RPC and a bounded walk on one worker; under
BEHINDit also walks the pending range once (T106). - Config/env changes: none.
- Deployment notes: adds the
git.commit_log.getrequest/reply pair and therefresh.git.clonebroadcast; both ride existing bindings. Safe to deploy with the rest of the epic branch.
Checklist
- Tests added/updated
- Changelog entry added (
+ifc-3101-repository-commits-api.added.mdupdated) - External docs updated (if user-facing or ops-facing change) - deferred to T088 with the Commits tab
- Internal .md docs updated (spec set: FR-029/FR-030, data model, contracts, tasks)
- I have reviewed AI generated content